[Google-oriented Programming(GoP)] FAQs about Docker Containers or Kubernetes (K8S) Pods Containers

FAQs about Docker Containers or Kubernetes (K8S) Pods

Exit Code: 137

A Pod is terminated or restarted with exit code 137. It should happen Out Of Memory(OOM).

Run kubectl describe pod command to check pod state.

1
2
3
4
5
6
7
8
9
10
11
12
# kubectl describe pod <name>
...
State: Running
Started: Feb, 03 Sep 2020 02:46:17 +0800
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Sat, 01 Feb 2020 11:32:03 +0800
Finished: Mon, 03 Feb 2020 02:46:12 +0800
Ready: True
Restart Count: 1
...

Run journalctl command to check journal logs.

1
2
3
4
5
6
7
8
9
10
11
# journalctl -k | grep -i -e memory -e oom
Feb 03 02:46:14 s1 kernel: [<ffffffff81bbb076>] out_of_memory+0x4b6/0x4f0
Feb 03 02:46:14 s1 kernel: [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
Feb 03 02:46:15 s1 kernel: Out of memory: Kill process 22739 (bundle) score 1325 or sacrifice child
Feb 03 02:46:15 s1 kernel: kthreadd invoked oom-killer: gfp_mask=0x3000d0, order=2, oom_score_adj=0
Feb 03 02:46:15 s1 kernel: [<ffffffffc03cb71a>] ? virtballoon_oom_notify+0x2a/0x70 [virtio_balloon]
Feb 03 02:46:15 s1 kernel: [<ffffffff81bba834>] oom_kill_process+0x254/0x3d0
Feb 03 02:46:15 s1 kernel: [<ffffffff81bba2dd>] ? oom_unkillable_task+0xcd/0x120
Feb 03 02:46:15 s1 kernel: [<ffffffff81bbb076>] out_of_memory+0x4b6/0x4f0
Feb 03 02:46:15 s1 kernel: [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
Feb 03 02:46:15 s1 kernel: Out of memory: Kill process 23056 (util.rb:22) score 1326 or sacrifice child