오랜만입니다.
이번에 다룰 주제는?
kubespray의 최소 메모리 양은? 입니다.
짝짝짝....
이걸 굳이 왜 다루게 되었냐면...
홈페이지에 요래요래 나와 있는데요..(https://github.com/kubernetes-sigs/kubespray#requirements)
Hardware: These limits are safe guarded by Kubespray. Actual requirements for your workload can differ. For a sizing guide go to the Building Large Clusters guide.
- Master
- Memory: 1500 MB
- Node
- Memory: 1024 MB
막상 해 보면 에러를 맞게 되니....-_- 그것도 테스트도 엄청 오래걸립니다.
결과부터 얘기하면,
안전한 사정권은
Master에 2048MB 그리고 Worker Node에는 1280MB 정도입니다.
>>>추가
소스를 그래서 좀 열어서 봤는데요 (앤서블 소스는 쉬우니까요..-_-)
일단 에러는 이쪽에서 체크해서 넘기고요
[root@m11-k8s ~]# grep -r "Stop if memory is too small for masters" /kubespray/./kubespray/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml:- name: Stop if memory is too small for masters
그리고 체크는 여기서 ansible_memtotal_mb를 이용해서 하더라고요
[root@m11-k8s ~]# grep -r ansible_memtotal_mb ./kubespray/
./kubespray/roles/etcd/defaults/main.yml:etcd_memory_limit: "{% if ansible_memtotal_mb < 4096 %}512M{% else %}0{% endif %}"./kubespray/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml: that: ansible_memtotal_mb >= minimal_master_memory_mb./kubespray/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml: that: ansible_memtotal_mb >= minimal_node_memory_mb
그러면 느낌이... ansible_memtotal_mb 이 넘이 범인?
왜냐면 이렇게 체크하고 있거든요
[root@m11-k8s ~]# grep -r minimal_node_memory_mb ./kubespray/
./kubespray/roles/kubernetes/preinstall/defaults/main.yml:minimal_node_memory_mb: 1024./kubespray/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml: that: ansible_memtotal_mb >= minimal_node_memory_mb
[root@m11-k8s ~]# grep -r minimal_master_memory_mb ./kubespray/
./kubespray/roles/kubernetes/preinstall/defaults/main.yml:minimal_master_memory_mb: 1500./kubespray/roles/kubernetes/preinstall/tasks/0020-verify-settings.yml: that: ansible_memtotal_mb >= minimal_master_memory_mb그래서 실제로 얼마 정도로 체크하나 봤죠.
대충 local에 가 ansible_memtotal_mb 얼마나 용량으로 인자 값을 가지고 오나 보려고요
[root@m11-k8s ~]# ansible-playbook test.yaml [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source [WARNING]: No inventory was parsed, only implicit localhost is available [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' PLAY [localhost] ************************************************************************************************************** TASK [Gathering Facts] ******************************************************************************************************** ok: [localhost] TASK [debug] ****************************************************************************************************************** ok: [localhost] => { "msg": 1998 } PLAY RECAP ******************************************************************************************************************** localhost : ok=2 changed=0 unreachable=0 failed=0
찾았네요 문제? 라고 해야하나....
VM에서는 2048로 줬는데...1998로 인자 값을 물고 오네요...버그인가...고로 문제의 원인은...
ansible_memtotal_mb 입니다!!!
컨디션 마다 실행되는 에러는 다음과 같아요~!
시간을 아낄수 있으면 좋겠네요 :)
[ Master 1500MB, Worker 1024MB ]
TASK [kubernetes/preinstall : Stop if memory is too small for masters] **************************************************************************
skipping: [w101-k8s]
skipping: [w102-k8s]
fatal: [m11-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
skipping: [w103-k8s]
fatal: [m12-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
fatal: [m13-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
skipping: [w104-k8s]
skipping: [w105-k8s]
skipping: [w106-k8s]
[ Master 1536MB, Worker 1024MB ]
TASK [kubernetes/preinstall : Stop if memory is too small for masters] **************************************************************************
skipping: [w101-k8s]
skipping: [w102-k8s]
skipping: [w103-k8s]
skipping: [w104-k8s]
skipping: [w105-k8s]
skipping: [w106-k8s]
fatal: [m11-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
fatal: [m12-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
fatal: [m13-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_master_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
[ Master 2048MB, Worker 1024MB ]
TASK [kubernetes/preinstall : Stop if memory is too small for nodes] **************************************************************************
fatal: [w101-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
fatal: [w102-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
skipping: [m11-k8s]
fatal: [w103-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
skipping: [m12-k8s]
fatal: [w104-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
fatal: [w105-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
skipping: [m13-k8s]
fatal: [w106-k8s]: FAILED! => {
"assertion": "ansible_memtotal_mb >= minimal_node_memory_mb",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}
