안녕하세요
일을 하다 보니...console이 필요한데... 클라우드에서 제공하는 console이 있는 경우도 있고..없는 경우도 있고 때로는 내가 원하는 환경을 잠시 쓰거나 또는 내가 원하는걸 쉽게 만들어서 쓰거나
등등등등....으로 이유로 만든 애들을 소개하면 좋을꺼 같아서 정리합니다.
vagrant를 통해서 virtualbox에 프로비저닝하도록 되어 있습니다.
https://github.com/sysnet4admin/IaC/tree/master/k8s/k8s-console
쿠버네티스의 kubeconfig만 가지고 오면 가볍게 사용할 수 있도록 하는 콘솔
이에 따라 다음의 내용들이 적용되어 있음
안녕하세요
gcpdiag라는게 새로 나와서 좀 테스트를 해 봤는데요.
결론부터 얘기하자면…뭐랄까…aquasecurity/kube-bench 도구 같은 느낌이네요?
그리고 좀 단점부터 보이는게…컨테이너 베이스로 동작을 시키다 보니…
linux 커널이 필요한거 같다? 이고…근데 왜 이걸 오픈소스화 했을까…?
어짜피 GCP에서만 쓸꺼 같은 이름인데…
어쨌든 GCP를 쓰고, 내부적으로 audit을 전체적으로 하고 싶다 하면 써볼만 할꺼 같기도 합니다.
실행 내용과 결과는 다음과 같습니다.
[gcpdiag run]
[hj@cs-491314827780-default ~ (☸️ |hj-gke:default)]$ gcpdiag
Unable to find image 'us-docker.pkg.dev/gcpdiag-dist/release/gcpdiag:0.55' locally
0.55: Pulling from gcpdiag-dist/release/gcpdiag
1fe172e4850f: Pull complete
caf521ccaac6: Pull complete
3ead6fa29328: Pull complete
5c2a1cbceb83: Pull complete
a8d5f1318db7: Pull complete
358ca086baa3: Pull complete
30197a52639a: Pull complete
65545a04dace: Pull complete
80cf14a4e373: Pull complete
00f344331f1a: Pull complete
5fbfd71d5b4d: Pull complete
Digest: sha256:0b5fcc0fd3e2f1b822cec492b0128f7e1df5173c19990570ee072c80cf6164c4
Status: Downloaded newer image for us-docker.pkg.dev/gcpdiag-dist/release/gcpdiag:0.55
gcpdiag 🩺 - Diagnostics for Google Cloud Platform
Usage:
gcpdiag COMMAND [OPTIONS]
Commands:
help Print this help text.
lint Run diagnostics on GCP projects.
version Print gcpdiag version.
See: gcpdiag COMMAND --help for command-specific usage.
실행 결과 실패한 내용을 rough하게 살펴보면…
뭐 이정도면…한번쯤 cloud shell에서 돌려봐도 무방할꺼 같기도 하네요.
[gcpdiag --project]
[hj@cs-491314827780-default ~ (☸️ |hj-gke:default)]$ gcpdiag lint --project=hj-int-20200908
gcpdiag
Starting lint inspection (project: hj-int-20200908)...
🔎 gce/BP/2021_001: Serial port logging is enabled.
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [FAIL]
- hj-int-20200908/windows [FAIL]
- hj-int-20200908/zipkin-vm [FAIL]
Serial port output can be often useful for troubleshooting, and enabling
serial logging makes sure that you don't lose the information when the VM is
restarted. Additionally, serial port logs are timestamped, which is useful to
determine when a particular serial output line was printed.
https://gcpdiag.dev/rules/gce/BP/2021_001
🔎 gce/BP/2021_002: GCE nodes have an up to date ops agent installed.
- hj-int-20200908/gui-pipe [FAIL] not installed
- hj-int-20200908/windows [FAIL] not installed
- hj-int-20200908/zipkin-vm [FAIL] not installed
Verify that the ops agent is used by the GCE instances and that the agent is
recent enough. If the monitoring agent is found it is recommended to upgrade
to the ops agent. see:
https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent
https://gcpdiag.dev/rules/gce/BP/2021_002
🔎 gce/ERR/2021_003: Google APIs service agent has the Editor role.
- hj-int-20200908 [ OK ]
🔎 gce/ERR/2021_004: Serial logs don't contain Secure Boot error messages
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/ERR/2021_005: Serial logs don't contain mount error messages
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_001: GCE instance service account permissions for logging.
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_003: GCE instance service account permissions for monitoring.
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_004: Serial logs don't contain disk full messages
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_005: Serial logs don't contain out-of-memory messages
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_006: Serial logs don't contain "Kernel panic" messages
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2021_007: Serial logs don't contain "BSOD" messages
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2022_001: GCE connectivity: IAP service can connect to SSH/RDP port on instances.
- hj-int-20200908/gke-gke-nodes-634bd260-gefn [ OK ]
- hj-int-20200908/gui-pipe [ OK ]
- hj-int-20200908/windows [ OK ]
- hj-int-20200908/zipkin-vm [ OK ]
🔎 gce/WARN/2022_002: Instance groups named ports are using unique names.
- hj-int-20200908/gke-gke-nodes-634bd260-grp [ OK ]
🔎 gce/WARN/2022_003: GCE VM instances quota is not near the limit.
- projects/hj-int-20200908/regions/asia-northeast3 [ OK ]
- projects/hj-int-20200908/regions/us-central1 [ OK ]
🔎 gcs/BP/2022_001: Buckets are using uniform access
- hj-int-20200908/artifacts.hj-int-20200908.appspot.com [FAIL]
it is recommend to use uniform access on your bucket
- hj-int-20200908/hoon [FAIL]
it is recommend to use uniform access on your bucket
Google recommends using uniform access for a Cloud Storage bucket IAM policy
https://cloud.google.com/storage/docs/access-
control#choose_between_uniform_and_fine-grained_access
https://gcpdiag.dev/rules/gcs/BP/2022_001
🔎 gke/BP/2021_001: GKE system logging and monitoring enabled.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/BP/2022_001: GKE clusters are regional.
- hj-int-20200908/us-central1-c/gke [FAIL]
is not regional
The availability of regional clusters (both control plane and nodes) is
higher for regional clusters as they are replicated across zones in the
region. It is recommended to use regional clusters for the production
workload.
https://gcpdiag.dev/rules/gke/BP/2022_001
🔎 gke/BP/2022_002: GKE clusters are using unique subnets.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_001: GKE nodes service account permissions for logging.
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/ERR/2021_002: GKE nodes service account permissions for monitoring.
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/ERR/2021_004: GKE nodes aren't reporting connection issues to apiserver.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_005: GKE nodes aren't reporting connection issues to storage.google.com.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_006: GKE Autoscaler isn't reporting scaleup failures.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_007: GKE service account permissions.
- hj-int-20200908 [ OK ]
🔎 gke/ERR/2021_008: Google APIs service agent has Editor role.
- hj-int-20200908 [ OK ]
🔎 gke/ERR/2021_009: Version skew between cluster and node pool.
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/ERR/2021_010: Check internal peering forwarding limits which affect GKE.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_011: ip-masq-agent not reporting errors
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_012: Node pool service account exists and not is disabled.
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/ERR/2021_013: GKE cluster firewall rules are configured.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2021_015: GKE connectivity: node to pod communication.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2022_001: GKE connectivity: pod to pod communication.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/ERR/2022_002: GKE nodes of private clusters can access Google APIs and services.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/SEC/2021_001: GKE nodes don't use the GCE default service account.
- hj-int-20200908/us-central1-c/gke/nodes [FAIL]
node pool uses the GCE default service account
The GCE default service account has more permissions than are required to run
your Kubernetes Engine cluster. You should either use GKE Workload Identity
or create and use a minimally privileged service account.
https://gcpdiag.dev/rules/gke/SEC/2021_001
🔎 gke/WARN/2021_003: GKE cluster size close to maximum allowed by pod range
- hj-int-20200908/us-central1-c/gke [ OK ] 1/1024 nodes used.
🔎 gke/WARN/2021_004: GKE system workloads are running stable.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2021_005: GKE nodes have good disk performance.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2021_006: GKE nodes aren't reporting conntrack issues.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2021_007: GKE nodes have enough free space on the boot disk.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2021_009: GKE nodes use a containerd image.
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/WARN/2022_001: GKE clusters with workload identity are regional.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2022_002: GKE metadata concealment is not in use
- hj-int-20200908/us-central1-c/gke/nodes [ OK ]
🔎 gke/WARN/2022_003: GKE service account permissions to manage project VPC firewall rules.
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 gke/WARN/2022_004: Cloud Logging API enabled when GKE logging is enabled
- hj-int-20200908/us-central1-c/gke [ OK ]
🔎 iam/SEC/2021_001: No service accounts have the Owner role
- hj-int-20200908 [ OK ]
Rules summary: 24 skipped, 40 ok, 5 failed
