kubernetes调试利器——kubectl debug工具
通常情况下，业务容器所使用的镜像是非常精简的，而一旦业务容器出现问题，通过kubectl exec进入到容器时，我们会发现自己需要使用的工具都没有，也无法通过apt, apt-get, yum等包管理工具下载需要的工具。
想要解决这个尴尬的窘境，有两种手段，其一是提前把需要使用的工具打入到镜像当中，除了问题我们可以随时进行debug。其二是利用kubectl debug工具。显然，第一种方式有很多弊端，譬如业务容器镜像过大，占用磁盘空间更多；另外每个人使用的工具可能会不同，我们不可能把所有的工具都打入到镜像当中，这是极不合理的。 而如果我们能够把需要的工具打入到一个debug镜像当中，需要的时候如果能把这个debug镜像跑起来，并且attach到我们需要排查问题的业务容器上，同时这两个容器可以共享network, pid名称空间的话，就能很好的解决这个问题。而恰好kubectl debug就有这样的功能
kubectl debug命令的帮助文档如下
root@k8s-master1:~# kubectl debug --help
Debug cluster resources using interactive debugging containers.
'debug' provides automation for common debugging tasks for cluster objects identified by resource and name. Pods will
be used by default if no resource is specified.
The action taken by 'debug' varies depending on what resource is specified. Supported actions include:
* Workload: Create a copy of an existing pod with certain attributes changed, for example changing the image tag to a
new version.
* Workload: Add an ephemeral container to an already running pod, for example to add debugging utilities without
restarting the pod.
* Node: Create a new pod that runs in the node's host namespaces and can access the node's filesystem.
Examples:
# Create an interactive debugging session in pod mypod and immediately attach to it.
kubectl debug mypod -it --image=busybox
# Create an interactive debugging session for the pod in the file pod.yaml and immediately attach to it.
# (requires the EphemeralContainers feature to be enabled in the cluster)
kubectl debug -f pod.yaml -it --image=busybox
# Create a debug container named debugger using a custom automated debugging image.
kubectl debug --image=myproj/debug-tools -c debugger mypod
# Create a copy of mypod adding a debug container and attach to it
kubectl debug mypod -it --image=busybox --copy-to=my-debugger
# Create a copy of mypod changing the command of mycontainer
kubectl debug mypod -it --copy-to=my-debugger --container=mycontainer -- sh
# Create a copy of mypod changing all container images to busybox
kubectl debug mypod --copy-to=my-debugger --set-image=*=busybox
# Create a copy of mypod adding a debug container and changing container images
kubectl debug mypod -it --copy-to=my-debugger --image=debian --set-image=app=app:debug,sidecar=sidecar:debug
# Create an interactive debugging session on a node and immediately attach to it.
# The container will run in the host namespaces and the host's filesystem will be mounted at /host
kubectl debug node/mynode -it --image=busybox
Options:
--arguments-only=false:
If specified, everything after -- will be passed to the new container as Args instead of Command.
--attach=false:
If true, wait for the container to start running, and then attach as if 'kubectl attach ...' were called.
Default false, unless '-i/--stdin' is set, in which case the default is true.
-c, --container='':
Container name to use for debug container.
--copy-to='':
Create a copy of the target Pod with this name.
--env=[]:
Environment variables to set in the container.
-f, --filename=[]:
identifying the resource to debug
--image='':
Container image to use for debug container.
--image-pull-policy='':
The image pull policy for the container. If left empty, this value will not be specified by the client and
defaulted by the server.
--profile='legacy':
Debugging profile. Options are "legacy", "general", "baseline", "netadmin", or "restricted".
-q, --quiet=false:
If true, suppress informational messages.
--replace=false:
When used with '--copy-to', delete the original Pod.
--same-node=false:
When used with '--copy-to', schedule the copy of target Pod on the same node.
--set-image=[]:
When used with '--copy-to', a list of name=image pairs for changing container images, similar to how 'kubectl
set image' works.
--share-processes=true:
When used with '--copy-to', enable process namespace sharing in the copy.
-i, --stdin=false:
Keep stdin open on the container(s) in the pod, even if nothing is attached.
--target='':
When using an ephemeral container, target processes in this container name.
-t, --tty=false:
Allocate a TTY for the debugging container.
Usage:
kubectl debug (POD | TYPE[[.VERSION].GROUP]/NAME) [ -- COMMAND [args...] ] [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).
想要完成上述功能，主要是利用--target参数，这个参数主要用于指定debug Pod中的哪个容器;--image参数就是用于指定使用哪个镜像来debug，这个镜像包含我们需要使用的工具即可。用法如下：
root@k8s-master1:~# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-5578b9475d-5fc8d 1/1 Running 1 (130m ago) 156m
nginx-deployment-775b6549b5-bgdfp 1/1 Running 2 (130m ago) 3h39m
nginx-deployment-775b6549b5-ghz9t 1/1 Running 2 (130m ago) 3h39m
nginx-deployment-775b6549b5-pcw82 1/1 Running 1 (130m ago) 167m
root@k8s-master1:~#
root@k8s-master1:~# kubectl debug mysql-5578b9475d-5fc8d --image=ubuntu:20.04 -it --target=mysql
Targeting container "mysql". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-jlw5l.
If you don't see a command prompt, try pressing enter.
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
999 1 0.0 4.8 1163028 195532 ? Ssl 03:26 0:07 mysqld
root 128 0.0 0.0 4248 3256 pts/0 Ss 05:40 0:00 bash
root 136 0.0 0.0 5900 2780 pts/0 R+ 05:41 0:00 ps -aux
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 0 1111892 2120 1823828 0 0 42 346 696 1192 1 4 95 0 0
root@mysql-5578b9475d-5fc8d:/# pidstt