k8s监控 heapster部署实践

更多kubernetes文章： k8s专栏目录

使用  https://github.com/kubernetes/heapster github上的配置启动heapster

目录位置 deploy/kube-config/influxdb/，创建里面3个文件

再创建 deploy/kube-config/rbac/heapster-rbac.yaml

安装失败时

[root@tensorflow1 ~]# kubectl top node
error: metrics not available yet

安装成功时

[root@tensorflow1 rbac]# kubectl top node
NAME          CPU(cores)   CPU%      MEMORY(bytes)   MEMORY%   
tensorflow0   1046m        26%       1759Mi          22%       
tensorflow1   211m         5%        3237Mi          42%       
tensorflow2   61m          1%        1406Mi          18%

注: influxdb 和 grafana 还有问题。只是启动了heapster

问题：

E1118 07:05:45.339132 1 reflector.go:203] k8s.io/heapster/metrics/sources/kubelet/kubelet.go:342: Failed to list *api.Node: Get https://kubernetes.default/api/v1/nodes?resourceVersion=0 : dial tcp: i/o timeout

kubernetes.default 访问不到，实际上是kube-dns出了问题，暂时还没解决。

这边先写死了ip:port来访问的，我的宿主机是192.168.1.138，配置的就是192.168.1.138:6443，就可以访问了。

修改heapster.yaml配置- --source=kubernetes: https://kubernetes.default  为 - --source=kubernetes:https://192.168.1.138:6443

参考  https://github.com/opsnull/follow-me-install-kubernetes-cluster/issues/173

解决 dns问题  目前还没解决

参考 https://github.com/kubernetes/heapster/issues/1658 

https://github.com/kubernetes/kubernetes/issues/45479

https://github.com/kubernetes/heapster/issues/1658

https://github.com/kubernetes/heapster/issues/1655

E0409 06:05:27.448890 1 reflector.go:190] k8s.io/heapster/metrics/util/util.go:30: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:heapster" cannot list nodes at the cluster scope

缺角色 无权访问

再创建 heapster/deploy/kube-config/rbac/heapster-rbac.yaml

就可以访问了

E0409 07:40:05.082253 1 influxdb.go:208] Failed to create influxdb: Retention Policy creation failed: json: cannot unmarshal number into Go value of type struct { Results []client.Result "json:\"results,omitempty\""; Err string "json:\"error,omitempty\"" }

暂时不知道什么问题

kube top 可以看node 和pod 的资源占用情况 需要预先安装heapster

[root@tensorflow1 ~]# kubectl top
Display Resource (CPU/Memory/Storage) usage.

The top command allows you to see the resource consumption for nodes or pods.

This command requires Heapster to be correctly configured and working on the server.

Available Commands:
  node        Display Resource (CPU/Memory/Storage) usage of nodes
  pod         Display Resource (CPU/Memory/Storage) usage of pods

Usage:
  kubectl top [options]

Use "kubectl  --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

参考   https://github.com/kubernetes/kubernetes/pull/58289

更多资料

kubernetes1.8版本heapster部署

http://blog.csdn.net/liukuan73/article/details/78704395