K8S笔记

kubectl get po -n qa-test

kubectl get all -A

查看部署的环境和chart

[root@xff2 qa-test]# helm search repo cbg-helm-public-repo/llmserver-test

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

index.go:353: skipping loading invalid entry for chart "ioss-service-production" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production-eu1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production-ap1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production-ap1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-test" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-test" "V1.0.1201" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1201" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production-eu1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-develop" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-develop" "V1.0.1201" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1201" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-local-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

NAME CHART VERSION APP VERSION DESCRIPTION

cbg-helm-public-repo/llmserver-test 1.0.100000 1.16.0 xff llmserver

[root@xff2 qa-test]# ls -l

total 32

-rw-r--r-- 1 root root 17232 Oct 7 15:36 prompt_templates.sql

-rw-r--r-- 1 root root 8619 Oct 7 16:04 xff_ops.sql

[root@xff2 qa-test]# helm fetch cbg-helm-public-repo/llmserver-test

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

index.go:353: skipping loading invalid entry for chart "ioss-service-production-eu1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production-ap1" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

index.go:353: skipping loading invalid entry for chart "ioss-service-production" "V1.0.1200" from /root/.cache/helm/repository/cbg-helm-public-repo-index.yaml: validation: chart.metadata.version "V1.0.1200" is invalid

Error: failed to fetch https://artifacts-pub.iflytek.com:443/artifactory/api/helm/cbg-helm-public-repo/llmserver-test/llmserver-test-1.0.100000.tgz : 204 No Content

[root@xff2 qa-test]# ls -l

total 32

-rw-r--r-- 1 root root 17232 Oct 7 15:36 prompt_templates.sql

-rw-r--r-- 1 root root 8619 Oct 7 16:04 xff_ops.sql

[root@xff2 qa-test]# helm repo list

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

NAME URL

cbg-helm-public-repo https://artifacts-pub.iflytek.com/artifactory/cbg-helm-public-repo

cbg-helm-local https://artifacts.iflytek.com/artifactory/cbg-helm-local

[root@xff2 qa-test]# helm repo add cbg-helm-local https://artifacts.iflytek.com/artifactory/cbg-helm-local --username=cbg_push_helm --password=

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

Password:

"cbg-helm-local" already exists with the same configuration, skipping

[root@xff2 qa-test]# helm fetch --help

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

Retrieve a package from a package repository, and download it locally.

This is useful for fetching packages to inspect, modify, or repackage. It can

also be used to perform cryptographic verification of a chart without installing

the chart.

There are options for unpacking the chart after download. This will create a

directory for the chart and uncompress into that directory.

If the --verify flag is specified, the requested chart MUST have a provenance

file, and MUST pass the verification process. Failure in any part of this will

result in an error, and the chart will not be saved locally.

Usage:

helm pull [chart URL | repo/chartname] [...] [flags]

Aliases:

pull, fetch

Flags:

--ca-file string verify certificates of HTTPS-enabled servers using this CA bundle

--cert-file string identify HTTPS client using this SSL certificate file

-d, --destination string location to write the chart. If this and untardir are specified, untardir is appended to this (default ".")

--devel use development versions, too. Equivalent to version '>0.0.0-0'. If --version is set, this is ignored.

-h, --help help for pull

--insecure-skip-tls-verify skip tls certificate checks for the chart download

--key-file string identify HTTPS client using this SSL key file

--keyring string location of public keys used for verification (default "/root/.gnupg/pubring.gpg")

--pass-credentials pass credentials to all domains

--password string chart repository password where to locate the requested chart

--prov fetch the provenance file, but don't perform verification

--repo string chart repository url where to locate the requested chart

--untar if set to true, will untar the chart after downloading it

--untardir string if untar is specified, this flag specifies the name of the directory into which the chart is expanded (default ".")

--username string chart repository username where to locate the requested chart

--verify verify the package before using it

--version string specify a version constraint for the chart version to use. This constraint can be a specific tag (e.g. 1.1.1) or it may reference a valid range (e.g. ^2.0.0). If this is not specified, the latest version is used

Global Flags:

--burst-limit int client-side default throttling limit (default 100)

--debug enable verbose output

--kube-apiserver string the address and the port for the Kubernetes API server

--kube-as-group stringArray group to impersonate for the operation, this flag can be repeated to specify multiple groups.

--kube-as-user string username to impersonate for the operation

--kube-ca-file string the certificate authority file for the Kubernetes API server connection

--kube-context string name of the kubeconfig context to use

--kube-insecure-skip-tls-verify if true, the Kubernetes API server's certificate will not be checked for validity. This will make your HTTPS connections insecure

--kube-tls-server-name string server name to use for Kubernetes API server certificate validation. If it is not provided, the hostname used to contact the server is used

--kube-token string bearer token used for authentication

--kubeconfig string path to the kubeconfig file

-n, --namespace string namespace scope for this request

--registry-config string path to the registry config file (default "/root/.config/helm/registry/config.json")

--repository-cache string path to the file containing cached repository indexes (default "/root/.cache/helm/repository")

--repository-config string path to the file containing repository names and URLs (default "/root/.config/helm/repositories.yaml")

[root@xff2 qa-test]# helm fetch --repo cbg-helm-local https://artifacts.iflytek.com/artifactory/cbg-helm-local --username=cbg_push_helm --password=

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

Error: could not find protocol handler for:

[root@xff2 qa-test]# helm fetch --repo cbg-helm-public-repo/llmserver-test

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config

WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config

Error: "helm pull" requires at least 1 argument

Usage: helm pull [chart URL | repo/chartname] [...] [flags]

[root@xff2 qa-test]# ls -l

total 32

-rw-r--r-- 1 root root 17232 Oct 7 15:36 prompt_templates.sql

-rw-r--r-- 1 root root 8619 Oct 7 16:04 xff_ops.sql

[root@xff2 qa-test]#

kubectl get po -n qa-tes

K8S的配置文件

cd /deploy

查看namespace下的应用

helm list -n qa-test

查看映射的端口

kubectl get service -n qa-test

查看pod在哪台节点上

kubectl get pod -n qa-test -o wide

查看部署的环境和chart

切换到valuse的同级目录进行安装

helm install connectserver ./ -n qa-test

查看是否运行起来

kubectl get po -n qa-test

查看问题的日志

kubectl describe connectserver-connect-server-integration-5b4b8b7fb-dshmj -n qa-test

config.in 集成测试环境

config.perf 压测环境

如何确保k8s工作正常

node是如何和master通信

ps -ef|grep kubelet

进入指定的mysql数据库

mysql -h 172.16.82.66 -P 3306 -u root -p

jxma7@jxma7-PC MINGW64 ~

$ kubectl get configmap connectserver-configmap -o yaml -n qa-test

Error from server (Forbidden): configmaps "connectserver-configmap" is forbidden: User "public-prov3" cannot get resource "configmaps" in API group "" in the namespace "qa-test"

jxma7@jxma7-PC MINGW64 ~

$ kubectl --kubeconfig D:/k8s_test/config get configmap connectserver-configmap -o yaml -n qa-test

apiVersion: v1

data:

cert.pem: |

-----BEGIN CERTIFICATE-----

MIIEODCCAqCgAwIBAgIPe5KTz0fnBqfGteH6sKu7MA0GCSqGSIb3DQEBCwUAMHkx

HjAcBgNVBAoTFW1rY2VydCBkZXZlbG9wbWVudCBDQTEnMCUGA1UECwwec2NndW9A

c2NndW9zLU1hY0Jvb2stUHJvLmxvY2FsMS4wLAYDVQQDDCVta2NlcnQgc2NndW9A

c2NndW9zLU1hY0Jvb2stUHJvLmxvY2FsMB4XDTIzMDgxNjAxNTQ0MFoXDTI1MTEx

NjAxNTQ0MFowUjEnMCUGA1UEChMebWtjZXJ0IGRldmVsb3BtZW50IGNlcnRpZmlj

YXRlMScwJQYDVQQLDB5zY2d1b0BzY2d1b3MtTWFjQm9vay1Qcm8ubG9jYWwwggEi

MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCu4Vjfmr7kNWb9pxXLklJof8Fi

8386oppgjUoNbAimPG0d7z92LAWIBGfzyembvzugJ6SOndz2lBxj4VlsaZSo5lqz

m+Tt2rkVJXeA5ZUrZwm9OHnU5kLayGb64463vgtHSWdHFn8IX9GDBoSyM8QD5Mlk

xARHaxGQ84I3azpOFtjhMhzYJIJaEpNqjd4U6giTYHTNqBn4JnScbYSfLa+08HYR

g77WmH6PbEQok+GSesNwtxfbxJfPFLXI2+G5Hf64jvFbVwBVhRTnwNNtvJCbmNFE

RYbMQLJKWCmudRJkzy5FZQnzvZtnijRVw6fqlgveRFYl+dB0wbXCQ07Gg0yhAgMB

AAGjZDBiMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAfBgNV

HSMEGDAWgBQDdmP12m/cAC439Ypbk+7SPJjwLjAaBgNVHREEEzARgglsb2NhbGhv

c3SHBH8AAAEwDQYJKoZIhvcNAQELBQADggGBAISdGeFok1542F/k0a3AJe4BzyVF

tRRuFbH0+t8BLSF7xWTGU2peWkgGKtdUWAbkCCuEn8V5AVgAs209fxJ/H7YOLkdx

GrZkldpV4zEXjiXfODWaEGsScqFNENAsnKheGl4il3dtHPEPTp1+zaQRSF1rG7iX

o9l7Av8TJ2TNMVoLo9RVM+QJPTQ6edIPUMckXrcLCHDotzZzgUVXX8EVfDZ8twB5

vz8tpvGv5mZpnhXx44YwFP1NzLwTX9IGFI4sXJX5gdZwd4WPqqXEypsQjpJkF7ug

UGEgYKnzbq1eFffcH1cgZfP1vVST/9typXymfBJvAtAkeR16E3OzSoYrPpOKwb/R

6XRBauwq2czfrq10ce1AydfvJv4grcElVC8lxPWM6DF4wiQSpLndKpx3shXrxl4w

VnGWnXxzJ7bGkiEw4AvolaVK24OGAWZL51LlDrOGe3PAl27XdaGuADEOrjDTQttz

H27JWbmUWn8gOfUzFCnSsMnbZzL2J3Kq8ytE+g==

-----END CERTIFICATE-----

key.pem: |

-----BEGIN PRIVATE KEY-----

MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCu4Vjfmr7kNWb9

pxXLklJof8Fi8386oppgjUoNbAimPG0d7z92LAWIBGfzyembvzugJ6SOndz2lBxj

4VlsaZSo5lqzm+Tt2rkVJXeA5ZUrZwm9OHnU5kLayGb64463vgtHSWdHFn8IX9GD

BoSyM8QD5MlkxARHaxGQ84I3azpOFtjhMhzYJIJaEpNqjd4U6giTYHTNqBn4JnSc

bYSfLa+08HYRg77WmH6PbEQok+GSesNwtxfbxJfPFLXI2+G5Hf64jvFbVwBVhRTn

wNNtvJCbmNFERYbMQLJKWCmudRJkzy5FZQnzvZtnijRVw6fqlgveRFYl+dB0wbXC

Q07Gg0yhAgMBAAECggEAbyG3J4mgC0fAZiq61/3pLo0ZKLeSP4G7VuYSjAr+SB2Y

PfCexIrfDPu3z52dPaM90d2/Xf4FwBCJJhO7jwBdPyP7ua20wk4/VafUlG3Q6J72

+m8cROX60CuP/ZmERVGYuzlzTyKFoZ26d2g6BUCOLOdD8Cv8iCILLLECLBORHrOr

IERgPPLAi79QHAV154Y1ZZlG4aAUT6ayLwaXe1VEoamQUYmB3mmJO4VkScSyYHLX

mawQwJyKFakd5nBIUYIcUivL3E5zUtkDgALCzTAGPUsPTRX41IARxa8+v+Bh3UWp

tU8PCI5dxpRZGg6CP/U3m5cStfCcNAKXcfThRwJTUQKBgQDZ1Yo1s81XNr8AA1Bv

MvyNGrCyjem0/H19pdoyS90IPvjTRPqOtamZG5dsFRbWzk4Bw0BW9sKMG7uqB0rf

jSf0iHNO+v7Xq3AYFtcTS9Fnkzlkx1TvjRRmxV/wOlzXdjBPcrT4O2M0yeGOTivp

ri2swJUZ/ELZd1VzCpEU4HTgrQKBgQDNhTUvsq8x+4nXbdHedHzPYkufcJAKg711

oMD6Rxck2hSpZm5H0Cqj5TnCRzodG5asax9XieWWYSzBCYCkChOk8lyR6H18L8f6

H6KFeRIH2Ys3gDh3kA+LVPfm8WFGgkQTG9TIVasW1FQct7p7x2kt8SI8TFBKz7jR

RevTzJ92RQKBgGhAh43eCGc1/0E9VNBzQWUyoeBChNgmgtSQfJXXsugQYzuRZqzI

CmwJRXXnve99+t1jGbbFECTH3Q+pv86vmrCwYScqfq6vEM2mmLZWcdi1ICG4OFER

XvM0gQ7Sj75ahRJw3mVy3q7T84wMo32J67gizvkJvuCe+Rk+tGKebbFdAoGAc45v

jkcozLDDG1cLRso2lyCMprRSuSbDs0zU1KR+NeFM9sa6QTShlQ/A/ySvX0hw2rdM

oxrM9VXkJRHju9hSnU8Mj5mqXNY82b1J00R0x+cgqO82ULu2kcAPt1GHQW7jOd9P

L+lDTo3KUW9Yslo4IHXRqxe5mabcIwbuh2MpUgUCgYAOyTO30k8gylXedcWg0mQ5

yRLRSSJF75DjGmvnAqMkQPg+qrxvEkpsy2n4b51Rs2Ldfw1A8PvMQah+M25BAMhT

GrWwNYDGOKe56kLlm3H+xX5yhnA7s1ePGGTobZKNKIi4pn96mPEm3/taIR/DMWvg

sZUIxuEbi2/4rMiE0enZuw==

-----END PRIVATE KEY-----

services.yaml: "GrpcPort: 8080\r\nHttpPort: 8090\r\nWSPort: 8091\r\nAPIPort: 8070\r\nMetricPort:

8060\r\nAgentOpsPort: 8050\r\n\r\nMaxCurrency: 10000\r\nNodeIntervalSecond: 10\r\nNodeRegUrl:

http://connectmanage-connectmanage-perf:31180/connect/register\r\nClientRegUrl:

http://clientmanage-clientmanage-perf:8081/client/register\r\nClientUnRegUrl:

http://clientmanage-clientmanage-perf:8081/client/offline\r\n\r\nServiceLocalPort:

32180\r\nExportInfos:\r\n - LocalIP: 172.16.82.62\r\n ExportIP: 172.16.82.62\r\n

\ ExportPort: 32180\r\n - LocalIP: 172.16.82.63\r\n ExportIP: 172.16.82.63\r\n

\ ExportPort: 32180\r\n\r\nUPStreamTimeout: 3000\r\n\r\nApiRouter:\r\n - Name:

default\r\n Url: http://businessserver-businessserver-perf:9007\r\n - Name:

/recommend/get\r\n Url: http://recommend-recommend-integration:7281\r\n -

Name: /mpOperator\r\n Url: http://operatorserver-operatorserver-perf:5090\r\n\r\nLogLevel:

DEBUG\r\nLogOut: file\r\nLogFile: ./logs/ConnectServer.log\r\nLogMaxDays: 30\r\n\r\nCpName:

ConnectServer\r\n#LogType: log-query-service\r\nLogType: xff-server\r\nIsRemoteBizLog:

false\r\nIsLocalBizLog: true\r\nAsyncChainSize: 20000\r\nDefaultAppID: CB2W28VA\r\nDefaultProductLine:

com\r\nDefaultBizID: com\r\nKafkaClientExt:\r\n bootstrap.servers: \"h0082116:9010,h0082116:9020\"\r\n

\ acks: \"all\"\r\n\r\nAppID: 84HYCXBT\r\nModelID: Web\r\n#test\r\nAccountUrl:

https://accounttest.xfinfr.com/session/check\r\nAccessKey: id77m87pebncdppm\r\nAccessSecret:

3f6khye546nbzmacjsnf5trktes6mat2\r\n#live\r\n#AccountUrl: https://account.xfinfr.com/session/check\r\n#AccessKey:

KFNfe54fJYFJNgct\r\n#AccessSecret: Kf6FJlxavX6nx2ZwYu2dyUhuxDDqmOSw\r\n\r\nWitheApiRouter:\r\n

\ - /subscribe/getByLabel\r\n - /message/welcome\r\n - /mpOperator/userApply\r\n

\ - /mpOperator/userApply/status\r\n - /mpOperator/get_water_mark\r\n - /mpOperator/get_user_privacy\r\n

\ - /user/findConfig\r\n\r\nCheckApplyStatusFlag: false\r\nCheckSessionFlag: false\r\n\r\nRedisClusterAdr:\r\n

\ - 172.31.114.162:7001\r\n - 172.31.114.162:7002\r\n - 172.31.114.162:7000\r\n

\ - 172.31.114.162:7003\r\n - 172.31.114.162:7004\r\n - 172.31.114.162:7005\r\nRedisMaxRetries:

3\r\nRedisMinRetryBackoff: 8\r\nRedisMaxRetryBackoff: 512\r\nRedisDialTimeout:

5\r\nRedisReadTimeout: 3\r\nRedisWriteTimeout: 3\r\nRedisPoolSize: 10\r\n\r\nSessionCheckExpire:

30\r\n"

kind: ConfigMap

metadata:

annotations:

meta.helm.sh/release-name: connectserver

meta.helm.sh/release-namespace: qa-test

creationTimestamp: "2023-10-08T08:27:54Z"

labels:

app.kubernetes.io/managed-by: Helm

name: connectserver-configmap

namespace: qa-test

resourceVersion: "15948099"

uid: dd52b2c2-9f14-4cdc-9c6a-315e3658e4a0

jxma7@jxma7-PC MINGW64 ~

$ kubectl --kubeconfig D:/k8s_test/config edit configmap connectserver-configmap -n qa-test

Edit cancelled, no changes made.

jxma7@jxma7-PC MINGW64 ~

$ kubectl get pod --output wide -n public-prov3

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

articlesyncserver-articlesyncserver-integration-7469b8886dffbvd 1/1 Running 0 16d 192.180.196.206 172.16.199.171

businessserver-businessserver-integration-5485db7556-9jxqh 1/1 Running 0 128m 192.180.196.118 172.16.199.171

clientmanage-clientmanage-integration-6bc949fb78-d24d2 1/1 Running 0 19d 192.180.157.245 172.16.199.172

connectmanage-connectmanage-integration-5747ffc998-4c4gm 1/1 Running 0 23d 192.180.37.176 172.16.199.174

connectserver-connect-server-integration-b497fcb9b-b5d95 1/1 Running 0 17d 192.180.37.215 172.16.199.174

infoloader-infoloader-integration-8675b9fb84-65w4c 1/1 Running 0 11d 192.180.196.204 172.16.199.171

llmserver-llmserver-integration-59c449b477-9m7lh 1/1 Running 0 10d 192.180.5.184 172.16.199.170

operatorserver-operatorserver-integration-6f4f7c8998-2z96w 1/1 Running 0 11d 192.180.157.134 172.16.199.172

recommend-recommend-integration-7b6bbb87dd-bbccw 1/1 Running 0 11d 192.180.157.222 172.16.199.172

ubuntu-pod 1/1 Running 0 25d 192.180.37.189 172.16.199.174

jxma7@jxma7-PC MINGW64 ~

$ kubectl get pod --output wide -n qa-test

Error from server (Forbidden): pods is forbidden: User "public-prov3" cannot list resource "pods" in API group "" in the namespace "qa-test"

jxma7@jxma7-PC MINGW64 ~

$ kubectl --kubeconfig D:/k8s_test/config get pod --output wide -n qa-test

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

connectserver-connect-server-integration-5b4b8b7fb-dshmj 0/1 CrashLoopBackOff 17 (113s ago) 64m 172.16.180.19 xff0

llmserver-llmserver-integration-6c475f8787-tg6rj 0/1 Running 0 10s 172.16.178.16 xff2

llmserver-llmserver-integration-6c8bcf9b66-xkkv7 0/1 Running 1 (68s ago) 3m16s 172.16.180.22 xff0

jxma7@jxma7-PC MINGW64 ~

$ ping xff0

Ping 请求找不到主机 xff0。请检查该名称，然后重试。

jxma7@jxma7-PC MINGW64 ~

$ kubectl --kubeconfig D:/k8s_test/config get pod --output wide -n qa-test