前言
为了方便用户安装istio和操作相应的manifest,社区将原本的纯helm安装模式演变成如今的cli和operator模式(https://docs.google.com/docum...)。同时为了保证后向兼容和良好的代码复用,cli复用operator提供的命令行api,operator命令行基于helm的chart渲染进行实现。基本的模式结构如下图所示:
因此,本文从istioctl提供的install和manifest指令出发,按照istioctl->operator->helm的方向进行分析,重点描述operator中cmd的实现流程。
Install
istioctl
命令行参数与实例
此节可参考https://istio.io/latest/docs/...官方文档。
代码入口
istioctl本质上是利用cobra编写的命令行程序,在/istio/istioctl/cmd/root.go中将InstallCmd加入RootCmd中,而InstallCmd由operator实现。
installCmd := mesh.InstallCmd(loggingOptions)
hideInheritedFlags(installCmd, FlagNamespace, FlagIstioNamespace, FlagCharts)
rootCmd.AddCommand(installCmd)
operator
代码流程与重要函数
整体结构如下图所示。
接下来,以istioctl install --set installPackagePath=/istio/manifests --set profile=empty -f test0.yaml -f test1.yaml为例,分析各个函数。其中test0.yaml和test1.yaml的内容如下图所示。
# test0.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: minimal
hub: docker.io/istio
tag: 1.1.4
components:
pilot:
enabled: true
# test1.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
hub: docker.io/istio
tag: 1.1.5
components:
pilot:
enabled: true
ParseYAMLFiles
ParseYAMLFiles从左到右读取并覆盖-f标志所传入的IstioOperator yaml,并提取出其中的profile字段。因此提取出的profile为minimal,test0.yaml和test1.yaml的初步处理结果temp0.yaml如下所示。
# temp0.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: minimal
hub: docker.io/istio
tag: 1.1.5
components:
pilot:
enabled: true
ReadYaml(And)Profile
ReadYamlProfile根据--set profile=empty对ParseYAMLFiles返回的profile结果进行覆盖,因此最终的profile为empty。
GetProfileYAML
GetProfileYAML从--set installPackagePath=/istio/manifests指定的目录读取empty yaml,因为istio所提供的所有内置profile都基于default profile,因此利用empty yaml去覆盖default profile,最终得到temp1.yaml。
#empty yaml
# The empty profile has everything disabled
# This is useful as a base for custom user configuration
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
base:
enabled: false
pilot:
enabled: false
ingressGateways:
- name: istio-ingressgateway
enabled: false
#default yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
spec:
hub: gcr.io/istio-testing
tag: latest
# You may override parts of meshconfig by uncommenting the following lines.
meshConfig:
defaultConfig:
proxyMetadata: {}
enablePrometheusMerge: true
# Opt-out of global http2 upgrades.
# Destination rule is used to opt-in.
# h2_upgrade_policy: DO_NOT_UPGRADE
# Traffic management feature
components:
base:
enabled: true
pilot:
enabled: true
# Istio Gateway feature
ingressGateways:
- name: istio-ingressgateway
enabled: true
egressGateways:
- name: istio-egressgateway
enabled: false
# Istio CNI feature
cni:
enabled: false
# Remote and config cluster configuration for an external istiod
istiodRemote:
enabled: false
# Global values passed through to helm global.yaml.
# Please keep this in sync with manifests/charts/global.yaml
values:
defaultRevision: ""
global:
istioNamespace: istio-system
istiod:
enableAnalysis: false
logging:
level: "default:info"
logAsJson: false
pilotCertProvider: istiod
jwtPolicy: third-party-jwt
proxy:
image: proxyv2
clusterDomain: "cluster.local"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 2000m
memory: 1024Mi
logLevel: warning
componentLogLevel: "misc:error"
privileged: false
enableCoreDump: false
statusPort: 15020
readinessInitialDelaySeconds: 1
readinessPeriodSeconds: 2
readinessFailureThreshold: 30
includeIPRanges: "*"
excludeIPRanges: ""
excludeOutboundPorts: ""
excludeInboundPorts: ""
autoInject: enabled
tracer: "zipkin"
proxy_init:
image: proxyv2
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 10m
memory: 10Mi
# Specify image pull policy if default behavior isn't desired.
# Default behavior: latest images will be Always else IfNotPresent.
imagePullPolicy: ""
operatorManageWebhooks: false
tracer:
lightstep: {}
zipkin: {}
datadog: {}
stackdriver: {}
imagePullSecrets: []
oneNamespace: false
defaultNodeSelector: {}
configValidation: true
multiCluster:
enabled: false
clusterName: ""
omitSidecarInjectorConfigMap: false
network: ""
defaultResources:
requests:
cpu: 10m
defaultPodDisruptionBudget:
enabled: true
priorityClassName: ""
useMCP: false
sds:
token:
aud: istio-ca
sts:
servicePort: 0
meshNetworks: {}
mountMtlsCerts: false
base:
enableCRDTemplates: false
validationURL: ""
pilot:
autoscaleEnabled: true
autoscaleMin: 1
autoscaleMax: 5
replicaCount: 1
image: pilot
traceSampling: 1.0
env: {}
cpu:
targetAverageUtilization: 80
nodeSelector: {}
keepaliveMaxServerConnectionAge: 30m
enableProtocolSniffingForOutbound: true
enableProtocolSniffingForInbound: true
deploymentLabels:
podLabels: {}
configMap: true
telemetry:
enabled: true
v2:
enabled: true
metadataExchange:
wasmEnabled: false
prometheus:
wasmEnabled: false
enabled: true
stackdriver:
enabled: false
logging: false
monitoring: false
topology: false
configOverride: {}
istiodRemote:
injectionURL: ""
gateways:
istio-egressgateway:
env: {}
autoscaleEnabled: true
type: ClusterIP
name: istio-egressgateway
secretVolumes:
- name: egressgateway-certs
secretName: istio-egressgateway-certs
mountPath: /etc/istio/egressgateway-certs
- name: egressgateway-ca-certs
secretName: istio-egressgateway-ca-certs
mountPath: /etc/istio/egressgateway-ca-certs
istio-ingressgateway:
autoscaleEnabled: true
type: LoadBalancer
name: istio-ingressgateway
env: {}
secretVolumes:
- name: ingressgateway-certs
secretName: istio-ingressgateway-certs
mountPath: /etc/istio/ingressgateway-certs
- name: ingressgateway-ca-certs
secretName: istio-ingressgateway-ca-certs
mountPath: /etc/istio/ingressgateway-ca-certs
#temp1.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
spec:
components:
base:
enabled: false
cni:
enabled: false
egressGateways:
- enabled: false
name: istio-egressgateway
ingressGateways:
- enabled: false
name: istio-ingressgateway
istiodRemote:
enabled: false
pilot:
enabled: false
hub: gcr.io/istio-testing
meshConfig:
defaultConfig:
proxyMetadata: {}
enablePrometheusMerge: true
tag: latest
values:
base:
enableCRDTemplates: false
validationURL: ""
defaultRevision: ""
gateways:
istio-egressgateway:
autoscaleEnabled: true
env: {}
name: istio-egressgateway
secretVolumes:
- mountPath: /etc/istio/egressgateway-certs
name: egressgateway-certs
secretName: istio-egressgateway-certs
- mountPath: /etc/istio/egressgateway-ca-certs
name: egressgateway-ca-certs
secretName: istio-egressgateway-ca-certs
type: ClusterIP
istio-ingressgateway:
autoscaleEnabled: true
env: {}
name: istio-ingressgateway
secretVolumes:
- mountPath: /etc/istio/ingressgateway-certs
name: ingressgateway-certs
secretName: istio-ingressgateway-certs
- mountPath: /etc/istio/ingressgateway-ca-certs
name: ingressgateway-ca-certs
secretName: istio-ingressgateway-ca-certs
type: LoadBalancer
global:
configValidation: true
defaultNodeSelector: {}
defaultPodDisruptionBudget:
enabled: true
defaultResources:
requests:
cpu: 10m
imagePullPolicy: ""
imagePullSecrets: []
istioNamespace: istio-system
istiod:
enableAnalysis: false
jwtPolicy: third-party-jwt
logAsJson: false
logging:
level: default:info
meshNetworks: {}
mountMtlsCerts: false
multiCluster:
clusterName: ""
enabled: false
network: ""
omitSidecarInjectorConfigMap: false
oneNamespace: false
operatorManageWebhooks: false
pilotCertProvider: istiod
priorityClassName: ""
proxy:
autoInject: enabled
clusterDomain: cluster.local
componentLogLevel: misc:error
enableCoreDump: false
excludeIPRanges: ""
excludeInboundPorts: ""
excludeOutboundPorts: ""
image: proxyv2
includeIPRanges: '*'
logLevel: warning
privileged: false
readinessFailureThreshold: 30
readinessInitialDelaySeconds: 1
readinessPeriodSeconds: 2
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 100m
memory: 128Mi
statusPort: 15020
tracer: zipkin
proxy_init:
image: proxyv2
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 10m
memory: 10Mi
sds:
token:
aud: istio-ca
sts:
servicePort: 0
tracer:
datadog: {}
lightstep: {}
stackdriver: {}
zipkin: {}
useMCP: false
istiodRemote:
injectionURL: ""
pilot:
autoscaleEnabled: true
autoscaleMax: 5
autoscaleMin: 1
configMap: true
cpu:
targetAverageUtilization: 80
deploymentLabels: null
enableProtocolSniffingForInbound: true
enableProtocolSniffingForOutbound: true
env: {}
image: pilot
keepaliveMaxServerConnectionAge: 30m
nodeSelector: {}
podLabels: {}
replicaCount: 1
traceSampling: 1
telemetry:
enabled: true
v2:
enabled: true
metadataExchange:
wasmEnabled: false
prometheus:
enabled: true
wasmEnabled: false
stackdriver:
configOverride: {}
enabled: false
logging: false
monitoring: false
topology: false
OverlayIOP
OverlayIOP将ReadYaml(And)Profile所得到的temp0.yaml覆盖于temp1.yaml之上,得到final.yaml
#empty yaml
# The empty profile has everything disabled
# This is useful as a base for custom user configuration
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
base:
enabled: false
pilot:
enabled: false
ingressGateways:
- name: istio-ingressgateway
enabled: false
# final.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
creationTimestamp: null
namespace: istio-system
spec:
components:
base:
enabled: false
cni:
enabled: false
egressGateways:
- enabled: false
name: istio-egressgateway
ingressGateways:
- enabled: false
name: istio-ingressgateway
istiodRemote:
enabled: false
pilot:
enabled: true
hub: docker.io/istio
installPackagePath: /istio/manifests
meshConfig:
defaultConfig:
proxyMetadata: {}
enablePrometheusMerge: true
profile: empty
tag: 1.1.5
values:
base:
enableCRDTemplates: false
validationURL: ""
defaultRevision: ""
gateways:
istio-egressgateway:
autoscaleEnabled: true
env: {}
name: istio-egressgateway
secretVolumes:
- mountPath: /etc/istio/egressgateway-certs
name: egressgateway-certs
secretName: istio-egressgateway-certs
- mountPath: /etc/istio/egressgateway-ca-certs
name: egressgateway-ca-certs
secretName: istio-egressgateway-ca-certs
type: ClusterIP
istio-ingressgateway:
autoscaleEnabled: true
env: {}
name: istio-ingressgateway
secretVolumes:
- mountPath: /etc/istio/ingressgateway-certs
name: ingressgateway-certs
secretName: istio-ingressgateway-certs
- mountPath: /etc/istio/ingressgateway-ca-certs
name: ingressgateway-ca-certs
secretName: istio-ingressgateway-ca-certs
type: LoadBalancer
global:
configValidation: true
defaultNodeSelector: {}
defaultPodDisruptionBudget:
enabled: true
defaultResources:
requests:
cpu: 10m
imagePullPolicy: ""
imagePullSecrets: []
istioNamespace: istio-system
istiod:
enableAnalysis: false
jwtPolicy: third-party-jwt
logAsJson: false
logging:
level: default:info
meshNetworks: {}
mountMtlsCerts: false
multiCluster:
clusterName: ""
enabled: false
network: ""
omitSidecarInjectorConfigMap: false
oneNamespace: false
operatorManageWebhooks: false
pilotCertProvider: istiod
priorityClassName: ""
proxy:
autoInject: enabled
clusterDomain: cluster.local
componentLogLevel: misc:error
enableCoreDump: false
excludeIPRanges: ""
excludeInboundPorts: ""
excludeOutboundPorts: ""
image: proxyv2
includeIPRanges: '*'
logLevel: warning
privileged: false
readinessFailureThreshold: 30
readinessInitialDelaySeconds: 1
readinessPeriodSeconds: 2
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 100m
memory: 128Mi
statusPort: 15020
tracer: zipkin
proxy_init:
image: proxyv2
resources:
limits:
cpu: 2000m
memory: 1024Mi
requests:
cpu: 10m
memory: 10Mi
sds:
token:
aud: istio-ca
sts:
servicePort: 0
tracer:
datadog: {}
lightstep: {}
stackdriver: {}
zipkin: {}
useMCP: false
istiodRemote:
injectionURL: ""
pilot:
autoscaleEnabled: true
autoscaleMax: 5
autoscaleMin: 1
configMap: true
cpu:
targetAverageUtilization: 80
deploymentLabels: null
enableProtocolSniffingForInbound: true
enableProtocolSniffingForOutbound: true
env: {}
image: pilot
keepaliveMaxServerConnectionAge: 30m
nodeSelector: {}
podLabels: {}
replicaCount: 1
traceSampling: 1
telemetry:
enabled: true
v2:
enabled: true
metadataExchange:
wasmEnabled: false
prometheus:
enabled: true
wasmEnabled: false
stackdriver:
configOverride: {}
enabled: false
logging: false
monitoring: false
topology: false
GenerateConfig
配置部分总结如下,operator将绝大部分面向用户的配置复杂度都隐藏在了内置profile中,其中base profile是基础演变项,演变过程为base profile -> select profile -> user config -> set flag,越往后优先级越高。在具体实践中,内置profile能满足大多数需求,若有额外需求则使用user config解决,set flag不便于管理。
RenderCharts
前面提到,operator要使用helm的chart rendering,因此通过GenerateConfig生成的final.yaml可以映射到helm的values.yaml。并且,为了便于用户理解与维护helm chart,istio控制面的每一个组件都有一个chart与之对应,final.yaml中各个组件的配置最终映射到各个chart的values.yaml进行渲染。charts目录如下所示。
processRecursive
每个组件的chart渲染成功后生成所需的manifest,并利用k8s的api与apiserver进行交互,最后成功创建或更新istio正常运行所需的所有资源。
Manifest
Manifest命令基本与Install命令的流程类似,只是通过helm渲染后不需要与apiserver交互并创建资源,因此不再赘述。