k8s：code-generator代码生成器的使用

code-generator

code-generator是K8S官⽅提供的⼀组代码⽣成⼯具，它主要有两个应⽤场景：

为CRD编写⾃定义controller时，可以使⽤它来⽣成我们需要的versionedclient、informer、lister以及其他⼯具⽅法；

编写⾃定义APIServer时，可以⽤它来internal和versioned类型的转换defaulters、internal和versioned的clients和informers；

项目视图

sample-controller                                   *  modulename
    ├── go.mod                                      *  go mod init 出来的文件
    ├── go.sum                                      *  go mod init 出来的检测文件
    ├── hack                                   
    │   ├── boilerplate.go.txt                      *  operator代码框架版权信息头内容
    │   ├── tools.go                                *  用于vendor自动下载code-generator文件的配置信息
    │   ├── update-codegen.sh                       *  官方sample-controller项目的模板文件，用于填写code-generator中的相关变量
    │   └── verify-codegen.sh                       *  调用update-codegen.sh，并检测代码变化的脚本，CI过程中非常重要
    ├── pkg                                         *  标准目录
    │   ├── apis                                    *  标准目录
    │   │   └── samplecontroller                    *  group,API组
    │   │       └── v1alpha1                        *  version,API版本
    │   │           ├── doc.go                      *  代码生成器模板，package相关内容，全局标签填写文件
    │   │           ├── types.go                    *  代码生成器模板，API类型相关内容，局部标签填写文件
    │   │           └── zz_generated.deepcopy.go    *  代码生成器运行后生成的深拷贝go文件
    │   └── generated
    │       ├── clientset                           *  代码生成器生成的clientset相关代码
    │       ├── informers                           *  代码生成器生成的informers相关代码
    │       └── listers                             *  代码生成器生成的listers相关代码
    └── vendor                                      *  vendor包管理，项目依赖包会放此处，包括code-generator
        ├── github.com
        ├── golang.org
        ├── gopkg.in
        └── k8s.io
            ├── apimachinery
            └── code-generator
                └── generate-groups.sh              * 代码生成器主要脚本

流程本质：通过运行hack目录下的update-codegen.sh文件，去调用code-generator中的generate-groups.sh，生成代码框架。

generate-groups.sh讲解

由update-codegen.sh调用的generate-groups.sh，输出的完整指令为：

$ vendor/k8s.io/code-generator/generate-groups.sh "deepcopy,client,informer,lister" \
  sample-controller/pkg/generated sample-controller/pkg/apis \
  samplecontroller:v1alpha1 \
  --output-base "${GOPATH}/src" \
  --go-header-file "${GOPATH}/src/sample-controller/hack/boilerplate.go.txt"

这个命令一共有6个参数，其中，第一个参数"deepcopy,client,informer,lister"，可用all代替，其代表着4种标准代码生成器：
deepcopy-gen：生成DeepCopy方法实现对对象的深拷贝。
client-gen：生成强类型的客户端集合。
informer-gen：为自定义资源生成informer。
lister-gen：为自定义资源生成lister对象。
接着，我们看看其他参数含义：
1.第二个参数用于指定客户端、lister和informer代码框架的包名；
2.第三个参数是API组的基础包名；
3.第四个参数是API组列表与版本号；
4.第五个参数–output-base用于代码生成器查找包输出的基础路径；
5.第六个参数是自定义代码所用到的版权信息头。

从零开始搭建项目

1、创建项目与相关文件夹

在gopath目录下的src里，创建名为sample-controller的文件夹，同时在文件夹里创建hack、pkg/apis目录，以及在apis目录下创建对应API组和版本名称的文件夹。

$ mkdir -p sample-conrtoller/hack sample-conrtoller/pkg/apis/
$ tree
sample-controller                                  *  modulename
    ├── hack                                   
    └── pkg                                        *  标准目录
        └── apis                                   *  标准目录
            └──samplecontroller                    *  group,API组
                └── v1alpha1                       *  version,API版本

2、准备hack脚本

$ ls
boilerplate.go.txt  tools.go  update-codegen.sh  verify-codegen.sh

boilerplate.go.txt文件内容：

/*
Copyright The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

tools.go文件内容：

// +build tools

/*
Copyright 2019 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// This package imports things required by build scripts, to force `go mod` to see them as dependencies
package tools

import _ "k8s.io/code-generator"

update-codegen.sh文件内容（根据项目实际情况修改参数信息）：

#!/usr/bin/env bash

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd "${SCRIPT_ROOT}"; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../code-generator)}
bash "${CODEGEN_PKG}"/generate-groups.sh "deepcopy,client,lister,informer" \
${MODULE}/${OUTPUT_PKG} ${MODULE}/${APIS_PKG} \
${GROUP}:${VERSION} \
--output-base "${SCRIPT_ROOT}" \
--go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt \
#####################样例 start##################################
#注意事项：
#MODULE需和go.mod文件内容一致
#"${CODEGEN_PKG}"/generate-groups.sh "deepcopy,client,informer,lister" \
#  sample-controller/pkg/generated sample-controller/pkg/apis \
#  samplecontroller:v1alpha1 \
#  --output-base "$(dirname "${BASH_SOURCE[0]}")/../.." \
#  --go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt
#####################样例 end##################################

verify-codegen.sh文件内容：

#!/usr/bin/env bash

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..

DIFFROOT="${SCRIPT_ROOT}/pkg"
TMP_DIFFROOT="${SCRIPT_ROOT}/_tmp/pkg"
_tmp="${SCRIPT_ROOT}/_tmp"

cleanup() {
  rm -rf "${_tmp}"
}
trap "cleanup" EXIT SIGINT

cleanup

mkdir -p "${TMP_DIFFROOT}"
cp -a "${DIFFROOT}"/* "${TMP_DIFFROOT}"

"${SCRIPT_ROOT}/hack/update-codegen.sh"
echo "diffing ${DIFFROOT} against freshly generated codegen"
ret=0
diff -Naupr "${DIFFROOT}" "${TMP_DIFFROOT}" || ret=$?
cp -a "${TMP_DIFFROOT}"/* "${DIFFROOT}"
if [[ $ret -eq 0 ]]
then
  echo "${DIFFROOT} up to date."
else
  echo "${DIFFROOT} is out of date. Please run hack/update-codegen.sh"
  exit 1
fi

3、编写代码生成器标签文件

通过标签来控制代码生成时的属性，标签分为2种：

doc.go文件，在package前的标签为全局标签
type.go文件中的类型声明前的局部标签

这2个文件都放置在pkg/apis/{group}/{version}中，标签含义内容众多，本文暂不介绍，下边为样例文件：

doc.go文件内容
(样例中+k8s:deepcopy-gen=package即为全局标签)：

/*Go
Copyright 2017 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// +k8s:deepcopy-gen=package
// +groupName=samplecontroller.k8s.io

// Package v1alpha1 is the v1alpha1 version of the API.
package v1alpha1 // import "sample-controller/pkg/apis/samplecontroller/v1alpha1"

type.go文件内容
(样例中+k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object为局部标签):

package v1alpha1

import (
        metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

const (
        PhasePending = "PENDING"
        PhaseRunning = "RUNNING"
        PhaseDone    = "DONE"
)

// AtSpec defines the desired state of At
type AtSpec struct {
        // Schedule is the desired time the command is supposed to be executed.
        // Note: the format used here is UTC time https://www.utctime.net
        Schedule string `json:"schedule,omitempty"`
        // Command is the desired command (executed in a Bash shell) to be executed.
        Command string `json:"command,omitempty"`
        // Important: Run "make" to regenerate code after modifying this file
}

// AtStatus defines the observed state of At
type AtStatus struct {
        // Phase represents the state of the schedule: until the command is executed
        // it is PENDING, afterwards it is DONE.
        Phase string `json:"phase,omitempty"`
        // Important: Run "make" to regenerate code after modifying this file
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// At runs a command at a given schedule.
type At struct {
        metav1.TypeMeta   `json:",inline"`
        metav1.ObjectMeta `json:"metadata,omitempty"`

        Spec   AtSpec   `json:"spec,omitempty"`
        Status AtStatus `json:"status,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// AtList contains a list of At
type AtList struct {
        metav1.TypeMeta `json:",inline"`
        metav1.ListMeta `json:"metadata,omitempty"`
        Items           []At `json:"items"`
}

到此步骤，当前项目结构如下：

sample-controller                  *  modulename
    ├── hack                                   
    │   ├── boilerplate.go.txt     *  operator代码框架版权信息头内容
    │   ├── tools.go               *  用于vendor自动下载code-generator文件的配置信息
    │   ├── update-codegen.sh      *  官方sample-controller项目的模板文件，用于填写code-generator中的相关变量
    │   └── verify-codegen.sh      *  调用update-codegen.sh，并检测代码变化的脚本，CI过程中非常重要
    └── pkg                        *  标准目录
        └── apis                   *  标准目录
            └── samplecontroller   *  group,API组
                └── v1alpha1       *  version,API版本
                    ├── doc.go     *  代码生成器模板，package相关内容，全局标签填写文件
                    └── types.go   *  代码生成器模板，API类型相关内容，局部标签填写文件

自此，需要手动创建的文件已完成，接下来引入vendor，通过vendor管理code-generator包，也可以手动下载code-generator，此时要注意hack/update-codegen.sh中的CODEGEN_PKG变量信息。

4、引入vendor包管理工具

在项目根目录初始化项目（注意目录层级和modulename间的关系）：

$ go mod init {modulename} #注意，modulename应为$gopath/src后的内容，对应生成go.mod和go.sum文件
$ ls 
go.mod go.sum hack pkg
$ pwd
/data/gopath/src/k8s.io/sample-controller
$ cat go.mod
// This is a generated file. Do not edit directly.

module k8s.io/sample-controller

go 1.16

自动读取包内依赖，并下载相关依赖：

$ go mod tidy
$ go mod vendor
$ ls
go.mod go.sum hack pkg vendor

此时，vendor目录内包含k8s.io/code-generator文件，以及其他依赖包。

5、运行代码生成工具

$ chmod +x ./hack//update-codegen.sh
$ ./hack/update-codegen.sh 

执行完后，会自动生成下列源代码文件：

pkg/generated/clientset/*
pkg/generated/informers/*
pkg/generated/listers/*
pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go

1.执行脚本后，不生成informers和listers文件，只有clientset，或者都没有。
原因在于go.mod文件里的moduleName与update-codegen.sh脚本里的moduleName不一致，或者，该变量与实际情况（$gopath/src后的地址）不一致。

2.执行完后，在goland里报错，显示传参类型不一致。
我的情况是goland编辑器版本较低，不适配1.7和1.8的golang版本，降低至1.6即可。

3.vendor里没k8s.io/code-generator文件.
检查下hack目录里的tool.go文件，是否引入了k8s.io/code-generator包，也可以在任意go文件里import这个包，go mod tidy后会自动加载。

常⽤code-generator标记
deepcopy相关标

//关闭
//+k8s:deepcopy-gen=false
//打开
//+k8s:deepcopy-gen=true
//⽣成DeepCopyObject⽅法
//+k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

clientset,lister,informer相关标记

//+genclient
//+genclient:noStatus
//cluster级别的
//+genclient:nonNamespaced
//+genclient:noVerbs
//+genclient:onlyVerbs=create,delete//+genclient:skipVerbs=get,list,create,update,patch,delete,deleteCollection,watch
//+genclient:method=Create,verb=create,result=k8s.io/apimachinery/pkg/apis/meta/v1.Status

参考资料：链接