1. 前言
转载请说明原文出处, 尊重他人劳动成果!
源码位置: https://github.com/nicktming/kubernetes/tree/tming-v1.13/staging/src/k8s.io/code-generator
分支: tming-v1.13 (基于v1.13版本)
本文将在上文 [k8s源码分析][code-generator] crd代码生成的基础上进行分析代码生成是如何工作的. 本文着重分析
list, 由于代码生成这个功能的作用就是为了简化繁琐的操作, 即使不使用代码生成, 自己也可以手写相关代码, 所以说不能一个不能或缺的功能, 只是说锦上添花而已, 因此本文将简单分析其流程, 不会特别细致到每个变量的作用等等.
2. 分析
上文生成的
client, 下面有三个文件夹, 主要看一下listers文件夹是如何生成的.
[root@master client]# pwd
/root/go/src/github.com/nicktming/k8s-crd-controller/pkg/client
[root@master client]# ls
clientset informers listers
[root@master client]# tree listers
listers
└── example.com
└── v1
├── database.go
└── expansion_generated.go
2 directories, 2 files
[root@master client]#
上文关于
lister的生成命令是/root/go/bin/lister-gen --input-dirs github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1 --output-package github.com/nicktming/k8s-crd-controller/pkg/client/listers
接下来将以这条主线进行分析:
2.1 main方法
// kubernetes/staging/src/k8s.io/code-generator/cmd/lister-gen/main.go
// Run it.
if err := genericArgs.Execute(
generators.NameSystems(),
generators.DefaultNameSystem(),
generators.Packages,
);
genericArgs是一个GeneratoArgs对象, 在k8s.io/gengo/args/args.go中, 可以不用管, 只需要知道此处会执行什么就可以了.
//k8s.io/gengo/args/args.go
func (g *GeneratorArgs) Execute(nameSystems namer.NameSystems, defaultSystem string, pkgs func(*generator.Context, *GeneratorArgs) generator.Packages) error {
...
b, err := g.NewBuilder()
if err != nil {
return fmt.Errorf("Failed making a parser: %v", err)
}
c, err := generator.NewContext(b, nameSystems, defaultSystem)
if err != nil {
return fmt.Errorf("Failed making a context: %v", err)
}
c.Verify = g.VerifyOnly
packages := pkgs(c, g)
if err := c.ExecutePackages(g.OutputBase, packages); err != nil {
return fmt.Errorf("Failed executing generator: %v", err)
}
return nil
}
可以看到传进来三个参数
nameSystems,defaultSystem和pkgs, 这里的作用是利用pkgs方法得到需要生成的packages, 然后调用ExecutePackages去生成真正的文件.
这里需要注意的是pkgs中第一个参数c已经解析了传进来的--input-dirs对应的文件的内容, 在后面会看到.
2.2 Packages
这个方法就是生成最终需要生成的
Packages, 然后GeneratorArgs调用ExecutePackages方法根据Packages生成最终的文件. 定义如下:
// k8s.io/gengo/generator/generator.go
type Packages []Package
type Package interface {
// Name returns the package short name.
Name() string
// Path returns the package import path.
Path() string
// 判断是否需要过滤此Type
Filter(*Context, *types.Type) bool
// 头部信息
Header(filename string) []byte
// 如何生成
Generators(*Context) []Generator
}
由于该方法是核心方法, 所以就拆分一段一段进行分析:
生成头部信息
boilerplate, err := arguments.LoadGoBoilerplate()
此段代码是文件头部信息, 内容如下:
/*
Copyright The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Code generated by lister-gen. DO NOT EDIT.
分析每个InputDirs
p := context.Universe.Package(inputDir)
对应内容如下, 上文已经说过
context中已经对inputDir解析过了, 这里u[packagePath]就会直接返回该packagePath下解析处理的Package.
func (u Universe) Package(packagePath string) *Package {
if p, ok := u[packagePath]; ok {
return p
}
p := &Package{
Path: packagePath,
Types: map[string]*Type{},
Functions: map[string]*Type{},
Variables: map[string]*Type{},
Imports: map[string]*Package{},
}
u[packagePath] = p
return p
}
Package对象如下: 该结构体描述了整个packagePath的所有信息.
type Package struct {
// Canonical name of this package-- its path.
Path string
// The location this package was loaded from
SourcePath string
// Short name of this package; the name that appears in the
// 'package x' line.
Name string
// The comment right above the package declaration in doc.go, if any.
DocComments []string
// All comments from doc.go, if any.
Comments []string
// 所有的Type(结构体和接口)
Types map[string]*Type
// 所有的方法
Functions map[string]*Type
// 所有的变量
Variables map[string]*Type
// 所有的import
Imports map[string]*Package
}
对应的
Package信息如下:
inputDir:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1
Path:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1
SourcePath:/root/go/src/github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1
Name:v1
DocComments:[+groupName=nicktming.example.com]
Comments:[+k8s:deepcopy-gen=package,register +groupName=nicktming.example.com]
Types:map[Database:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.Database DatabaseList:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.DatabaseList DatabaseSpec:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.DatabaseSpec]
Functions:map[Kind:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.Kind Resource:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.Resource addKnownTypes:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.addKnownTypes]
Variables:map[AddToScheme:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.AddToScheme SchemeBuilder:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.SchemeBuilder SchemeGroupVersion:github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.SchemeGroupVersion]
Imports:map[k8s.io/apimachinery/pkg/apis/meta/v1:0xc005497d00 k8s.io/apimachinery/pkg/runtime:0xc005497a80 k8s.io/apimachinery/pkg/runtime/schema:0xc005497c00]
生成objectMeta和group version
objectMeta, internal, err := objectMetaForPackage(p)
if err != nil {
klog.Fatal(err)
}
if objectMeta == nil {
// no types in this package had genclient
continue
}
fmt.Printf("objectMeta:%v, internal:%v\n", objectMeta.Name, internal)
var gv clientgentypes.GroupVersion
var internalGVPkg string
if internal {
lastSlash := strings.LastIndex(p.Path, "/")
if lastSlash == -1 {
klog.Fatalf("error constructing internal group version for package %q", p.Path)
}
gv.Group = clientgentypes.Group(p.Path[lastSlash+1:])
internalGVPkg = p.Path
} else {
// 根据Path来区别group verion
parts := strings.Split(p.Path, "/")
gv.Group = clientgentypes.Group(parts[len(parts)-2])
gv.Version = clientgentypes.Version(parts[len(parts)-1])
internalGVPkg = strings.Join(parts[0:len(parts)-1], "/")
}
groupPackageName := strings.ToLower(gv.Group.NonEmpty())
// If there's a comment of the form "// +groupName=somegroup" or
// "// +groupName=somegroup.foo.bar.io", use the first field (somegroup) as the name of the
// group when generating.
if override := types.ExtractCommentTags("+", p.Comments)["groupName"]; override != nil {
gv.Group = clientgentypes.Group(strings.SplitN(override[0], ".", 2)[0])
}
fmt.Printf("Group:%v, verion:%v\n", gv.Group, gv.Version)
寻找第一个带有
genclient和ObjectMeat的Type, 并且将其返回.
// objectMetaForPackage returns the type of ObjectMeta used by package p.
func objectMetaForPackage(p *types.Package) (*types.Type, bool, error) {
generatingForPackage := false
for _, t := range p.Types {
// filter out types which dont have genclient.
// 如果该Type上面没有genclient注释 直接跳过
if !util.MustParseClientGenTags(append(t.SecondClosestCommentLines, t.CommentLines...)).GenerateClient {
continue
}
generatingForPackage = true
for _, member := range t.Members {
// 返回拥有ObjectMeta属性的Type 在例子是Database
if member.Name == "ObjectMeta" {
return member.Type, isInternal(member), nil
}
}
}
if generatingForPackage {
return nil, false, fmt.Errorf("unable to find ObjectMeta for any types in package %s", p.Path)
}
return nil, false, nil
}
// tag中不拥有json 表明是internal
// isInternal returns true if the tags for a member do not contain a json tag
func isInternal(m types.Member) bool {
return !strings.Contains(m.Tags, "json")
}
输出结果如下:
objectMeta:k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta, internal:false
Group:nicktming, verion:v1
这里需要说明一下,
group名字选的是+groupName中按.划分的第一个, 这里只是为了生成方法, 但是真正的GroupVersion是在register.go中的变量
// k8s-crd-controller/pkg/apis/example.com/v1/register.go
var SchemeGroupVersion = schema.GroupVersion{Group: "nicktming.example.com", Version: "v1"}
// k8s-crd-controller/pkg/client/clientset/versioned/typed/example.com/v1/example.com_client.go
func setConfigDefaults(config *rest.Config) error {
gv := v1.SchemeGroupVersion
config.GroupVersion = &gv
config.APIPath = "/apis"
config.NegotiatedSerializer = serializer.DirectCodecFactory{CodecFactory: scheme.Codecs}
if config.UserAgent == "" {
config.UserAgent = rest.DefaultKubernetesUserAgent()
}
return nil
}
在
setConfigDefaults中使用了SchemeGroupVersion然后在rest中生成的request URL会用到. 比如:curl http://localhost:8080/apis/nicktming.example.com/v1/namespaces/default/databases/my-database.
生成typesToGenerate
var typesToGenerate []*types.Type
for _, t := range p.Types {
tags := util.MustParseClientGenTags(append(t.SecondClosestCommentLines, t.CommentLines...))
if !tags.GenerateClient || !tags.HasVerb("list") || !tags.HasVerb("get") {
continue
}
typesToGenerate = append(typesToGenerate, t)
}
if len(typesToGenerate) == 0 {
continue
}
orderer := namer.Orderer{Namer: namer.NewPrivateNamer(0)}
typesToGenerate = orderer.OrderTypes(typesToGenerate)
packagePath := filepath.Join(arguments.OutputPackagePath, groupPackageName, strings.ToLower(gv.Version.NonEmpty()))
fmt.Printf("typesToGenerate:%v, packagePath:%v\n", typesToGenerate, packagePath)
这段代码的意思是确定哪些
Type需要生成lister. (哪些含有+genclient注解的或者tag中含有list或get的), 很显然只有Database符合.
输出结果如下:
typesToGenerate:[github.com/nicktming/k8s-crd-controller/pkg/apis/example.com/v1.Database],
packagePath:github.com/nicktming/k8s-crd-controller/pkg/client/listers/example.com/v1
生成Package并加入到packageList中
主要看一下
GeneratorFunc
GeneratorFunc: func(c *generator.Context) (generators []generator.Generator) {
generators = append(generators, &expansionGenerator{
DefaultGen: generator.DefaultGen{
OptionalName: "expansion_generated",
},
packagePath: filepath.Join(arguments.OutputBase, packagePath),
types: typesToGenerate,
})
for _, t := range typesToGenerate {
generators = append(generators, &listerGenerator{
DefaultGen: generator.DefaultGen{
OptionalName: strings.ToLower(t.Name.Name),
},
outputPackage: arguments.OutputPackagePath,
groupVersion: gv,
internalGVPkg: internalGVPkg,
typeToGenerate: t,
imports: generator.NewImportTracker(),
objectMeta: objectMeta,
})
}
return generators
},
可以看到第一个方法是生成
expansion_generated.go文件. 对应着expansionGenerator类型.
第二个方法是生成database.go文件. 对应着listerGenerator类型.
expansionGenerator
func (g *expansionGenerator) GenerateType(c *generator.Context, t *types.Type, w io.Writer) error {
sw := generator.NewSnippetWriter(w, c, "$", "$")
for _, t := range g.types {
tags := util.MustParseClientGenTags(append(t.SecondClosestCommentLines, t.CommentLines...))
if _, err := os.Stat(filepath.Join(g.packagePath, strings.ToLower(t.Name.Name+"_expansion.go"))); os.IsNotExist(err) {
sw.Do(expansionInterfaceTemplate, t)
if !tags.NonNamespaced {
sw.Do(namespacedExpansionInterfaceTemplate, t)
}
}
}
return sw.Error()
}
var expansionInterfaceTemplate = `
// $.|public$ListerExpansion allows custom methods to be added to
// $.|public$Lister.
type $.|public$ListerExpansion interface {}
`
var namespacedExpansionInterfaceTemplate = `
// $.|public$NamespaceListerExpansion allows custom methods to be added to
// $.|public$NamespaceLister.
type $.|public$NamespaceListerExpansion interface {}
`
就是替换
template中的某些字段成该type的某些字段就可以了.
listerGenerator
func (g *listerGenerator) GenerateType(c *generator.Context, t *types.Type, w io.Writer) error {
...
sw.Do(typeListerStruct, m)
sw.Do(typeListerConstructor, m)
sw.Do(typeLister_List, m)
...
return sw.Error()
}
var typeListerInterface = `
// $.type|public$Lister helps list $.type|publicPlural$.
type $.type|public$Lister interface {
// List lists all $.type|publicPlural$ in the indexer.
List(selector labels.Selector) (ret []*$.type|raw$, err error)
// $.type|publicPlural$ returns an object that can list and get $.type|publicPlural$.
$.type|publicPlural$(namespace string) $.type|public$NamespaceLister
$.type|public$ListerExpansion
}
`
...
作用一样, 替换字段即可.
4. 总结
1. 遍历每一个
inputDir.
2. 解析该inputDir下面所有文件并整理成一个package结构体, 里面存有关于该inputDir下面所有类型(Type), 方法(funcs), 变量(variables)等等.
3. 找到需要生成lister的类型. 指定的是那些带有+genclient或tag中带有list或get的类型(Type). (tag指的是Encoding string `json:"encoding,omitempty"后面用`那段包括的内容)
4. 根据已经定义好的模板创建DefaultPackage, 里面包括了如何生成等等.
5. 调用ExecutePackages将返回的Packages生成对应文件.