promu的使用及源码分析

promu是prometheus组件的二进制构建工具，包括prometheus、node-exporter都是使用promu构建出来的二进制文件。

具体来说，promu封装了go命令，为目标定义了使用go命令进行编译、构建的流程(比如构建几个binary，从哪里构建)，添加了编译的参数(flags/ldflags)。

一. promu的使用

1. promu构建prometheus

prometheus源码中Makefile.common文件，描述了使用promu构建prometheus的过程：

• common-build:

  • 首先，common-build依赖promu；
  • 然后，使用promu build命令，构建prometheus的二进制文件(包括prometheus/promtool)；

• promu:

  • 使用curl从github下载promu二进制文件，然后copy到~/go/bin中；

.PHONY: common-build
common-build: promu
    @echo ">> building binaries"
    GO111MODULE=$(GO111MODULE) $(PROMU) build --prefix $(PREFIX) $(PROMU_BINARIES)

PROMU_VERSION ?= 0.5.0
PROMU_URL     := https://github.com/prometheus/promu/releases/download/v$(PROMU_VERSION)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM).tar.gz

.PHONY: promu
promu: $(PROMU)

$(PROMU):
    $(eval PROMU_TMP := $(shell mktemp -d))
    curl -s -L $(PROMU_URL) | tar -xvzf - -C $(PROMU_TMP)
    mkdir -p $(FIRST_GOPATH)/bin
    cp $(PROMU_TMP)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM)/promu $(FIRST_GOPATH)/bin/promu
    rm -r $(PROMU_TMP)

2. promu的配置文件

promu使用的配置文件，若未指定，则默认使用当前目录的.promu.yml文件。

以prometheus的.promu.yml文件为例：

# cat .promu.yml
go:
    # Whenever the Go version is updated here,
    # .circle/config.yml should also be updated.
    version: 1.14
repository:
    path: github.com/prometheus/prometheus
build:
    binaries:
        - name: prometheus
          path: ./cmd/prometheus
        - name: promtool
          path: ./cmd/promtool
        - name: tsdb
          path: ./tsdb/cmd/tsdb
    flags: -mod=vendor -a -tags netgo,builtinassets
    ldflags: |
        -X github.com/prometheus/common/version.Version={{.Version}}
        -X github.com/prometheus/common/version.Revision={{.Revision}}
        -X github.com/prometheus/common/version.Branch={{.Branch}}
        -X github.com/prometheus/common/version.BuildUser={{user}}@{{host}}
        -X github.com/prometheus/common/version.BuildDate={{date "20060102-15:04:05"}}
tarball:
    files:
        - consoles
        - console_libraries
        - documentation/examples/prometheus.yml
        - LICENSE
        - NOTICE
        - npm_licenses.tar.bz2
crossbuild:
    platforms:
        - linux/amd64
        - linux/386
...

由于构建时使用promu build命令，我们重点关注配置文件中的build部分：

• binary：定义build哪些二进制

  • 从配置文件可以看出，build了prometheus/promtool/tsdb

• flags: 定义go build时候的参数

  • -mod=vendor指定使用go module的本地vendor编译；
• ldflags: 定义go build时候的动态链接参数；

promu通过.promu.yml规定编译构建的流程，具体来说：

• 它最终还是调用go build进行编译；
• 它给go build增加了各种编译参数，比如-mod vendor，ldflags等；
• 它定义了编译的目标，比如prometheus/promtool/tsdb等；

二. promu的源码分析

promu最终调用的还是go的命令，比如promu build调用go build。

1. 配置文件解析

默认从.promu.yml读配置文件：

const (
    // DefaultConfigFilename contains the default filename of the promu config file
    DefaultConfigFilename = ".promu.yml"
)
configFile = app.Flag("config", "Path to config file").Short('c').
            Default(DefaultConfigFilename).String()

文件解析过程：

// cmd/promu.go
func initConfig(filename string) {
    info(fmt.Sprintf("Using config file: %v", filename))
    configData, err := ioutil.ReadFile(filename)
    checkError(err, "Unable to read config file: "+filename)
    config = NewConfig()
    err = yaml.Unmarshal(configData, config)
    checkError(err, "Unable to parse config file: "+filename)
}

所有的配置，最终被反序列化到Config对象中：

type Config struct {
    Build struct {
        Binaries   []Binary
        Flags      string
        LDFlags    string
        ExtLDFlags []string
        Prefix     string
        Static     bool
    }
    Crossbuild struct {
        Platforms []string
    }
    Repository struct {
        Path string
    }
    Go struct {
        CGo     bool
        Version string
    }
    Tarball struct {
        Files  []string
        Prefix string
    }
}

我们重点关注promu build命令，以及.promu.yml中build参数的使用。

2. promu build命令解析

promu支持不同的子命令，比如build子命令的执行流程：

// cmd/promu.go
func Execute() {
    var err error
    projInfo, err = NewProjectInfo()
    checkError(err, "Unable to initialize project info")

    command := kingpin.MustParse(app.Parse(os.Args[1:]))    // 子命令
    sh.Verbose = *verbose
    initConfig(*configFile)

    info(fmt.Sprintf("Running command: %v %v", command, os.Args[2:]))

    switch command {
    case buildcmd.FullCommand():        // build子命令
        runBuild(optArg(*binariesArg, 0, "all"))
    ...
    case versioncmd.FullCommand():       // version子命令
        runVersion()
    }
}

典型的，prometheus中.promu.yml的build参数的定义：

build:
    binaries:
        - name: prometheus
          path: ./cmd/prometheus
        - name: promtool
          path: ./cmd/promtool
        - name: tsdb
          path: ./tsdb/cmd/tsdb
    flags: -mod=vendor -a -tags netgo,builtinassets
    ldflags: |
        -X github.com/prometheus/common/version.Version={{.Version}}
        -X github.com/prometheus/common/version.Revision={{.Revision}}
        -X github.com/prometheus/common/version.Branch={{.Branch}}
        -X github.com/prometheus/common/version.BuildUser={{user}}@{{host}}
        -X github.com/prometheus/common/version.BuildDate={{date "20060102-15:04:05"}}

build.binaries参数的处理

build.binary指定了build的目标main.go；

对每个binary配置，执行go build -o binaryName命令：

// cmd/build.go
func buildAll(ext string, prefix string, ldflags string, binaries []Binary) {
    for _, binary := range binaries {
        buildBinary(ext, prefix, ldflags, binary)
    }
}

func buildBinary(ext string, prefix string, ldflags string, binary Binary) {
    ...
    binaryName := fmt.Sprintf("%s%s", binary.Name, ext)
    fmt.Printf(" >   %s\n", binaryName)
    ...
    params := []string{"build",
        "-o", path.Join(prefix, binaryName),
        "-ldflags", ldflags,
    }
    ...
    // 执行 go build命令
    if err := sh.RunCommand("go", params...); err != nil {
        fatal(errors.Wrap(err, "command failed: "+strings.Join(params, " ")))
    }
}

build.flags参数的处理

build.flags指定了build的参数，比如-mod vendor；

// cmd/build.go
func buildBinary(ext string, prefix string, ldflags string, binary Binary) {
    info("Building binary: " + binary.Name)
    binaryName := fmt.Sprintf("%s%s", binary.Name, ext)
    fmt.Printf(" >   %s\n", binaryName)

    repoPath := config.Repository.Path
    flags := config.Build.Flags

    params := []string{"build",
        "-o", path.Join(prefix, binaryName),
        "-ldflags", ldflags,
    }
    // flags的参数被添加到 go build的参数列表中
    params = append(params, sh.SplitParameters(flags)...)
    params = append(params, path.Join(repoPath, binary.Path))
    info("Building binary: " + "go " + strings.Join(params, " "))
    if err := sh.RunCommand("go", params...); err != nil {
        fatal(errors.Wrap(err, "command failed: "+strings.Join(params, " ")))
    }
}

build.ldflags参数的处理

build.ldflags指定了build的动态参数，比如-X version=v1等：

// cmd/build.go
func buildBinary(ext string, prefix string, ldflags string, binary Binary) {
    info("Building binary: " + binary.Name)
    binaryName := fmt.Sprintf("%s%s", binary.Name, ext)
    fmt.Printf(" >   %s\n", binaryName)

    repoPath := config.Repository.Path
    flags := config.Build.Flags

    params := []string{"build",
        "-o", path.Join(prefix, binaryName),
        "-ldflags", ldflags,
    }
    params = append(params, sh.SplitParameters(flags)...)
    params = append(params, path.Join(repoPath, binary.Path))
    info("Building binary: " + "go " + strings.Join(params, " "))
    if err := sh.RunCommand("go", params...); err != nil {
        fatal(errors.Wrap(err, "command failed: "+strings.Join(params, " ")))
    }
}

参考：

1.https://ost.51cto.com/posts/1...
2.https://github.com/prometheus...