Argo研究

Argo

参考文档

https://github.com/terrytangyuan/awesome-argo

概述

Argo 由一组开源工具组成，用于在 Kubernetes 上部署和运行应用程序和工作负载，包括：‎

• Argo Workflows: Kubernetes-native workflow engine supporting DAG and step-based workflows.

  用于构造管道/工作流并处理编排和调度的通用框架。有许多不同的用例，一些组织将其用于 CI/CD。‎

• Argo CD: Declarative continuous delivery with a fully-loaded UI.

  专注于CD，更多的是在Kubernetes上进行声明式和GitOps风格的持续交付，您可以使用kustomize/helm等工具来管理清单。‎

• Argo Rollouts: Advanced Kubernetes deployment strategies such as Canary and Blue-Green made easy.

• Argo Events: Event based dependency management for Kubernetes.

In addition, argoproj-labs is a separate GitHub org for community contributions related to the Argo ecosystem.

Argo cd

参考文档

Argo Project (github.com)

Argo CD - Declarative GitOps CD for Kubernetes (argo-cd.readthedocs.io)

概述

Argo CD——这是一个GitOps工具，可以让你在Git中维护Kubernetes资源的状态。Argo CD会自动将你的Kubernetes资源与Git仓库中的资源进行同步，同时也确保集群内对manifest的手动更改会自动还原。这保证了你的声明式部署模式。

Argo CD被实现为kubernetes控制器，该控制器连续监视正在运行的应用程序， 并将当前的活动状态与所需的目标状态（在Git存储库中指定）进行比较。 其活动状态偏离目标状态的已部署应用程序被标记为OutOfSync。 Argo CD报告并可视化差异，同时提供了自动或手动将实时状态同步回所需目标状态的功能。 在Git存储库中对所需目标状态所做的任何修改都可以自动应用并反映在指定的目标环境中。

1.2 Argo CD 能落地 GitOps

Argo CD 是以 Kubernetes 为基础设施的 GitOps 持续部署工具。下面是来自 Argo CD 社区的原理图：

1. Argo CD 从 Git Repo 拉取应用的配置，部署在 Kubernetes 集群中。
2. 当有人新增功能时，提交一个 Pull Requests 到 Git Repo 修改应用的部署配置，等待合并。
3. 在 Pull Requests 合并之后，通过 Webhook 触发 Argo CD 执行更新操作。
4. 应用得到更新，发送通知

理解起来很容易，将运维过程自动化，持续的部署。

Getting Started

1. 在 Kubernetes 上部署 Argo CD，并Login with Argo UI

# 1、新建命名空间，部署 Argo CD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# 2、Access The Argo CD API Server,将服务改为 NodePort 类型，方便访问
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

# 3、查看 admin 账户密码
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

admin/Jt5yuFrBGJTcoS4c

# 4、Login with Argo CD UI，打开页面 http://{HOST_IP}:NodePort,使用admin和上步骤密码登陆
kubectl -n argocd get svc
NAME                                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
argocd-server                             NodePort    10.96.69.10     <none>        80:30938/TCP,443:31949/TCP   19h

2. 安装 CLI 工具，并Login with Argo CLI

# 1. Download Argo CD CLI  (linux)
curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x /usr/local/bin/argocd

# 2.Login with Argo CD CLI
argocd login {HOST_IP}:NodePort --username admin --password Jt5yuFrBGJTcoS4c

argocd login 10.167.168.156:30938 --username admin --password Jt5yuFrBGJTcoS4c

# 3.更新 admin 密码, 方便下次登录
argocd account update-password --account admin --current-password Jt5yuFrBGJTcoS4c --new-password C2m-12345

3. Create An Application From A Git Repository

# 5. Register A Cluster To Deploy Apps To (Optional)
‎此步骤将群集的凭据注册到 Argo CD，并且仅在部署到外部群集时才需要。
在内部部署时（部署到运行 Argo CD 的同一集群）https://kubernetes.default.svc 应用作应用程序的 K8s API 服务器地址。‎

4. Create An Application From A Git Repository

(1)Creating Apps Via CLI

可以通过 UI 进行创建应用，但为了能通过复制、粘贴快速体验 Argo CD，这里通过 CLI 工具进行创建。

argocd app create guestbook --repo https://github.com/argoproj/argocd-example-apps.git --path guestbook --dest-server https://kubernetes.default.svc --dest-namespace default

参数说明：

- repo, 指定 Git 仓库
- path, 指定部署文件在 Git 仓库中的相对路径
- dest-server, 集群的访问地址
- dest-namespace, 部署到哪个命名空间

(2)Creating Apps Via UI

登陆 Argo CD UI 后，选择 NEW APP 创建 application，选择 EDIT AS AYML:

粘贴以下内容，SAVE 后点击左上 CREATE，当然也可以直接使用 kubectl apply 命令执行以下内容，效果相同。

# 样例
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: javademo
  namespace: Argo CD
  finalizers:
    - resources-finalizer.Argo CD.argoproj.io
spec:
  project: default
  source:
    path: javademo
    repoURL: http://10.39.140.196:10080/gogs/argocd-gitops.git
    targetRevision: HEAD
  destination:
    namespace: apps
    server: https://kubernetes.default.svc
  syncPolicy:
    automated: #自动同步Automatic,或 手动Manual触发同步
      prune: true  #如果选中，Argo 将删除未在 Git 中定义的资源
      selfHeal: true #如果选中，当检测到集群中的偏差时，Argo 会将 Git 中定义的状态强制放入集群中
      allowEmpty: false
    syncOptions:
    - Validate=false #Skip Schema Validation跳过架构验证
    - CreateNamespace=true
    # - PruneLast=true
    # - RespectIgnoreDifferences=true  
    # - ApplyOutOfSyncOnly=true
    # - PrunePropagationPolicy=background # foreground（默认）,background,orphan
    # - Replace=true #资源将使用“kubectl replace/create”命令进行同步，这是一个潜在的破坏性操作，可能会导致资源重新创建。
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

参数说明：

• metadata 字段：指定了应用名称，命名空间必须指定 Argo CD，添加 finalizers 字段可在删除应用时级联删除相关 k8s 资源；
• source 字段：指定了 yaml 文件所在 git 仓库 URL，及要监测的 yaml 文件存放目录，该目录下文件有任何变更 Argo CD 都会自动将其更新部署到 k8s 集群；
• destination 字段：指定监测的 yaml 文件要部署到哪个 k8s 集群及哪个命名空间下；
• syncPolicy 字段：指定自动同步Automatic策略和频率，不配置时需要手动Manual触发同步。

创建应用如下：

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: 'kustomize-helloworld'
  namespace: argocd
  finalizers:
    - resources-finalizer.Argo CD.argoproj.io
spec:
  project: 'default'
  source:
    path: 'examples/helloWorld/base'
    repoURL: 'http://10.34.252.90:90/goip/kustomize.git'
    targetRevision: HEAD
  destination:
    name: ''
    namespace: ''
    server: 'https://kubernetes.default.svc'
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
    - Validate=false
    - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-5NNuUSZJ-1662432898592)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220901151119575.png)]

5. Sync (Deploy) The Application

(1)Syncing via CLI

‎创建guestbook应用程序后，可以查看其状态：‎

argocd app get kustomize-helloworld

[root@hz-sd-qingzhou-dev2-08 ~]# argocd app get kustomize-helloworld
Name:               kustomize-helloworld
Project:            default
Server:             https://kubernetes.default.svc
Namespace:
URL:                https://10.167.168.156:30938/applications/kustomize-helloworld
Repo:               http://10.34.252.90:90/goip/kustomize.git
Target:             HEAD
Path:               examples/helloWorld/base
SyncWindow:         Sync Allowed
Sync Policy:        <none>
Sync Status:        OutOfSync from HEAD (df33b50)
Health Status:      Missing

GROUP  KIND        NAMESPACE    NAME            STATUS     HEALTH   HOOK  MESSAGE
       ConfigMap   csztest-dev  the-map         OutOfSync  Missing
       Service     csztest-dev  the-service     OutOfSync  Missing
apps   Deployment  csztest-dev  the-deployment  OutOfSync  Missing

应用程序状态Sync Status: OutOfSync from HEAD (497c70a)最初处于‎OutOfSync状态，Health Status: Missing因为应用程序尚未部署，并且尚未创建 Kubernetes 资源。18 hours ago (Thu Sep 01 2022 15:20:51 GMT+0800)

若要同步（部署）应用程序，请运行：

argocd app sync kustomize-helloworld

[root@hz-sd-qingzhou-dev2-08 ~]# argocd app get kustomize-helloworld
Name:               kustomize-helloworld
Project:            default
Server:             https://kubernetes.default.svc
Namespace:
URL:                https://10.167.168.156:30938/applications/kustomize-helloworld
Repo:               http://10.34.252.90:90/goip/kustomize.git
Target:             HEAD
Path:               examples/helloWorld/base
SyncWindow:         Sync Allowed
Sync Policy:        <none>
Sync Status:        Synced to HEAD (df33b50)
Health Status:      Healthy

GROUP  KIND        NAMESPACE    NAME            STATUS  HEALTH   HOOK  MESSAGE
       ConfigMap   csztest-dev  the-map         Synced                 configmap/the-map created
       Service     csztest-dev  the-service     Synced  Healthy        service/the-service created
apps   Deployment  csztest-dev  the-deployment  Synced  Healthy        deployment.apps/the-deployment created

此命令从存储库中检索清单并执行清单。kustomize-helloworld应用现在正在运行，你现在可以查看其资源组件、日志、事件和评估的运行状况。‎

Sync Status: Synced to HEAD (497c70a)
Health Status: Healthy

GROUP  KIND        NAMESPACE    NAME            STATUS  HEALTH   HOOK  MESSAGE
       ConfigMap   csztest-dev  the-map         Synced                 configmap/the-map created
       Service     csztest-dev  the-service     Synced  Healthy        service/the-service created
apps   Deployment  csztest-dev  the-deployment  Synced  Healthy        deployment.apps/the-deployment created

LAST SYNC RESULT：Sync OK To 497c70a

Succeeded 3 minutes ago (Fri Sep 02 2022 09:40:49 GMT+0800)

Author:曹少哲 [email protected] -

Comment:Update deployment.yaml

2、9:43直接在git上修改deploy的副本数，由2改成3，查看

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-564qYKb3-1662432898593)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902094554543.png)]

deploy的副本仍然是2个

应用程序状态Sync Status处于‎OutOfSync状态

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Gzmui5lm-1662432898594)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902094803136.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tSaV8QDA-1662432898595)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902094830938.png)]3、点击页面的sync

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rZQrJL0v-1662432898597)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902095024313.png)]

deploy的副本由2个变成3个

修改策略为auto-sync

修改git中的deploy，modify deploy replicas from 3 to 4

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vPF2GawL-1662432898598)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902095249441.png)]

自动同步，deploy由3变成4个

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iSjtQUxS-1662432898599)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902095634808.png)]

modify deploy replicas from 4 to 5 (Fri Sep 02 2022 09:53:08 GMT+0800)–怀疑git时间不准

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-twjl9iC1-1662432898600)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902100107770.png)]

Rollback application

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iDuCnwox-1662432898601)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902100653983.png)]

Auto-Sync needs to be disabled in order for rollback to occur. Are you sure you want to disable auto-sync and rollback application ‘kustomize-helloworld’?

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-LW4E5Atl-1662432898602)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902100951117.png)]

rollback3-4，pod个数不应该是3吗？？？？？

再次rollback

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0DBLbJ9D-1662432898604)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902101238365.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-63XjF5qf-1662432898605)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902101334905.png)]

点击页面sync按钮

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-z1Mezsva-1662432898608)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902101437246.png)]

是否自动根据git上的来更新k8s应用配置 ，取决于SYNC POLICY： Manual Automatic

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tkTZucwp-1662432898608)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220901152402545.png)]

auto-sync

从本地推送文件到git，modify deploy from 5 to 6，一段时间后自动更新

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-o7w8E8l8-1662432898609)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902103811516.png)]

从本地推送文件到git，‘modify deploy from 6 to 4’，点击页面sync

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Z3D1DSxa-1662432898610)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902104147605.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-SItXUlgJ-1662432898612)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902104822374.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CddrAm3n-1662432898613)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902105052639.png)]

—自动等时间

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-daS2zYBl-1662432898614)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902110211182.png)]

---

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-3QNc5Q31-1662432898616)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902110318385.png)]

add cm-extra

[root@hz-sd-qingzhou-dev2-08 ~]# argocd app get kustomize-helloworld
Name: kustomize-helloworld
Project: default
Server: https://kubernetes.default.svc
Namespace:
URL: https://10.167.168.156:30938/applications/kustomize-helloworld
Repo: http://10.34.252.90:90/goip/kustomize.git
Target: HEAD
Path: examples/helloWorld/base
SyncWindow: Sync Allowed
Sync Policy: Automated
Sync Status: Synced to HEAD (b1e6250)
Health Status: Healthy

GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE
ConfigMap csztest-dev the-map Synced configmap/the-map unchanged
Service csztest-dev the-service Synced Healthy service/the-service unchanged
apps Deployment csztest-dev the-deployment Synced Healthy deployment.apps/the-deployment configured

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-y0A6Rbio-1662432898617)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902110946783.png)]

delete cm-extra

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-4vBkztOd-1662432898618)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902132025688.png)]

手工点页面sync

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-nx7ZOjE8-1662432898618)(C:\Users\shaozhe.cao\AppData\Roaming\Typora\typora-user-images\image-20220902132105845.png)]

‘move cm from base to extranal’

直接在git上修改deploy的副本数，由4改成2，不点击GUI，查看

deploy的副本仍是4个

14 minutes ago (Thu Sep 01 2022 15:20:51 GMT+0800)

deploy副本仍是4个，OutOfSync

点击sync后，副本变成2个，状态变成 Synced

[root@hz-sd-qingzhou-dev2-08 ~]# argocd app get kustomize-helloworld
Name:               kustomize-helloworld
Project:            default
Server:             https://kubernetes.default.svc
Namespace:
URL:                https://10.167.168.156:30938/applications/kustomize-helloworld
Repo:               http://10.34.252.90:90/goip/kustomize.git
Target:             HEAD
Path:               examples/helloWorld/base
SyncWindow:         Sync Allowed
Sync Policy:        
Sync Status:        OutOfSync from HEAD (df33b50)
Health Status:      Healthy

GROUP  KIND        NAMESPACE    NAME            STATUS     HEALTH   HOOK  MESSAGE
       ConfigMap   csztest-dev  the-map         Synced                    configmap/the-map unchanged
       Service     csztest-dev  the-service     Synced     Healthy        service/the-service unchanged
apps   Deployment  csztest-dev  the-deployment  OutOfSync  Healthy        deployment.apps/the-deployment configured

(2)Syncing via UI

Delete application

Sync Options - Argo CD - Declarative GitOps CD for Kubernetes (argo-cd.readthedocs.io)

‎选择用于删除应用程序的传播策略Prune Propagation Policy

‎Foreground 前台级联删除

在前台级联删除中，正在被你删除的属主对象首先进入 deletion in progress 状态。 在这种状态下，针对属主对象会发生以下事情：

• Kubernetes API 服务器将对象的 metadata.deletionTimestamp 字段设置为对象被标记为要删除的时间点。
• Kubernetes API 服务器也会将 metadata.finalizers 字段设置为 foregroundDeletion。
• 在删除过程完成之前，通过 Kubernetes API 仍然可以看到该对象。

当属主对象进入删除过程中状态后，控制器删除其依赖对象。控制器在删除完所有依赖对象之后， 删除属主对象。这时，通过 Kubernetes API 就无法再看到该对象。

在前台级联删除过程中，唯一可能阻止属主对象被删除的是那些带有 ownerReference.blockOwnerDeletion=true 字段的依赖对象。 参阅使用前台级联删除 以了解进一步的细节。

Background 后台级联删除

在后台级联删除过程中，Kubernetes 服务器立即删除属主对象，控制器在后台清理所有依赖对象。 默认情况下，Kubernetes 使用后台级联删除方案，除非你手动设置了要使用前台删除， 或者选择遗弃依赖对象。

Non-cascading：only delete the application，but do not cascade delete its resources

只删除应用程序，但不级联删除其资源

argo workflow

参考文档

Getting Started

Install Argo Workflows

#Controller and Server
kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.3.9/install.yaml

#Patch argo-server authentication
kubectl patch deployment \
  argo-server \
  --namespace argo \
  --type='json' \
  -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/args", "value": [
  "server",
  "--auth-mode=server"
]}]'

#Port-forward the UI
kubectl -n argo port-forward deployment/argo-server 2746:2746
kubectl -n argo patch svc argo-server -p '{"spec": {"type": "NodePort"}}'

Install the Argo Workflows CLI(Linux)

# Download the binary
curl -sLO https://github.com/argoproj/argo-workflows/releases/download/v3.3.9/argo-linux-amd64.gz

# Unzip
gunzip argo-linux-amd64.gz

# Make binary executable
chmod +x argo-linux-amd64

# Move binary to path
mv ./argo-linux-amd64 /usr/bin/argo

# Test installation
argo version

Submitting an example workflow

Submit an example workflow (CLI)

argo submit -n argo --watch https://raw.githubusercontent.com/argoproj/argo-workflows/master/examples/hello-world.yaml

The --watch flag used above will allow you to observe the workflow as it runs and the status of whether it succeeds. When the workflow completes, the watch on the workflow will stop.

You can list all the Workflows you have submitted by running the command below:

argo list -n argo

You will notice the Workflow name has a hello-world- prefix followed by random characters. These characters are used to give Workflows unique names to help identify specific runs of a Workflow. If you submitted this Workflow again, the next Workflow run would have a different name.

Using the argo get command, you can always review details of a Workflow run. The output for the command below will be the same as the information shown as when you submitted the Workflow:

argo get -n argo @latest

The @latest argument to the CLI is a short cut to view the latest Workflow run that was executed.

You can also observe the logs of the Workflow run by running the following:

argo logs -n argo @latest

Submit an example workflow (GUI)

• Open a port-forward so you can access the UI:

kubectl -n argo port-forward deployment/argo-server 2746:2746

• Navigate your browser to https://localhost:2746.
• Click + Submit New Workflow and then Edit using full workflow options
• You can find an example workflow already in the text field. Press + Create to start the workflow.