如何动态选择要在 Jenkins 构建中使用的 git 分支

How to dynamically pick a git branch to use in Jenkins build

提问by Ladislav Mrnka

I'm trying to create a new project configuration for Jenkins build server. To simplify what I'm trying to do, I will use only two components to describe the problem.

我正在尝试为 Jenkins 构建服务器创建一个新的项目配置。为了简化我想要做的事情，我将只使用两个组件来描述问题。

ComponentA

组件A

• Change in this component triggers the build of this project on CI server.
• CI server has statically configured branch to monitor for changes and build. Eg. master or develop branch.
• This component contains a configuration file with required version of ComponentB it depends on.

• 此组件的更改会触发此项目在 CI 服务器上的构建。
• CI 服务器具有静态配置的分支来监视更改和构建。例如。掌握或开发分支。
• 此组件包含一个配置文件，其中包含它所依赖的 ComponentB 的所需版本。

ComponentB

组件B

• Changes to this component don't trigger build of this project on CI server (there will be another project to cover development of ComponentB).
• Individual versions of component are tagged
• ComponentA has required version of ComponentB in its configuration file
• CI server does not know what branch (tag) to checkout until configuration file of ComponentA is somehow parsed.

• 对此组件的更改不会触发此项目在 CI 服务器上的构建（将有另一个项目来涵盖 ComponentB 的开发）。
• 组件的各个版本被标记
• ComponentA 在其配置文件中具有所需的 ComponentB 版本
• CI 服务器不知道要检出哪个分支（标签），直到以某种方式解析 ComponentA 的配置文件。

What is the right way to achieve this on Jenkins? I was trying to find out how to add this dynamic behavior of parsing the config file and making Git Plugin to check out branch based on expected version of ComponentB but so far I have no clue.

在 Jenkins 上实现这一目标的正确方法是什么？我试图找出如何添加这种解析配置文件的动态行为，并使 Git 插件根据 ComponentB 的预期版本签出分支，但到目前为止我一无所知。

In the next step I may even want to have wildcards (like 5.3.*) in configuration file so I will have to find a the newest ComponentB's tag matching the wildcard.

在下一步中，我什至可能希望在配置文件中有通配符（如 5.3.*），因此我必须找到与通配符匹配的最新 ComponentB 标签。

EDIT

编辑

Now I see that I simplified my problem too much and due to the simplification, the main limitation is no longer present.

现在我发现我过于简化了我的问题，并且由于简化，主要限制不再存在。

The main limitation is that Component A and B must be built together. It is not possible to build them separately as they form one executable / library and the build script needs source files from both components.

主要限制是组件 A 和 B 必须一起构建。不可能单独构建它们，因为它们形成一个可执行文件/库，并且构建脚本需要来自两个组件的源文件。

If you ask why such a strange configuration, let's give Component A and B some description:

如果你问为什么这么奇怪的配置，让我们给组件 A 和 B 一些描述：

• ComponentA: Native platform specific code
• ComponentB: Native platform independent code

• ComponentA：本机平台特定代码
• ComponentB：独立于本机平台的代码

There may be many Component As - one for each platform, but just single Component B. Merging particular A with B produces full source code for single platform but not every platform may be updated to the latest version of B so it needs to have control over which version of B should be used for built.

可能有许多组件 As - 每个平台一个，但只有一个组件 B。将特定的 A 与 B 合并为单个平台生成完整的源代码，但并非每个平台都可以更新到 B 的最新版本，因此它需要控制应该使用哪个版本的 B 来构建。

采纳答案by zeppelin

One option to achieve what you want is to use the following setup:

实现您想要的一种选择是使用以下设置：

Create two Jenkins jobs:

创建两个 Jenkins 作业：

• "Component A" (automatically triggered on SCM changes)
• "Component B" ("manually" triggered)

• “组件 A”（在 SCM 更改时自动触发）
• “组件 B”（“手动”触发）

Step #1

第1步

Define the branchbuild parameterfor "Component B":

定义“组件 B”的branch构建参数：

Use this parameter as the "Git Plugin" branch specifier:

将此参数用作“Git Plugin”分支说明符：

Now you should be able to manually trigger "Component B" build, by specifying a proper branch (tag) parameter to it, e.g. tags/5.3.0.

现在您应该能够通过为其指定适当的分支（标签）参数来手动触发“组件 B”构建，例如tags/5.3.0.

Step #2

第2步

Add a new "Execute Shell" build step to your "Component A" build, which will extract the "Component B" version from the configuration file in the workspace, and prepare b.propertiesfile with the "Component B" build parameters.

在“组件 A”构建中添加一个新的“执行外壳”构建步骤，这将从工作区的配置文件中提取“组件 B”版本，并准备b.properties带有“组件 B”构建参数的文件。

Install a Parameterized TriggerJenkins plugin, and add a new "Trigger/call builds on other projects" build step to "Component A" job:

安装参数化触发器Jenkins 插件，并在“组件 A”作业中添加一个新的“触发器/调用在其他项目上构建”构建步骤：

Using your b.propertiesfile as the source of build parameters.

使用您的b.properties文件作为构建参数的来源。

Now each time "Component A" is re-build, a new "Component B" build will get triggered, with the target branch/tag as a build parameter.

现在，每次重新构建“组件 A”时，都会触发一个新的“组件 B”构建，并将目标分支/标签作为构建参数。

Adding wildcard support

添加通配符支持

If you want to support wildcard versions, you can use git ls-remotecommand to find the latest tag, like that:

如果你想支持通配符版本，你可以使用git ls-remote命令来查找最新的标签，如下所示：

#B=$(obtain B version from the config file in a usual way)   

LATEST=$(\
    git ls-remote --tags YOUR_REPOSITORY_URL "$B"\
    |cut -d / -f3|sort -r --version-sort|head -1\
)

cat <<EOF > b.properties
    branch=tags/$LATEST
EOF

This will list all the tags, matching "B" version pattern, in the remote "Component B" repository, and save the latest version number in the LATESTvariable.

这将列出远程“组件 B”存储库中与“B”版本模式匹配的所有标签，并将最新版本号保存在LATEST变量中。

Add this to your "Execute Shell" step of the "Component A" job, and it should be able to handle version numbers patterns like: 5.3.*

将此添加到“组件 A”作业的“执行外壳”步骤中，它应该能够处理版本号模式，例如： 5.3.*

The catch is that the shell script will run as the Jenkins daemon user, so it must have proper credentials configured, to access the remote Git repository (e.g. via the ssh pubkey).

问题在于，shell 脚本将作为 Jenkins 守护程序用户运行，因此它必须配置正确的凭据，才能访问远程 Git 存储库（例如，通过 ssh 公钥）。

Alternatively you may want to look into the Credentials Binding Plugin, to reuse the Git credentials stored in Jenkins itself.

或者，您可能需要查看Credentials Binding Plugin，以重用存储在 Jenkins 本身中的 Git 凭证。

Using Jenkins 2.0 style pipeline

使用 Jenkins 2.0 风格的管道

You can also solve the task at hand by using a Jenkins 2.0-style Pipeline, which will allow you to checkout the code for components A and B, into a single workspace, and then apply some common build step to them.

您还可以使用 Jenkins 2.0 样式的Pipeline解决手头的任务，这将允许您将组件 A 和 B 的代码签出到单个工作区中，然后对它们应用一些常见的构建步骤。

Your pipeline could look something like this:

您的管道可能如下所示：

node {

   //Settings
   def credentialsId = '8fd28e34-b04e-4bc5-874a-87f4c0e05a03'    
   def repositoryA = 'ssh://[email protected]/projects/a.git'
   def repositoryB = 'ssh://[email protected]/projects/b.git'

   stage('Checkout component A') {
      git credentialsId: credentialsId , 
      url: repositoryA , branch : "master"
   }

   stage("Resolve and checkout component B") {
      def deps = readProperties file: 'meta.properties'
      echo "Resolved B version = ${deps['b']}"

      dir("module/b") {
           //Clone/Fetch Component B 
           checkout scm:[
                $class: 'GitSCM', 
                userRemoteConfigs: [[url: repositoryB, credentialsId: credentialsId]], 
                branches: [[name: 'refs/tags/*']]
           ], 
           changelog: false, poll: false

           //Checkout the tag, matching deps['b'] pattern     
           sshagent([credentialsId]) {
                sh "git checkout $(git tag -l \"${deps['b']}\" |sort -r --version-sort|head -1)"
           }
      }
   }

   stage("Build A+B") {
        //Apply a common build step
   }

}

Here we use the "readProperties" command, which is part of the Pipeline Utility Steps Pluginto extract the "Component B" version pattern from meta.properties. There are also readYaml, readJSON commands available.

在这里，我们使用“readProperties”命令，它是Pipeline Utility Steps Plugin 的一部分，用于从meta.properties. 还有 readYaml、readJSON 命令可用。

Next we fetch/clone the "Component B", with the changelog: false, poll: falseflags, to prevent it from being registered for the SCM poll, into the "module/b" folder of the current workspace.

接下来，我们将带有changelog: false, poll: false标志的“组件 B”提取/克隆到当前工作区的“module/b”文件夹中，以防止它被注册用于 SCM 轮询。

Then invoke a shell command to select the tag, based on the version pattern, we have obtained above, and checkout it (5.3.* style wildcards should also work).

然后调用一个 shell 命令来选择标签，基于我们在上面获得的版本模式，并检查它（5.3.* 风格的通配符也应该有效）。

The shinvocation, is wrapped in the sshagent, to make it reuse the appropriate credentials from the Jenkins credential store.

该sh调用包含在sshagent 中，以使其重用来自 Jenkins 凭证存储的适当凭证。

回答by Chananel P

Using the Credentials Binding Pluginworked very well for me (also mentioned by @zeppelin)

使用凭据绑定插件对我来说效果很好（@zeppelin 也提到了）

Steps:

脚步：

In the Global credentialssection:

在全局凭据部分：

1. Add Credentialsof the type: "Username with password". This should be the username and password for component B repository git server using HTTPSprotocol (the SSH option is not good for this purpose)

1. Add Credentials类型：“带密码的用户名”。这应该是使用HTTPS协议的组件B存储库git服务器的用户名和密码（SSH选项不适用于此目的）

In your Jenkins job configuration:

在您的 Jenkins 作业配置中：

1. Put component A in the regular Source Code Managementunder the Gitsection all required fields (Repositories, Branches, etc.).
   • It will clearer and cleaner to put the repository in a sub-directory: under Additional Behaviourschoose Check out to a sub-directoryand write: component_a
2. Make sure also to check in Build Triggersthe Build when a change is pushed to GitHub

3. In the Build Environment section tick the Use secret text(s) or file(s)

   • put in Variablesome name: MY_CRED
   • in Credentialschoose the Specific credentialsyou created in step 1.

   

4. Now using the MY_CREDin the Execute shellcode you will have access to the component B repository:

   DIR="component_b"
   if [ "$(ls -A $DIR/.git)" ]; then
       cd $DIR
       git fetch
   else
       git clone https://[email protected]/proj/component_b.git $DIR 
       cd $DIR
   fi
   git show
   

   

   • Note: you will NOT see the user and password in the logs, so it should be safe. you would see: git clone 'https://****@github.com/proj/component_b.git' component_b

5. Do all your parsing of config from component A to get the desired tag: TAG=$(cat ./component_a/config.cfg | grep ... | sed ...)

6. Checkout the desired Tag: cd component_b; git checkout -f $TAG
   • Note: the -fforce tag.
7. Now run the code and test as desired...

2. 将组件 A 放在常规Source Code ManagementGit部分下的所有必填字段（Repositories、Branches等）中。
   • 将存储库放在子目录中会更清晰：在附加行为下选择Check out to a sub-directory并写入：component_a
3. 确保也在检查构建触发器的Build when a change is pushed to GitHub

4. 在构建环境部分勾选 Use secret text(s) or file(s)

   • 输入Variable一些名称：MY_CRED
   • 在Credentials选择您在步骤 1 中创建的特定凭据。

   

5. 现在MY_CRED在Execute shell代码中使用 ，您将可以访问组件 B 存储库：

   DIR="component_b"
   if [ "$(ls -A $DIR/.git)" ]; then
       cd $DIR
       git fetch
   else
       git clone https://[email protected]/proj/component_b.git $DIR 
       cd $DIR
   fi
   git show
   

   

   • 注意：您不会在日志中看到用户和密码，因此应该是安全的。你会看到：git clone 'https://****@github.com/proj/component_b.git' component_b

6. 对组件 A 中的配置进行所有解析以获得所需的标签： TAG=$(cat ./component_a/config.cfg | grep ... | sed ...)

7. 签出所需的标签： cd component_b; git checkout -f $TAG
   • 注意：-f力标签。
8. 现在运行代码并根据需要进行测试...

回答by LeGEC

1 - would adding project Bas a sub repo of project Abe a possible solution ?

1 - 将项目添加为项目B的子仓库A是一种可能的解决方案吗？

2- (if including the full source code for B should really be avoided) : would pushing builds of B to some B_buildsrepo, and adding this repo as a sub-repo of Abe a possible solution ?

2-（如果真的应该避免包含 B 的完整源代码）：将 B 的构建推送到某个存储B_builds库，并将此存储库添加为子存储库A是一种可能的解决方案吗？

---

---

Rationale : one way to make the dependency between Aand Bmore explicit is to represent it inside the dependencies of the repository.

理由：一个方法，使之间的依赖关系A和B更明确的是代表它的储存库的依赖内部。

This would require to add an extra step when managing the Aproject :

这将需要在管理A项目时添加一个额外的步骤：

update `B` sub repo in `A` project, and push this to `A`

each time you produce a new version for B.

每次为B.

However, you would have a clear view, from A, about when the versions of Bwere integrated (e.g : "we only used B 2.0.1starting from A 4.3.2") , and pushing to Awould trigger your usual Jenkins flow.

但是，从 开始，您可以清楚地了解A的版本何时B集成（例如：“我们只使用B 2.0.1从A 4.3.2”开始），并且推到A会触发您通常的 Jenkins 流程。