git 为什么 stash/unstash 在这个 Jenkinsfile 中不起作用？

Question

提问by vextorspace

I've got a Jenkins server running onsite, and it uses a Jenkinsfile to manage a pipeline that uses the parallel test executor plugin to run all my JUnit tests on several agents to speed the test up. We have a blade server we made (way cheaper than buying one!) and it sped our tests from close to 2 hours to 22 minutes. The JUnit plugin works great with parallel tests.

我有一个在现场运行的 Jenkins 服务器，它使用 Jenkinsfile 来管理一个管道，该管道使用并行测试执行器插件在多个代理上运行我的所有 JUnit 测试以加快测试速度。我们制造了一台刀片服务器（比购买一台便宜得多！），它使我们的测试从近 2 小时缩短到 22 分钟。JUnit 插件适用于并行测试。

The Jacoco Plugin however does not. So I am trying to get the coverage files merged to one file so that the Jacoco plugin can publish the coverage results. Stash/unstash is working in the storing of sources but it is not working when I try to stash the different Jacoco output files to unstash them on the master.

然而，Jacoco 插件没有。所以我试图将覆盖文件合并到一个文件中，以便 Jacoco 插件可以发布覆盖结果。Stash/unstash 正在存储源代码，但是当我尝试将不同的 Jacoco 输出文件存储在 master 上时它不起作用。

Any ideas why?

任何想法为什么？

Here is my Jenkinsfile:

这是我的 Jenkinsfile：

#!/usr/bin/env groovy

def branch
def hash

node('remote') {
  sh 'echo starting'

  branch = env.gitlabBranch ?: '**'
  echo "Branch: $branch"

  checkout([$class: 'GitSCM',
        branches: [[name: "$branch"]],
        extensions: [
          [$class: 'PruneStaleBranch'],
          [$class: 'CheckoutOption', timeout: 120],
          [$class: 'CloneOption', depth: 0, noTags: true, shallow: true, timeout: 180]
        ],
        doGenerateSubmoduleConfigurations: false,
        submoduleCfg: [],
        userRemoteConfigs: [[credentialsId: 'gitlabLabptop', url: '[email protected]:protocase/my_project_url.git']]
       ]
      )

  hash = sh (script: 'git rev-parse HEAD', returnStdout: true).trim()

  ### - this stash works fine -###
  stash name: 'sources', includes: '**', excludes: '**/.git,**/.git/**'
}

def numBranches = 9
def splits = splitTests count(numBranches)
def branches = [:]

for (int i = 0; i < splits.size(); i++) {
  def index = i // fresh variable per iteration; i will be mutated

  branches["split${i}"] = {
    timeout(time: 125, unit: 'MINUTES') {
      node('remote') {
    sh 'echo starting a node'
    deleteDir()

    ### - this unstash works fine - ###
    unstash 'sources'

    def exclusions = splits.get(index);
    writeFile file: 'test/exclusions.txt', text: exclusions.join("\n")

    sh 'ant clean'

    sh 'rm -rf build'

    sh 'ant jar'

    sh 'ant -buildfile build-test.xml buildTests'

    sh 'ant -buildfile build-test.xml jenkinsBatch'

    junit 'build/test/results/*.xml'

    sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
    echo "name: coverage$index, unclude jacoco${index}"

       ### - this stash appears to work - ### 
       stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
       echo "stashed"

      }
    }
  }
}

parallel branches


def branchIndecis = 0..numBranches

node('master') {
  if (currentBuild.result != "ABORTED") {

    echo "collecting exec files"

    branchIndecis.each {
      echo "unstash coverage${it}"

      ### !!! this unstash causes an error !!! ###
      unstash name: "coverage${it}"



      echo "make file name"
      def coverageFileName = "build/test/jacoco/jacoco${it}.exec"
      echo "merge file"
      sh "ant -buildfile build-test.xml -Dfile=${coverageFileName} coverageMerge"
    }

    echo "collected exec files"

    step([$class: 'JacocoPublisher',
      execPattern:'build/test/jacoco/jacoco.exec',
      classPattern: 'build/classes',
      sourcePattern: 'src'])

    echo "finishing ${branch} - ${hash}"

  }
}

the output I get is:

我得到的输出是：

[split7] [jdesigner] Running shell script
[split7] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco7.exec
[Pipeline] [split7] echo
[split7] name: coverage7, unclude jacoco7
[Pipeline] [split7] stash
[split7] Stashed 1 file(s)
[Pipeline] [split7] echo
[split7] stashed
[Pipeline] [split7] }
[Pipeline] [split7] // node
[Pipeline] [split7] }
[Pipeline] [split7] // timeout
[Pipeline] [split7] }
[Pipeline] // parallel
[Pipeline] node
Running on eightyeight in /var/jenkins/workspace/jdesigner
[Pipeline] {
[Pipeline] echo
collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] }
[Pipeline] End of Pipeline
Finished: FAILURE

[edit] the stash for coverage0 is

[编辑]coverage0 的藏匿处是

[split0] Recording test results
[Pipeline] [split0] sh
[split0] [jdesigner] Running shell script
[split0] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco0.exec
[Pipeline] [split0] echo
[split0] name: coverage0, include jacoco0
[Pipeline] [split0] stash
[split0] Stashed 1 file(s)
[Pipeline] [split0] echo
[split0] stashed
[Pipeline] [split0] }
[Pipeline] [split0] // node
[Pipeline] [split0] }
[Pipeline] [split0] // timeout
[Pipeline] [split0] }
[split3]     [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec
[split3]     [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec

note the line

注意这条线

[split0] name: coverage0, include jacoco0

is just my echo statement where I echo the name from this part of the script:

只是我的 echo 语句，我在脚本的这一部分回显了名称：

    sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
    echo "name: coverage$index, include jacoco${index}"

    stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
    echo "stashed"

Note the actual stashing is not done on the node, it is listed as pipeline even though it is done on a remote node. I've seen things that indicated the stash is done on the master but not really where that directory resides.

请注意，实际的存储不是在节点上完成的，它被列为管道，即使它是在远程节点上完成的。我已经看到一些事情表明存储是在主服务器上完成的，但并不是该目录所在的位置。

[[FURTHER EDIT]] - thanks to eis for the recommendations.

[[进一步编辑]] - 感谢 eis 的建议。

The jobs/jdesigner/builds/1639/stashes/ directory on master has coverage#.tar.gz files that include the appropriate jacoco#.exec files. When I put a try catch around the unstash:

master 上的jobs/jdesigner/builds/1639/stashes/ 目录有coverage#.tar.gz 文件，其中包括适当的jacoco#.exec 文件。当我在 unstash 周围放置一个 try catch 时：

try {
    unstash name: "coverage${it}"
} catch (error) {
    echo "error unstashing: ${error}"
}

the output I get is:

我得到的输出是：

collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] echo
error unstashing: java.io.NotSerializableException: groovy.lang.IntRange
[Pipeline] echo
make file name

Answer 1

采纳答案by eis

TLDR: this was a case of this problem where iterating style caused the issue, since key itused wasn't Serializable.

TLDR：这是迭代样式导致问题的一个案例，因为it使用的密钥不是Serializable.

Thing making this hard to debug was that the error message wasn't properly reported, possibly due to this issue. Catching the exception in code and "manual" reporting fixed that.

使这个难以调试的事情是错误消息没有正确报告，可能是由于这个问题。捕获代码中的异常和“手动”报告修复了这个问题。

Actual issue was fixed by using Serializablekeys.

实际问题已通过使用Serializable密钥修复。

Longer version:

更长的版本：

Since in your example this works:

因为在你的例子中这有效：

node('remote') {
    ### - this stash works fine -###
    stash name: 'sources', includes: '**', excludes: '**/.git,**/.git/**'
}
node('remote') {    
    ### - this unstash works fine - ###
    unstash 'sources'
}

But this doesn't:

但这不会：

node('remote') {

   ### - this stash appears to work - ### 
   stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
   echo "stashed"

}
node('master') {
   echo "unstash coverage${it}"

   ### !!! this unstash causes an error !!! ###
   unstash name: "coverage${it}"
}

I initially thought the working one is stashed and unstashed on your remote node, while the non-working one is stashed on your remote node but you try to unstash it on your master node (where it naturally won't be found).

我最初认为工作的一个是在你的远程节点上隐藏和取消的，而非工作的一个是在你的远程节点上，但你试图在你的主节点上取消它（它自然不会被找到）。

However, that wasn't the case. According to this,

然而，事实并非如此。根据此，

When you stash a file on a slave, the files are send to the master. The files will be stored in the Job folder, in the associated build folder under the stash folder. Each stash will be stored as a tar file. These files are deleted at the end of the build.

当您在从站上存储文件时，文件将发送到主站。这些文件将存储在 Job 文件夹中，位于 stash 文件夹下的关联构建文件夹中。每个 stash 都将存储为一个 tar 文件。这些文件在构建结束时被删除。

So master-remote separation shouldn't make a difference. Additionally, if it was about stash not being found, you can see from the sourcesthat it would fail with "No such saved stash ‘" + name + "', since according to AbortException javadoc"When this exception is caught, the specified message will be reported.". That is clearly not happening.

所以主远程分离不应该有什么不同。此外，如果是关于没有找到 stash，您可以从来源中看到它会失败"No such saved stash ‘" + name + "'，因为根据AbortException javadoc“当这个异常被捕获时，将报告指定的消息。”。这显然不会发生。

Instead, one should debug using a try-catch block to find out what is the real exception that is breaking the build.

相反，应该使用 try-catch 块进行调试，以找出破坏构建的真正异常是什么。

As to why it's not reported properly by default, there is this issue: "Serialization error at end of flow not reported properly in build log, only Jenkins log". The bug report claims its "fixed" but apparently only because on new versions, sometest of this behaviour didn't trigger the problem, so it might still exist.

至于为什么默认情况下没有正确报告，有这个问题：“构建日志中没有正确报告流结束时的序列化错误，只有 Jenkins 日志”。错误报告声称其“已修复”，但显然只是因为在新版本上，对此行为的某些测试并未触发问题，因此它可能仍然存在。

With the error message catched, one could see that the problem was this- we were trying to serialize an unserializable key when we were passing it on.

捕获到错误消息后，可以看出问题是这样的——当我们传递它时，我们试图序列化一个不可序列化的键。

Answer 2

回答by Nick

Two possibilities strike me when using parallel processes in Jenkins:

在 Jenkins 中使用并行进程时，有两种可能性让我印象深刻：

You might be trying to unstash in one (or more) processes before stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"is complete on the masternode
You might have name contention between the processes.

stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"在master节点上完成之前，您可能会尝试在一个（或多个）进程中取消隐藏
您可能在进程之间存在名称争用。

To explain (2):

解释（2）：

Process 1 creates a stash called stashed_files

进程 1 创建一个名为 stashed_files

Process 2 stashes under the same name, stashed_files, then successfully unstashes. stashed_filesis deleted.

进程 2 以相同的名称stashed_files隐藏，然后成功取消隐藏。stashed_files被删除。

Process 1 attempts to unstash stashed_files. It gets an error during unstashing because stashed_fileswas deleted by process 2.

进程 1 尝试取消隐藏stashed_files. 由于stashed_files已被进程 2 删除，因此在取消存储期间出现错误。

Some useful Groovy code to get around this problem can be found in this question.

一些有用的Groovy代码来解决这个问题，可以发现在这个问题上。

git 为什么 stash/unstash 在这个 Jenkinsfile 中不起作用？

提问by vextorspace

采纳答案by eis

回答by Nick

相关推荐

最近更新

标签

git 为什么 stash/unstash 在这个 Jenkinsfile 中不起作用？

提问by vextorspace

采纳答案by eis

回答by Nick

相关推荐

如何授予 Git 'ForcePush' 权限以将 bfg-repo-cleaner 结果推送到 VSTS？

Git 提交而无需暂存

git 在 Jenkins 管道中签出标签

使用来自 Git 的代码运行 Jenkins 管道

相关推荐

最近更新

标签