json 如何在 Jenkins 中处理 github webhook 负载？

Question

提问by Grant

I'm currently triggering my Jenkins builds through a GitHub webhook. How would I parse the JSON payload? If I try to parameterize my build and use the $payload variable, the GitHub webhook fails with the following error:

我目前正在通过 GitHub webhook 触发我的 Jenkins 构建。我将如何解析 JSON 有效负载？如果我尝试参数化我的构建并使用 $payload 变量，GitHub webhook 将失败并显示以下错误：

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 400 This page expects a form submission</title>
</head>
<body><h2>HTTP ERROR 400</h2>
<p>Problem accessing /job/Jumph-CycleTest/build. Reason:
<pre>    This page expects a form submission</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                
<br/>                                                

</body>
</html>

How can I get my GitHub webhook to work with a parameterized Jenkins build, and how could I then parse the webhook payload to use certain lines, such as the username of the committer, as conditionals in the build?

如何让我的 GitHub webhook 与参数化的 Jenkins 构建一起工作，然后我如何解析 webhook 有效负载以使用某些行（例如提交者的用户名）作为构建中的条件？

Answer 1

回答by killthrush

There are a few tricks to get this to work, and I found the (now defunct) chloky.comblog post to be helpful for most of it. Since it sounds like you've gotten the webhook communicating with your Jenkins instance at least, I'll skip over those steps for now. But, if you want more detail, just scroll past the end of my answer to see the content I was able to salvage from chloky.com- I do not know the original author and the information might be out of date but I did find it helpful.

有一些技巧可以chloky.com让它发挥作用，我发现（现已不存在的）博客文章对其中的大部分内容都有帮助。由于听起来您至少已经让 webhook 与您的 Jenkins 实例进行通信，因此我现在将跳过这些步骤。但是，如果您想了解更多详细信息，只需滚动到我的答案末尾即可查看我能够从中挽救的内容chloky.com- 我不知道原作者，信息可能已过时，但我确实觉得它很有帮助。

So to summarize, you can do the following to deal with the payload:

总而言之，您可以执行以下操作来处理有效负载：

Set up a string parameter called "payload" in your Jenkins job. If you are planning on manually running the build, it might be a good idea to give it a default JSON document at some point but you don't need one right now. This parameter name appears to be case-sensitive (I'm using Linux so that's no surprise...)
Set up the webhook in github to use the buildWithParametersendpoint instead of the buildendpoint, i.e. http://<<yourserver>>/job/<<yourjob>>/buildWithParameters?token=<<yourtoken>>
Configure your webhook to use application/x-www-form-encodedinstead of application/json. The former approach packs the JSON data in a form variable called "payload", which is presumably how Jenkins can assign it to an environment variable. The application/jsonapproach just POSTs raw JSON which does not seem to be mappable to anything (I couldn't get it to work). You can see the difference by pointing your webhook to something like requestbinand inspecting the results.
At this point, you should get your $payloadvariable when you kick off the build. To parse the JSON, I highly recommend installing jqon your Jenkins server and try out some of the parsing syntax here. JQ is especially nice because it's cross-platform.
From here, just parse what you need from the JSON into other environment variables. Combined with conditional build steps, this could give you a lot of flexibility.

在您的 Jenkins 作业中设置一个名为“payload”的字符串参数。如果您计划手动运行构建，那么在某个时候为其提供一个默认的 JSON 文档可能是个好主意，但您现在不需要它。此参数名称似乎区分大小写（我使用的是 Linux，所以这并不奇怪......）
在github中设置webhook使用buildWithParameters端点代替build端点，即 http://<<yourserver>>/job/<<yourjob>>/buildWithParameters?token=<<yourtoken>>
将您的 webhook 配置为使用application/x-www-form-encoded而不是application/json。前一种方法将 JSON 数据打包在一个名为“payload”的表单变量中，这大概是 Jenkins 可以将其分配给环境变量的方式。该应用程序/ JSON的办法只有帖子的原始JSON这似乎没有可映射到任何东西（我无法得到它的工作）。您可以通过将 webhook 指向requestbin 之类的内容并检查结果来查看差异。
此时，您应该在开始构建时获得$payload变量。要解析 JSON，我强烈建议在您的 Jenkins 服务器上安装jq并在此处尝试一些解析语法。JQ 特别好，因为它是跨平台的。
从这里开始，只需将您需要的 JSON 内容解析为其他环境变量。结合有条件的构建步骤，这可以为您提供很大的灵活性。

Hope this helps!

希望这可以帮助！

EDIThere's what I could grab from the original blog posts at http://chloky.com/tag/jenkins/, which has been dead for a while. Hopefully this content is also useful for someone.

编辑这里是我可以从原始博客文章中获取的内容，该文章http://chloky.com/tag/jenkins/已经死了一段时间。希望此内容对某人也有用。

Post #1 - July 2012

帖子 #1 - 2012 年 7 月

Github provides a nice way to fire off notifications to a CI system like jenkins whenever a commit is made against a repository. This is really useful for kicking off build jobs in jenkins to test the commits that were just made on the repo. You simply need to go to the administration section of the repository, click on service hooks on the left, click ‘webhook URLs' at the top of the list, and then enter the URL of the webhook that jenkins is expecting (look at this jenkins pluginfor setting up jenkins to receive these hooks from github).

Github 提供了一种很好的方式，可以在对存储库进行提交时向 jenkins 等 CI 系统发出通知。这对于在 jenkins 中启动构建作业以测试刚刚在 repo 上进行的提交非常有用。您只需要进入存储库的管理部分，单击左侧的服务钩子，单击列表顶部的“webhook URLs”，然后输入 jenkins 期望的 webhook 的 URL（看看这个 jenkins用于设置 jenkins 以从 github 接收这些钩子的插件）。

Recently though, I was looking for a way to make a webhook fire when a pull request is made against a repo, rather than when a commit is made to the repo. This is so that we could have jenkins run a bunch of tests on the pull request, before deciding whether to merge the pull request in – useful for when you have a lot of developers working on their own forks and regularly submitting pull requests to the main repo.

不过最近，我一直在寻找一种方法来在对 repo 发出拉取请求时触发 webhook，而不是在对 repo 进行提交时。这样我们就可以让 jenkins 对拉取请求运行一堆测试，然后再决定是否合并拉取请求——当你有很多开发人员在他们自己的分叉上工作并定期向主提交拉取请求时很有用回购。

It turns out that this is not as obvious as one would hope, and requires a bit of messing about with the github API.

事实证明，这并不像人们希望的那么明显，并且需要对 github API 进行一些处理。

By default, when you configure a github webhook, it is configured to only fire when a commit is made against a repo. There is no easy way to see, or change, this in the github web interface when you set up the webhook. In order to manipulate the webhook in any way, you need to use the API.

默认情况下，当您配置 github webhook 时，它被配置为仅在对 repo 进行提交时触发。当您设置 webhook 时，没有简单的方法可以在 github web 界面中查看或更改它。为了以任何方式操作 webhook，您需要使用 API。

To make changes on a repo via the github API, we need to authorize ourselves. We're going to use curl, so if we wanted to we could pass our username and password each time, like this:

要通过 github API 对 repo 进行更改，我们需要授权自己。我们将使用 curl，所以如果我们愿意，我们可以每次都传递我们的用户名和密码，如下所示：

# curl https://api.github.com/users/mancdaz --user 'mancdaz'
Enter host password for user 'mancdaz':

Or, and this is a much better option if you want to script any of this stuff, we can grab an oauth token and use it in subsequent requests to save having to keep entering our password. This is what we're going to do in our example. First we need to create an oauth authorization and grab the token:

或者，如果你想编写任何这些东西的脚本，这是一个更好的选择，我们可以获取一个 oauth 令牌并在后续请求中使用它，从而不必继续输入我们的密码。这就是我们在示例中要做的事情。首先，我们需要创建一个 oauth 授权并获取令牌：

curl https://api.github.com/authorizations --user "mancdaz" \
--data '{"scopes":["repo"]}' -X POST

You will be returned something like the following:

您将返回如下内容：

{
   "app":{
      "name":"GitHub API",
      "url":"http://developer.github.com/v3/oauth/#oauth-authorizations-api"
   },
   "token":"b2067d190ab94698a592878075d59bb13e4f5e96",
   "scopes":[
      "repo"
   ],
   "created_at":"2012-07-12T12:55:26Z",
   "updated_at":"2012-07-12T12:55:26Z",
   "note_url":null,
   "note":null,
   "id":498182,
   "url":"https://api.github.com/authorizations/498182"
}

Now we can use this token in subsequent requests for manipulating our github account via the API. So let's query our repo and find the webhook we set up in the web interface earlier:

现在我们可以在后续请求中使用这个令牌来通过 API 操作我们的 github 帐户。所以让我们查询我们的 repo 并找到我们之前在 web 界面中设置的 webhook：

# curl  https://api.github.com/repos/mancdaz/mygithubrepo/hooks?access_token=b2067d190ab94698592878075d59bb13e4f5e96
[
  {
    "created_at": "2012-07-12T11:18:16Z",
    "updated_at": "2012-07-12T11:18:16Z",
    "events": [
      "push"
    ],
    "last_response": {
      "status": "unused",
      "message": null,
      "code": null
    },
    "name": "web",
    "config": {
      "insecure_ssl": "1",
      "content_type": "form",
      "url": "http://jenkins-server.chloky.com/post-hook"
    },
    "id": 341673,
    "active": true,
    "url": "https://api.github.com/repos/mancdaz/mygithubrepo/hooks/341673"
  }
]

Note the important bit from that json output:

请注意该 json 输出中的重要部分：

"events": [
      "push"
    ]

This basically says that this webhook will only trigger when a commit (push) is made to the repo. The github API documentation describes numerous different event types that can be added to this list – for our purposes we want to add pull_request, and this is how we do it (note that we get the id of the webhook from the json output above. If you have multiple hooks defined, your output will contain all these hooks so be sure to get the right ID):

这基本上是说这个 webhook 只会在对 repo 进行提交（推送）时触发。github API 文档描述了许多可以添加到此列表中的不同事件类型——出于我们的目的，我们想要添加 pull_request，这就是我们的做法（注意，我们从上面的 json 输出中获得了 webhook 的 id。如果您定义了多个钩子，您的输出将包含所有这些钩子，因此请确保获得正确的 ID）：

# curl  https://api.github.com/repos/mancdaz/mygithubrepo/hooks/341673?access_token=b2067d190ab94698592878075d59bb13e4f5e96 -X PATCH --data '{"events": ["push", "pull_request"]}'
{
  "created_at": "2012-07-12T11:18:16Z",
  "updated_at": "2012-07-12T16:03:21Z",
  "last_response": {
    "status": "unused",
    "message": null,
    "code": null
  },
  "events": [
    "push",
    "pull_request"
  ],
  "name": "web",
  "config": {
    "insecure_ssl": "1",
    "content_type": "form",
    "url": "http://jenkins-server.chloky.com/post-hook"
  },
  "id": 341673,
  "active": true,
  "url": "https://api.github.com/repos/mancdaz/mygithubrepo/hooks/341673"
}

See!

看！

"events": [
    "push",
    "pull_request"
  ],

This webhook will now trigger whenever either a commit OR a pull request is made against our repo. Exactly what you do in your jenkins/with this webhook is up to you. We use it to kick off a bunch of integration tests in jenkins to test the proposed patch, and then actually merge and close (again using the API) the pull request automatically. Pretty sweet.

现在，只要对我们的存储库进行提交或拉取请求，就会触发此 webhook。您在 jenkins 中/使用此 webhook 做什么完全取决于您。我们使用它在 jenkins 中启动一堆集成测试来测试提议的补丁，然后实际合并并自动关闭（再次使用 API）拉取请求。蛮甜的。

Post #2 - September 2012

帖子 #2 - 2012 年 9 月

In an earlier post, I talked about configuring the github webhook to fire on a pull request, rather than just a commit. As mentioned, there are many events that happen on a github repo, and as per the github documentation, a lot of these can be used to trigger the webhook.

在之前的一篇文章中，我谈到了配置 github webhook 以触发拉取请求，而不仅仅是提交。如前所述，在 github 存储库上发生了许多事件，根据 github 文档，其中很多可用于触发 webhook。

Regardless of what event you decide to trigger on, when the webhook fires from github, it essentially makes a POST to the URL configured in the webhook, including a json payload in the body. The json payload contains various details about the event that caused the webhook to fire. An example payload that fired on a simple commit can be seen here:

无论您决定触发什么事件，当 webhook 从 github 触发时，它本质上都会对 webhook 中配置的 URL 进行 POST，包括正文中的 json 负载。json 负载包含有关导致 webhook 触发的事件的各种详细信息。可以在此处看到在简单提交上触发的示例有效负载：

payload
{
   "after":"c04a2b2af96a5331bbee0f11fe12965902f5f571",
   "before":"78d414a69db29cdd790659924eb9b27baac67f60",
   "commits":[
      {
         "added":[
            "afile"
         ],
         "author":{
            "email":"[email protected]",
            "name":"Darren Birkett",
            "username":"mancdaz"
         },
         "committer":{
            "email":"[email protected]",
            "name":"Darren Birkett",
            "username":"mancdaz"
         },
         "distinct":true,
         "id":"c04a2b2af96a5331bbee0f11fe12965902f5f571",
         "message":"adding afile",
         "modified":[

         ],
         "removed":[

         ],
         "timestamp":"2012-09-03T02:35:59-07:00",
         "url":"https://github.com/mancdaz/mygithubrepo/commit/c04a2b2af96a5331bbee0f11fe12965902f5f571"
      }
   ],
   "compare":"https://github.com/mancdaz/mygithubrepo/compare/78d414a69db2...c04a2b2af96a",
   "created":false,
   "deleted":false,
   "forced":false,
   "head_commit":{
      "added":[
         "afile"
      ],
      "author":{
         "email":"[email protected]",
         "name":"Darren Birkett",
         "username":"mancdaz"
      },
      "committer":{
         "email":"[email protected]",
         "name":"Darren Birkett",
         "username":"mancdaz"
      },
      "distinct":true,
      "id":"c04a2b2af96a5331bbee0f11fe12965902f5f571",
      "message":"adding afile",
      "modified":[

      ],
      "removed":[

      ],
      "timestamp":"2012-09-03T02:35:59-07:00",
      "url":"https://github.com/mancdaz/mygithubrepo/commit/c04a2b2af96a5331bbee0f11fe12965902f5f571"
   },
   "pusher":{
      "email":"[email protected]",
      "name":"mancdaz"
   },
   "ref":"refs/heads/master",
   "repository":{
      "created_at":"2012-07-12T04:17:51-07:00",
      "description":"",
      "fork":false,
      "forks":1,
      "has_downloads":true,
      "has_issues":true,
      "has_wiki":true,
      "name":"mygithubrepo",
      "open_issues":0,
      "owner":{
         "email":"[email protected]",
         "name":"mancdaz"
      },
      "private":false,
      "pushed_at":"2012-09-03T02:36:06-07:00",
      "size":124,
      "stargazers":1,
      "url":"https://github.com/mancdaz/mygithubrepo",
      "watchers":1
   }
}

This entire payload gets passed in the POST requests as a single parameter, with the imaginative title payload. It contains a ton of information about the event that just happened, all or any of which can be used by jenkins when we build jobs after the trigger. In order to use this payload in Jenkins, we have a couple of options. I discuss one below.

整个有效负载作为单个参数在 POST 请求中传递，并带有富有想象力的 title payload。它包含大量关于刚刚发生的事件的信息，当我们在触发器之后构建作业时，jenkins 可以使用所有或任何信息。为了在 Jenkins 中使用这个有效载荷，我们有几个选择。我在下面讨论一个。

Getting the $payload

获取 $payload

In jenkins, when creating a new build job, we have the option of specifying the names of parameters that we expect to pass to the job in the POST that triggers the build. In this case, we would pass a single parameter payload, as seen here:

在 jenkins 中，当创建新的构建作业时，我们可以选择指定我们希望在触发构建的 POST 中传递给作业的参数名称。在这种情况下，我们将传递一个参数payload，如下所示：

Passing parameters to a jenkins build job

将参数传递给 jenkins 构建作业

Further down in the job configuration, we can specify that we would like to be able to trigger the build remotely (ie. that we want to allow github to trigger the build by posting to our URL with the payload):

在作业配置的进一步下方，我们可以指定我们希望能够远程触发构建（即，我们希望允许 github 通过将有效负载发布到我们的 URL 来触发构建）：

Then, when we set up the webhook in our github repo (as described in the first post), we give it the URL that jenkins tells us to:

然后，当我们在 github 存储库中设置 webhook 时（如第一篇文章所述），我们给它提供 jenkins 告诉我们的 URL：

You can't see it all in the screencap, but the URL I specified for the webhook was the one that jenkins told me to:

您无法在屏幕截图中看到所有内容，但我为 webhook 指定的 URL 是 jenkins 告诉我的：

http://jenkins-server.chloky.com:8080/job/mytestbuild//buildWithParameters?token=asecuretokenNow, when I built my new job in jenkins, for the purposes of this test I simply told it to echo out the contents of the ‘payload' parameter (which is available in paramterized builds as a shell variable of the same name), using a simple script:

http://jenkins-server.chloky.com:8080/job/mytestbuild//buildWithParameters?token=asecuretoken现在，当我在 jenkins 中构建我的新工作时，为了这个测试的目的，我只是告诉它回显“payload”参数的内容（它在参数化构建中作为同名的 shell 变量可用），使用一个简单的脚本：

#!/bin/bash

echo "the build worked! The payload is $payload"

Now to test the whole thing we simply have to make a commit to our repo, and then pop over to jenkins to look at the job that was triggered:

现在要测试整个事情，我们只需要对我们的 repo 进行提交，然后跳到 jenkins 查看触发的作业：

mancdaz@chloky$ (git::master)$ touch myfile

mancdaz@chloky$ (git::master) git add myfile

mancdaz@chloky$ (git::master) git commit -m 'added my file'
[master 4810490] added my file
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 myfile

mancdaz@chloky$ (git::master) git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 232 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
To [email protected]:mancdaz/mygithubrepo.git
 c7ecafa..4810490 master -> master

And over in our jenkins server, we can look at the console output of the job that was triggered, and lo and behold there is our ‘payload' contained in the $payload variable and available for us to consume:

在我们的 jenkins 服务器中，我们可以查看被触发的作业的控制台输出，瞧，$payload 变量中包含我们的“有效负载”，可供我们使用：

So great, all the info about our github event is here. and fully available in our jenkins job! True enough, it's in a big json blob, but with a bit of crafty bash you should be good to go.

太好了，关于我们的 github 活动的所有信息都在这里。并且在我们的 jenkins 工作中完全可用！确实，它在一个大的 json blob 中，但是使用一些狡猾的 bash 你应该很高兴。

Of course, this example used a simple commit to demonstrate the principles of getting at the payload inside jenkins. As we discussed in the earlier post, a commit is one of many events on a repo that can trigger a webhook. What you do inside jenkins once you've triggered is up to you, but the real fun comes when you start interacting with github to take further actions on the repo (post comments, merge pull requests, reject commits etc) based on the results of your build jobs that got triggered by the initial event.

当然，这个例子使用了一个简单的提交来演示获取 jenkins 内部有效负载的原理。正如我们在之前的文章中所讨论的，提交是可以触发 webhook 的存储库中的众多事件之一。一旦触发，你在 jenkins 中做什么取决于你，但是当你开始与 github 交互以根据结果对 repo 采取进一步的操作（发表评论、合并拉取请求、拒绝提交等）时，真正的乐趣就来了由初始事件触发的构建作业。

Look out for a subsequent post where I tie it all together and show you how to process, run tests for, and finally merge a pull request if successful – all automatically inside jenkins. Automation is fun!

请留意后续的帖子，我将所有内容联系在一起，并向您展示如何处理、运行测试，并在成功时最终合并拉取请求——所有这些都在 jenkins 中自动完成。自动化很有趣！

Answer 2

回答by Tomas Bjerre

There is a Generic Webhook Triggerplugin that can contribute values from the post content to the build.

有一个Generic Webhook Trigger插件可以将帖子内容中的值贡献给构建。

If the post content is:

如果帖子内容是：

{
   "app":{
      "name":"GitHub API",
      "url":"http://developer.github.com/v3/oauth/#oauth-authorizations-api"
   }
}

You can configure it like this:

您可以像这样配置它：

And when triggering with some post content:

当用一些帖子内容触发时：

curl -v -H "Content-Type: application/json" -X POST -d '{ "app":{ "name":"GitHub API", "url":"http://developer.github.com/v3/oauth/" }}' http://localhost:8080/jenkins/generic-webhook-trigger/invoke?token=sometoken

It will resolv variables and make them available in the build job.

它将解析变量并使它们在构建作业中可用。

{  
   "status":"ok",
   "data":{  
      "triggerResults":{  
         "free":{  
            "id":2,
            "regexpFilterExpression":"",
            "regexpFilterText":"",
            "resolvedVariables":{  
               "app_name":"GitHub API",
               "everything_app_url":"http://developer.github.com/v3/oauth/",
               "everything":"{\"app\":{\"name\":\"GitHub API\",\"url\":\"http://developer.github.com/v3/oauth/\"}}",
               "everything_app_name":"GitHub API"
            },
            "searchName":"",
            "searchUrl":"",
            "triggered":true,
            "url":"queue/item/2/"
         }
      }
   }
}

json 如何在 Jenkins 中处理 github webhook 负载？

提问by Grant

回答by killthrush

Post #1 - July 2012

帖子 #1 - 2012 年 7 月

Post #2 - September 2012

帖子 #2 - 2012 年 9 月

Getting the $payload

获取 $payload

Passing parameters to a jenkins build job

将参数传递给 jenkins 构建作业

回答by Tomas Bjerre

相关推荐

最近更新

标签

json 如何在 Jenkins 中处理 github webhook 负载？

提问by Grant

回答by killthrush

Post #1 - July 2012

帖子 #1 - 2012 年 7 月

Post #2 - September 2012

帖子 #2 - 2012 年 9 月

Getting the $payload

获取 $payload

Passing parameters to a jenkins build job

将参数传递给 jenkins 构建作业

回答by Tomas Bjerre

相关推荐

json 使用新 API v3 获取 youtube 视频信息

在 Excel VBA 中解析 JSON

将 JSON 提要自动解析为 MS Access

如何使用groovy解析json

相关推荐

最近更新

标签