BASH CURL:顺序运行时不要关闭请求之间的连接

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/15319770/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 04:48:01  来源:igfitidea点击:

BASH CURL: Don't close connection in between requests when run sequentially

bashhttpcurlpersistent-connection

提问by ecbrodie

I am trying to write a BASH command that uses CURL to send a GET request to two different web pages but uses the same connection. For me, it is like sending a GET request to a login page to authenticate to the server and then the second request mimics the automatic redirect to the home page that would've happened in a web browser (via metarefresh tag). I need to chain the requests because the content of the home page (generated by the server) wil be different for a guest user than an authenticated user.

我正在尝试编写一个 BASH 命令,该命令使用 CURL 将 GET 请求发送到两个不同的网页,但使用相同的连接。对我来说,这就像向登录页面发送一个 GET 请求以对服务器进行身份验证,然后第二个请求模拟了在 Web 浏览器中发生的自动重定向到主页(通过刷新标记)。我需要链接请求,因为主页的内容(由服务器生成)对于访客用户与经过身份验证的用户不同。

I tried this command first based on recommendation from SOF post(assume that the variables $IPand $PORTwere already defined with valid values):

我首先根据SOF 帖子的建议尝试了此命令(假设变量$IP$PORT已使用有效值定义):

curl -u user:pass ${IP}:${PORT}/login.php && curl ${IP}:${PORT}/index.php

However, I always get something like this happening between the end of the first GET and the start of the second:

但是,我总是在第一个 GET 结束和第二个 GET 开始之间发生这样的事情:

* Connection #0 to host 10.0.3.153 left intact
* Closing connection #0

So was the SOF post wrong? Anyways, doing this command will successfully keep the connection open between two requests:

那么 SOF 的帖子是错误的吗?无论如何,执行此命令将成功保持两个请求之间的连接打开:

curl -u user:pass ${IP}:${PORT}/login.php ${IP}:${PORT}/index.php

However, I really would prefer a solution closer to the former command than the latter command. The main reason why is to separate output from the first page versus the second page into two different output files. So I want to do something like:

但是,与后一个命令相比,我真的更喜欢更接近前一个命令的解决方案。主要原因是将第一页和第二页的输出分成两个不同的输出文件。所以我想做一些类似的事情:

curl page1.html > output1 && curl page2.html > output2

Of course, I need to reuse the same connection because the contents of page2.html depends on me also doing a request to page1.html in the same HTTP session.

当然,我需要重用相同的连接,因为 page2.html 的内容取决于我也在同一个 HTTP 会话中对 page1.html 进行请求。

I am also open to solutions that use netcat or wget, BUT NOT PHP!

我也对使用 netcat 或 wget 的解决方案持开放态度,但不支持 PHP!

回答by DrC

Doing curl a.html && curl b.html will necessarily use two TCP (http) connections to fetch the data. Each curl operation is its own process and will open its own connection.

执行 curl a.html && curl b.html 必须使用两个 TCP (http) 连接来获取数据。每个 curl 操作都是它自己的过程,并且会打开它自己的连接。

However, a web site doesn't use the TCP/HTTP connection to track login information. Instead, some kind of token is placed in the session (usually using a cookie) that is passed in subsequent requests to the site. The site validates that token on subsequent requests.

但是,网站不使用 TCP/HTTP 连接来跟踪登录信息。相反,某种令牌被放置在会话中(通常使用 cookie),该令牌在后续请求中传递到站点。该站点在后续请求中验证该令牌。

Curl has an option -c to indicate where cookies should be stored between connections

Curl 有一个选项 -c 来指示应该在连接之间存储 cookie 的位置

curl -c cookiejar -u user:pass login.php && curl -c cookierjar index.php

will be closer. I say closer because many sites don't use the http based authentication supported by the -u option but instead use custom forms and secondly the invocations assume a cookie is used (as opposed to embedding something in javascript or a url path). The latter is likely but I wouldn't count on the first bit.

会更近。我说得更近一些,因为许多站点不使用 -u 选项支持的基于 http 的身份验证,而是使用自定义表单,其次调用假设使用了 cookie(而不是在 javascript 或 url 路径中嵌入某些内容)。后者是可能的,但我不会指望第一位。

回答by Yuriy Pazniak

According to curl manualthe synopsis is the following:

根据curl 手册,概要如下:

curl [options] [URL...]

curl [options] [URL...]

That means that you can specify several urls one after another in the same command. Curl will reuse the handle for each subsequent url:

这意味着您可以在同一命令中一个接一个地指定多个 url。Curl 将为每个后续 url 重用句柄:

curl will attempt to re-use connections for multiple file transfers, so that getting many files from the same server will not do multiple connects / handshakes. This improves speed. Of course this is only done on files specified on a single command lineand cannot be used between separate curl invokes.

curl 将尝试为多个文件传输重用连接,以便从同一服务器获取多个文件不会进行多次连接/握手。这提高了速度。当然,这仅在单个命令行上指定的文件上完成不能在单独的 curl 调用之间使用。

回答by BeniBela

Principally this is what I made my Xidelfor, you can write all requests and actions in a single command call and it will behave similar to a browser keeping cookies, and the connection alive:

这主要是我创建Xidel的目的,您可以在单个命令调用中编写所有请求和操作,它的行为类似于浏览器保持 cookie,并且连接处于活动状态:

xidel http://${IP}/login.php --download page1.html -f '"index.php"' --download page2.html 

Or if there is a link from the first page to the second one, it can directly follow that link:

或者如果有从第一页到第二页的链接,它可以直接点击该链接:

xidel http://${IP}/login.php --download page1.html -f //a --download page2.html 

However, it does not support http authentication or other ports than 80,8080 and 443 yet (the backend would support it, but in-between there is an url validation which rejects it as being an invalid url)

但是,它不支持 http 身份验证或除 80,8080 和 443 之外的其他端口(后端会支持它,但中间有一个 url 验证,将其拒绝为无效 url)