bash curl 编译重定向页面列表

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/8403937/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 01:08:32  来源:igfitidea点击:

curl to compile a list of redirected pages

bashcurl

提问by Vlueboy

Suppose I have a bash script that goes through a file that contains a list of old URLs that have all been redirected.

假设我有一个 bash 脚本,它遍历一个文件,该文件包含已全部重定向的旧 URL 列表。

curl --location http://destination.comwill process a page by following a redirect. However, I'm interested not in the content, but on where the redirect points so that I can update my records.

curl --location http://destination.com将通过重定向处理页面。但是,我对内容不感兴趣,而对重定向指向的位置感兴趣,以便我可以更新我的记录。

What is the command-line option for curl to output what that new location for the URL is?

curl 输出 URL 的新位置的命令行选项是什么?

回答by chown

You wou want to leave out the --location/-Lflag, and use -w, checking the redirect_urlvariable. curl -w "%{redirect_url}" http://someurl.comshould do it.

您想省略--location/-L标志,并使用-w,检查redirect_url变量。 curl -w "%{redirect_url}" http://someurl.com应该这样做。

Used in a script:

在脚本中使用:

REDIRECT=`curl -w "%{redirect_url}" http://someurl.com`
echo "http://someurl.com redirects to: ${REDIRECT}"


From the curl man page:

curl 手册页

-w, --write-out <format>

Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables. The format can be specified as a literal "string", or you can have curl read the format from a file with "@filename" and to tell curl to read the format from stdin you write "@-".

The variables present in the output format will be substituted by the value or text that curl thinks fit, as described below. All variables are specified as %{variable_name} and to output a normal % you just write them as %%. You can output a newline by using \n, a carriage return with \r and a tab space with \t.

NOTE:The %-symbol is a special symbol in the win32-environment, where all occurrences of % must be doubled when using this option.

The variables available are:

...
redirect_urlWhen an HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect wouldtake you to. (Added in 7.18.2)
...

-w, --write-out <format>

完成传输后,使 curl 在 stdout 上显示信息。格式是一个字符串,可以包含与任意数量的变量混合的纯文本。格式可以指定为文字“字符串”,或者您可以让 curl 从带有“@filename”的文件中读取格式,并告诉 curl 从您写入的 stdin 中读取格式“@-”。

输出格式中存在的变量将被 curl 认为合适的值或文本替换,如下所述。所有变量都指定为 %{variable_name} 并且要输出正常的 % 您只需将它们写为 %%。您可以使用 \n 输出换行符,使用 \r 输出回车符,使用 \t 输出制表符空间。

注意:%-symbol 是 win32-environment 中的特殊符号,使用此选项时,所有出现的 % 都必须加倍。

可用的变量有:

...
redirect_url当 HTTP 请求没有 -L 跟随重定向时,此变量将显示重定向带您到的实际 URL 。(在 7.18.2 中添加)
...

回答by ajreal

This might work (as a starting point)

这可能有效(作为起点)

curl -sI google.com | head -1 | grep 301 | wc -l

回答by Kent

man curl

then

然后

search redirect_url

redirect_urlWhen a HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect would take you to. (Added in 7.18.2)

redirect_url当一个没有 -L 的 HTTP 请求跟随重定向时,这个变量将显示重定向将带你到的实际 URL。(在 7.18.2 中添加)

the variable above is for -w/--write-out <format>

上面的变量是为了 -w/--write-out <format>