bash 使用 curl 循环遍历 url 的 Shell 脚本
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/16131321/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Shell script using curl to loop through urls
提问by Mena Ortega
I've been trying to create a simple script that will take a list of queries from a .txt file, append the main url variable, then scrape the content and output it to a text file.
我一直在尝试创建一个简单的脚本，该脚本将从 .txt 文件中获取查询列表，附加主 url 变量，然后抓取内容并将其输出到文本文件。
Here's what I have so far:
这是我到目前为止所拥有的：
#!/bin/bash
url="example.com/?q="
for i in $(cat query.txt); do
content=$(curl -o $url $i)
echo $url $i
echo $content >> output.txt
done
list:
列表：
images
news
stuff
other
error log:
错误日志：
curl: (6) Could not resolve host: other; nodename nor servname provided, or not known
example.com/?q= other
If I use this command straight from the command line I get some output into the file:
如果我直接从命令行使用这个命令，我会在文件中得到一些输出：
curl -L http://example.com/?q=other >> output.txt
Ultimately I would like the output to be:
最终我希望输出是：
fetched: http://example.com/?q=other
content: the output of the page
followed by the next query in the list.
回答by Gilles Quenot
Use more quotes !
使用更多引号！
- http://mywiki.wooledge.org/Quotes
- http://mywiki.wooledge.org/Arguments
- http://wiki.bash-hackers.org/syntax/words
- http://mywiki.wooledge.org/Quotes
- http://mywiki.wooledge.org/Arguments
- http://wiki.bash-hackers.org/syntax/words
Try this instead :
试试这个：
url="example.com/?q="
for i in $(cat query.txt); do
content="$(curl -s "$url/$i")"
echo "$content" >> output.txt
done
回答by David George
You've got nested quotes, try something like this:
你有嵌套的引号，试试这样的：
#!/bin/bash
url=https://www.google.fr/?q=
while read query
do
content=$(curl "{$url}${query}")
echo $query
echo $content >> output.txt
done < query.txt
