bash Shell：连接到网站并访问字段

Shell: Connecting to a website and accessing a field

提问by Chris

I want to write a script that takes an argument which is text, opens a connection to a specific website and input the arg into text field using the field's ID. Is this possible? How would I do this? I'm a total shell noob

我想编写一个脚本，它接受一个文本参数，打开与特定网站的连接，并使用字段的 ID 将 arg 输入到文本字段中。这可能吗？我该怎么做？我是一个彻头彻尾的贝壳菜鸟

Edit:

编辑：

Exact flow:

准确流量：

- start script with string
- input string into text field on web page
- click form button
- wait for processing
- click hyperlink

回答by Nejc

If you know exactly which field you need to fill, then this can be done using lynx. Assume you get the string Swith your script as an input argument. Then you create a command script, which will guide lynxthrough its behaviour.

如果您确切知道需要填写哪个字段，则可以使用lynx. 假设您S使用脚本获取字符串作为输入参数。然后您创建一个命令脚本，它将指导lynx其行为。

For example, suppose S=foo, and your field is the second field in the web page. After that, there are two more fields, and then the submit button. After that you wait for the page to load and click the hyperlink (after that you exit). The web page is www.something.com.

例如，假设S=foo，并且您的字段是网页中的第二个字段。之后，还有两个字段，然后是提交按钮。之后，您等待页面加载并单击超链接（然后退出）。该网页是 www.something.com。

The command script would be in a file bar.txt:

命令脚本将在文件 bar.txt 中：

key <tab> //get to first field
key <tab> //get to second field
key f     //input f
key o     //input o
key o     //input o
key <tab> //get to third field
key <tab> //get to fourth field
key <tab> //get to sumbit button
key ^J    //click submit and wait for load
key <tab> //get to hyperlink
key ^J    //click hyperlink and wait for load
key Q     //exit
key y     //confirm exit

The main command would then be lynx www.something.com -accept_all_cookies -cmd_script=bar.txt

主要命令将是 lynx www.something.com -accept_all_cookies -cmd_script=bar.txt

Now all you need to do is dynamically create the input string.

现在您需要做的就是动态创建输入字符串。

#!/bin/bash
script=bar.txt
input=
webpage=www.something.com
len=${#input}
echo 'key <tab>' > $script
echo 'key <tab>' >> $script
for i in `echo $input|fold -w1` 
do
    echo 'key '$i >> $script
done
echo 'key <tab>' >> $script
echo 'key <tab>' >> $script
echo 'key <tab>' >> $script
echo 'key ^J' >> $script
echo 'key <tab>' >> $script
echo 'key ^J' >> $script
echo 'key Q' >> $script
echo 'key y' >> $script

lnyx $webpage -accept_all_cookies -cmd_script=bar.txt

Now all you need to do is save the script, modify it to be executable and call it ./script your_string

现在您需要做的就是保存脚本，将其修改为可执行并调用它 ./script your_string

回答by Fredrik Pihl

To get you started, here is my script to order today's lunch from our local canteen:

为了让您开始，这是我从当地食堂订购今天午餐的脚本：

URL="https://lunch.com/lunch/cgi-bin/order.cgi"

O="order=Order"
A="amount_%d=%%d&amount_foil_container_%d=%%d"

function order_lunch() {
  if [[ -n "$@" ]]; then
    curl -u "$USER":"$PASSWORD" \
         -d $(printf $(printf "$O&$A&$A&$A&$A" 0 0 1 1 2 2 3 3) \
                     "${@:2:8}") \
         "$URL";
  else
    echo "Nothing to order.";
  fi;
}

Where input is a string in the following format

其中输入是以下格式的字符串

2012-08-23 1 0 0 0 0 0 0 0

where each field denotes a different dish, i.e. a 1 in the first position after the date is "1 pasta"

其中每个字段表示不同的菜，即日期后第一个位置的 1 是“1 意大利面”

Good luck.

祝你好运。

回答by mzet

... "opens a connection to a specific website and input the arg into text field using the field's ID" ...

...“打开与特定网站的连接，并使用字段的 ID 将 arg 输入到文本字段中”...

You mean you want to fill & send a HTML <form> ... </form>, right?

你的意思是你想填充并发送一个 HTML <form> ... </form>，对吗？

I would use curl (http://curl.haxx.se/). With curl you can automate HTTP POST requests very easily, suppose you have website with following form (excerpt from: http://curl.haxx.se/docs/httpscripting.html):

我会使用 curl (http://curl.haxx.se/)。使用 curl，您可以非常轻松地自动化 HTTP POST 请求，假设您有以下形式的网站（摘自：http: //curl.haxx.se/docs/httpscripting.html）：

<form method="POST" action="junk.cgi">
      <input type=text name="birthyear">
      <input type=submit name=press value=" OK ">
</form>

this command would fill & send the form (let's pretend that form is available on http://www.example.com/when.cgi):

此命令将填写并发送表单（让我们假设该表单可在http://www.example.com/when.cgi 上获得）：

curl --data "birthyear=1905&press=%20OK%20" http://www.example.com/when.cgi

回答by V H

think you need to get to grips with forms first before attempting this.

认为您需要先掌握表格，然后再尝试此操作。

Meaning you could reproduce thml file locally which includes all the values of the form, the forms action could be the end url steps 3 and 4, also look into auto submit java scripts

这意味着您可以在本地重现 thml 文件，其中包括表单的所有值，表单操作可能是第 3 步和第 4 步的结束 url，还可以查看自动提交 java 脚本

The final hyperlink well once the form is submitted - if the last step by parsing outcome of the post and then using curl or wget or something that would act as the click

提交表单后的最终超链接 - 如果最后一步是通过解析帖子的结果然后使用 curl 或 wget 或可以充当点击的东西

E2A the problem with a bash script is my concept above of creating the form is bull crap since to then execute a java script command line browser or links/lynx/wget/curl etc will be a challenge.

E2A bash脚本的问题是我上面创建表单的概念是废话，因为然后执行java脚本命令行浏览器或links/lynx/wget/curl等将是一个挑战。

tne first question is does the form support both get and post - if form action can only be post then you will not be able to send form fields as variables i.e.

第一个问题是表单是否同时支持获取和发布 - 如果表单操作只能发布，那么您将无法将表单字段作为变量发送，即

http://destinatio-form-url.com/acceptform.cgi?user=something&address=something_else

http://destinatio-form-url.com/acceptform.cgi?user=something&address=something_else

This above exampe is how you coud generate your form values if get is supported if however you need to post then a form needs to be gnerated with form action set to post to get to that url and it goes like I said you need to create the form.

上面的例子是你如何生成你的表单值，如果 get 被支持，如果你需要发布然后一个表单需要生成一个表单操作设置为 post 以获取该 url，就像我说的那样你需要创建形式。

If assuming you can send it via above format then the thing to watch for is where the response in the clickable link is if its another click away - you can see the problem if however it is returned on the same page submitted to then - it be pretty easy to parse the html by grepping for something specific and grepping/awking until you get exact url which you fire off,

如果假设您可以通过上述格式发送它，那么要注意的是可点击链接中的响应是否是另一次点击 - 您可以看到问题，如果它在提交给的同一页面上返回 - 它是通过搜索特定内容和 grepping/awking 来解析 html 非常容易，直到你得到你触发的确切 url，

take a look at my answer here

在这里看看我的回答

bash script to login to webpage

用于登录网页的 bash 脚本

This is how you would go about in java authenticating grabbing a cookie then progressing as a logged in user, the thing there is all you need to post your form

这就是您在 Java 身份验证中获取 cookie 然后以登录用户身份进行身份验证的方式，这就是您发布表单所需的全部内容

All I am saying is it is possible in bash but for url processing maybe done in a better language which gives you all the libraries to do this and makes it elegant rather than calling all sorts of system commands

我要说的是它在 bash 中是可能的，但是对于 url 处理可能用更好的语言完成，它为您提供所有库来执行此操作并使其优雅而不是调用各种系统命令

The example given is in Java but could be in any language, perl,php,python and so forth and they should all have libraries for this task, for Perl look up LWP html in google and lots of specific libraries like LWP HTML Parser and so forth you could use

给出的例子是在 Java 中，但可以是任何语言，perl、php、python 等等，他们都应该有这个任务的库，因为 Perl 在谷歌中查找 LWP html 和许多特定的库，如 LWP HTML Parser 等等你可以使用

Anyhow all the best

总之一切顺利

I think links supports java scripts if it helps..

如果有帮助，我认为链接支持 java 脚本。

it is like lynx but has a lot more addons

它就像lynx，但有更多的插件

    apt-cache search links|grep browser
amule-gnome-support - ed2k links handling support for GNOME web browsers
elinks - advanced text-mode WWW browser
elinks-data - advanced text-mode WWW browser - data files
elinks-doc - advanced text-mode WWW browser - documentation
elinks-lite - advanced text-mode WWW browser - lightweight version
libhtmlunit-core-js-java - GUI-Less browser for Java programs - JavaScript engine
libhtmlunit-java - GUI-Less browser for Java programs
libjenkins-htmlunit-java - Jenkins branch of HtmlUnit browser testing for web apps
libphp-snoopy - Snoopy is a PHP class that simulates a web browser
links - Web browser running in text mode
links2 - Web browser running in both graphics and text mode
man2html - browse man pages in your web browser
surf - simple web browser