如何使用 Pandas read_html 和请求库来读取表？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/19983406/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to use Pandas read_html and requests library to read the table?
提问by Terence Ng
How can I scrape the prices of a fund in:
我如何在以下位置抓取基金的价格：
http://www.prudential.com.hk/PruServlet?module=fund&purpose=searchHistFund&fundCd=JAS_U
http://www.prudential.com.hk/PruServlet?module=fund&purpose=searchHistFund&fundCd=JAS_U
It is wrong but how do I modify it:
这是错误的，但我如何修改它：
import pandas as pd
import requests
import re
url = 'http://www.prudential.com.hk/PruServlet?module=fund&purpose=searchHistFund&fundCd=JAS_U'
tables = pd.read_html(requests.get(url).text, attrs={"class":re.compile("fundPriceCell\d+")})
采纳答案by brechin
I like lxml for parsing and querying HTML. Here's what I came up with:
我喜欢用 lxml 来解析和查询 HTML。这是我想出的：
import requests
from lxml import etree
url = 'http://www.prudential.com.hk/PruServlet?module=fund&purpose=searchHistFund&fundCd=JAS_U'
doc = requests.get(url)
tree = etree.HTML(doc.content)
row_xpath = '//tr[contains(td[1]/@class, "fundPriceCell")]'
rows = tree.xpath(row_xpath)
for row in rows:
(date_string, v1, v2) = (td.text for td in row.getchildren())
print "%s - %s - %s" % (date_string, v1, v2)
回答by Terence Ng
My solution is similar to yours:
我的解决方案与您的类似：
import pandas as pd
import requests
from lxml import etree
url = "http://www.prudential.com.hk/PruServlet?module=fund&purpose=searchHistFund&fundCd=JAS_U"
r = requests.get(url)
html = etree.HTML(r.content)
data = html.xpath('//table//table//table//table//td[@class="fundPriceCell1" or @class="fundPriceCell2"]//text()')
if len(data) % 3 == 0:
df = pd.DataFrame([data[i:i+3] for i in range(0, len(data), 3)], columns = ['date', 'bid', 'ask'])
df = df.set_index('date')
df.index = pd.to_datetime(df.index, format = '%d/%m/%Y')
df.sort_index(inplace = True)
