如何从网页获取 JSON 到 Python 脚本

How to get JSON from webpage into Python script

提问by Chris B

Got the following code in one of my scripts:

在我的一个脚本中得到以下代码：

#
# url is defined above.
#
jsonurl = urlopen(url)

#
# While trying to debug, I put this in:
#
print jsonurl

#
# Was hoping text would contain the actual json crap from the URL, but seems not...
#
text = json.loads(jsonurl)
print text

What I want to do is get the {{.....etc.....}}stuff that I see on the URL when I load it in Firefox into my script so I can parse a value out of it. I've Googled a ton but I haven't found a good answer as to how to actually get the {{...}}stuff from a URL ending in .jsoninto an object in a Python script.

我想要做的是获取{{.....etc.....}}我在 Firefox 中将 URL 加载到我的脚本中时在 URL 上看到的内容，以便我可以从中解析出一个值。我在谷歌上搜索了很多，但我还没有找到关于如何{{...}}从.json以 Python 脚本结尾的 URL 中实际获取内容的好答案。

回答by bgporter

All that the call to urlopen()does (according to the docs) is return a file-like object. Once you have that, you need to call its read()method to actually pull the JSON data across the network.

调用urlopen()所做的所有事情（根据docs）都是返回一个类似文件的对象。一旦你有了它，你需要调用它的read()方法来实际通过网络拉取 JSON 数据。

Something like:

就像是：

jsonurl = urlopen(url)

text = json.loads(jsonurl.read())
print text

回答by Jon Clements

I'll take a guess that you actually want to get data from the URL:

我猜测您实际上想从 URL 获取数据：

jsonurl = urlopen(url)
text = json.loads(jsonurl.read()) # <-- read from it

Or, check out JSON decoderin the requestslibrary.

或者，查看请求库中的JSON 解码器。

import requests
r = requests.get('someurl')
print r.json() # if response type was set to JSON, then you'll automatically have a JSON response here...

回答by Anurag Uniyal

Get data from the URL and then call json.loadse.g.

从 URL 获取数据，然后调用json.loadseg

Python3 example:

Python3 示例：

import urllib.request, json 
with urllib.request.urlopen("http://maps.googleapis.com/maps/api/geocode/json?address=google") as url:
    data = json.loads(url.read().decode())
    print(data)

Python2 example:

Python2 示例：

import urllib, json
url = "http://maps.googleapis.com/maps/api/geocode/json?address=google"
response = urllib.urlopen(url)
data = json.loads(response.read())
print data

The output would result in something like this:

输出结果如下：

{
"results" : [
    {
    "address_components" : [
        {
            "long_name" : "Charleston and Huff",
            "short_name" : "Charleston and Huff",
            "types" : [ "establishment", "point_of_interest" ]
        },
        {
            "long_name" : "Mountain View",
            "short_name" : "Mountain View",
            "types" : [ "locality", "political" ]
        },
        {
...

回答by posit labs

There's no need to use an extra library to parse the json...

无需使用额外的库来解析 json ...

json.loads()returns a dictionary.

json.loads()返回一个字典。

So in your case, just do text["someValueKey"]

所以在你的情况下，就做 text["someValueKey"]

回答by Martin Thoma

This gets a dictionary in JSON format from a webpage with Python 2.X and Python 3.X:

这会从带有 Python 2.X 和 Python 3.X 的网页中获取 JSON 格式的字典：

#!/usr/bin/env python

try:
    # For Python 3.0 and later
    from urllib.request import urlopen
except ImportError:
    # Fall back to Python 2's urllib2
    from urllib2 import urlopen

import json


def get_jsonparsed_data(url):
    """
    Receive the content of ``url``, parse it as JSON and return the object.

    Parameters
    ----------
    url : str

    Returns
    -------
    dict
    """
    response = urlopen(url)
    data = response.read().decode("utf-8")
    return json.loads(data)


url = ("http://maps.googleapis.com/maps/api/geocode/json?"
       "address=googleplex&sensor=false")
print(get_jsonparsed_data(url))

See also: Read and write example for JSON

另请参阅：JSON 的读写示例

回答by Uxbridge

I have found this to be the easiest and most efficient way to get JSON from a webpage when using Python 3:

我发现这是在使用 Python 3 时从网页获取 JSON 的最简单、最有效的方法：

import json,urllib.request
data = urllib.request.urlopen("https://api.github.com/users?since=100").read()
output = json.loads(data)
print (output)

回答by aviso

In Python 2, json.load() will work instead of json.loads()

在 Python 2 中， json.load() 将代替 json.loads() 工作

import json
import urllib

url = 'https://api.github.com/users?since=100'
output = json.load(urllib.urlopen(url))
print(output)

Unfortunately, that doesn't work in Python 3. json.load is just a wrapper around json.loads that calls read() for a file-like object. json.loads requires a string object and the output of urllib.urlopen(url).read() is a bytes object. So one has to get the file encoding in order to make it work in Python 3.

不幸的是，这在 Python 3 中不起作用。 json.load 只是 json.loads 的包装器，它为类似文件的对象调用 read() 。json.loads 需要一个字符串对象，而 urllib.urlopen(url).read() 的输出是一个字节对象。因此，必须获得文件编码才能使其在 Python 3 中工作。

In this example we query the headers for the encoding and fall back to utf-8 if we don't get one. The headers object is different between Python 2 and 3 so it has to be done different ways. Using requestswould avoid all this, but sometimes you need to stick to the standard library.

在这个例子中，我们查询编码的标头，如果没有得到，则回退到 utf-8。标头对象在 Python 2 和 3 之间是不同的，因此必须以不同的方式完成。使用请求可以避免这一切，但有时您需要坚持使用标准库。

import json
from six.moves.urllib.request import urlopen

DEFAULT_ENCODING = 'utf-8'
url = 'https://api.github.com/users?since=100'
urlResponse = urlopen(url)

if hasattr(urlResponse.headers, 'get_content_charset'):
    encoding = urlResponse.headers.get_content_charset(DEFAULT_ENCODING)
else:
    encoding = urlResponse.headers.getparam('charset') or DEFAULT_ENCODING

output = json.loads(urlResponse.read().decode(encoding))
print(output)

回答by Keivan

you can use json.dumps:

你可以使用json.dumps：

import json

# Hier comes you received data

data = json.dumps(response)

print(data)

for loading json and write it on file the following code is useful:

对于加载 json 并将其写入文件，以下代码很有用：

data = json.loads(json.dumps(Response, sort_keys=False, indent=4))
with open('data.json', 'w') as outfile:
json.dump(data, outfile, sort_keys=False, indent=4)

回答by CONvid19

Late answer, but for python>=3.6you can use:

迟到的答案，但python>=3.6您可以使用：

import dload
j = dload.json(url)

---

---

Install dloadwith:

安装dload：

pip3 install dload