Python urllib.request.urlopen(url) 带身份验证

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/44239822/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-19 23:51:59  来源:igfitidea点击:

urllib.request.urlopen(url) with Authentication

pythonpython-3.xurlbeautifulsouprequest

提问by moritzg

I've been playing with beautiful soup and parsing web pages for a few days. I have been using a line of code which has been my saviour in all the scripts that I write. The line of code is :

几天来我一直在玩漂亮的汤和解析网页。我一直在使用一行代码,它在我编写的所有脚本中都是我的救星。代码行是:

r = requests.get('some_url', auth=('my_username', 'my_password')).

BUT ...

但 ...

I want to do the same thing with (OPEN A URL WITH AUTHENTICATION):

我想用(OPEN A URL WITH AUTHENTICATION)做同样的事情:

(1) sauce = urllib.request.urlopen(url).read() (1)
(2) soup = bs.BeautifulSoup(sauce,"html.parser") (2)

I'm not able to open a url and read, the webpage which needs authentication. How do I achieve something like this :

我无法打开 url 并阅读需要身份验证的网页。我如何实现这样的目标:

  (3) sauce = urllib.request.urlopen(url, auth=(username, password)).read() (3) 
instead of (1)

采纳答案by Christian K?nig

Have a look at the HOWTO Fetch Internet Resources Using The urllib Packagefrom the official docs:

查看官方文档中的HOWTO Fetch Internet Resources Using The urllib Package

# create a password manager
password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()

# Add the username and password.
# If we knew the realm, we could use it instead of None.
top_level_url = "http://example.com/foo/"
password_mgr.add_password(None, top_level_url, username, password)

handler = urllib.request.HTTPBasicAuthHandler(password_mgr)

# create "opener" (OpenerDirector instance)
opener = urllib.request.build_opener(handler)

# use the opener to fetch a URL
opener.open(a_url)

# Install the opener.
# Now all calls to urllib.request.urlopen use our opener.
urllib.request.install_opener(opener)

回答by moritzg

You're using HTTP Basic Authentication:

您正在使用HTTP Basic Authentication

import urllib2, base64

request = urllib2.Request(url)
base64string = base64.b64encode('%s:%s' % (username, password))
request.add_header("Authorization", "Basic %s" % base64string)   
result = urllib2.urlopen(request)

So you should base64encode the username and password and send it as an Authorizationheader.

因此,您应该base64对用户名和密码进行编码并将其作为Authorization标头发送。