python3 编码问题

[掌上鹅]

在用python3爬一个网页的源代码的时候 ，当一个链接中有中文时报错了
代码如下
# encoding:UTF-8
import urllib.request

url = "https://www.baidu.com/baidu?wd=马云&tn=monline_dg&ie=utf-8"
data = urllib.request.urlopen(url).read()
data = data.decode('UTF-8')
print(data)
报错代码：
UnicodeEncodeError: 'ascii' codec can't encode characters in position 14-15: ordinal not in range(128)

[sanGuo_uu]

上面那个有毛病，这个是新改的 要加headers，不然总有点问题

from urllib import request

url = "https://www.baidu.com"
url2='https://www.baidu.com/baidu?wd='+request.quote("马云")+'&tn=monline_dg&ie=utf-8'

headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36',  
'Referer': 'xxx',  
'Connection': 'keep-alive' 
} 
req=request.Request(url2,headers=headers)
data = request.urlopen(req).read()
data = data.decode('UTF-8')

print(data)

[掌上鹅]

引用 2 楼 u012536120 的回复:

上面那个有毛病，这个是新改的 要加headers，不然总有点问题

from urllib import request

url = "https://www.baidu.com"
url2='https://www.baidu.com/baidu?wd='+request.quote("马云")+'&tn=monline_dg&ie=utf-8'

headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36',  
'Referer': 'xxx',  
'Connection': 'keep-alive' 
} 
req=request.Request(url2,headers=headers)
data = request.urlopen(req).read()
data = data.decode('UTF-8')

print(data)

你这样写还是会报错，应该在第一行写加 #encoding：gbk 这样就可以运行了

[sanGuo_uu]

from urllib import request,parse

url = "https://www.baidu.com/baidu"
data = {  
	'wd': '马云',  
	'tn': 'monline_dg',  
	'ie': 'utf-8'
}  
data = parse.urlencode(data).encode('utf-8')  

req=request.Request(url,data=data)
data = request.urlopen(req).read()
data = data.decode('UTF-8')
print(data)