Python 爬取 .shtml 网址内容的时候会乱码

[xiaoben9]

url = 'http://www.pss-system.gov.cn/sipopublicsearch/patentsearch/showStatistics-showStatistics.shtml'
    values = {'searchCondition.executableSearchExp' : '((KW_CPP="大数据" AND AA="北京"))',
              'searchCondition.sortIndexCode':'PASTAT'}  
    headers = { 'Accept':'application/json, text/javascript, */*; q=0.01',
               'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36',
               'Referer':'http://www.pss-system.gov.cn/sipopublicsearch/patentsearch/tableSearch-showTableSearchIndex.shtml',
               'Connection':'keep-alive',
               'Content-Type':'application/x-www-form-urlencoded; charset=UTF-8' ,
               'Origin':'http://www.cnipsun.com',
               'X-Requested-With':'XMLHttpRequest',
               'faces-request':'partial/ajax',
               'Accept-Language':'zh-CN,zh;q=0.8',
               'Cookie':'WEE_SID=L9H7YGVbtR6S1b028xJmTdpG2GrvdKJpvQVRpgQzL8nvvXnNvW1w!1865118657!332130211!1476793851102; IS_LOGIN=true; wee_username=eXpiXzAwMQ%3D%3D; wee_password=eXpibWd5eWVhaA%3D%3D; JSESSIONID=L9H7YGVbtR6S1b028xJmTdpG2GrvdKJpvQVRpgQzL8nvvXnNvW1w!1865118657!332130211; Anonymity_SearchHistory_SessionId=L9H7YGVbtR6S1b028xJmTdpG2GrvdKJpvQVRpgQzL8nvvXnNvW1w!1865118657!332130211!1476793851102; Anonymity_SearchHistory=; _gscu_761734625=76528097g5xful93; _gscs_761734625=76801744whw8x493|pv:7; _gscbrs_761734625=1',
               'Host':'www.pss-system.gov.cn'
            }
    data = urllib.urlencode(values)  
    request = urllib2.Request(url, headers)  
    response = urllib2.urlopen(request,data)  
    print response.read()

这段代码运行的时候爬出来的内容是乱码。不知道为什么。难道是因为他是.shtml网址吗？因为我之前测试过使用同样的方式爬取其他网页的的时候是可以正确爬取网页内容的。初学者，希望大家指教！谢谢

[水哥阿乐]

代码不全,编码方式贴一下,encoding=?

[xiaoben9]

引用 2 楼 uiuiy1 的回复:

有压缩

a = response.read()

decp_string = zlib.decompress(a, 47)
unqu_string = urllib.unquote(decp_string)
print unqu_string.decode('utf-8')

谢谢你，是因为加密造成的。现在问题解决了

[xiaoben9]

引用 1 楼 wtuqi 的回复:

代码不全,编码方式贴一下,encoding=?

谢谢你，问题已经解决了。不好意思没看清点了个板砖

[xiaoben9]

现在问题解决了，是因为原网站的数据是压缩了的，可以参考2楼的方法来做，

[屎克螂]

有压缩

a = response.read()

decp_string = zlib.decompress(a, 47)
unqu_string = urllib.unquote(decp_string)
print unqu_string.decode('utf-8')

[leo_cuiyiming]

貌似还是看的不太懂usys