小白求解，编码搞的有点头大

[RainMan_123]

UnicodeEncodeError: 'gbk' codec can't encode character '\uf618' in position 0: illegal multibyte sequence

照着极客学院 Xpath与多线程爬虫的视频敲的代码，保存文件的时候会报错

自己也尝试了将requests.get的html编码设置为utf -8 或者 gb18030 失败告终

将文件打开的编码设置为utf -8 或者 gb18030 甚至都没看到目录下的文件

百度之后，还有：
import sys
import io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='UTF-8')

也失败了


求大神指导

[RainMan_123]

自己用的Py3

[sanGuo_uu]

是获取html时候报错、 还是把html保存成文档报错？ 对着页面，右键查看源码，看页面的charset，按这个值来decode。 -------------------------------------------------------------------------------------- 如果还不行 那你最好把页面url，代码贴下

[RainMan_123]

url : http://tieba.baidu.com/p/3522395718?pn=1 爬取代码如下： from multiprocessing.dummy import Pool as ThreadPool from lxml import etree import requests import json # import sys # import io # # sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='UTF-8') def writeTofile(contentdict): f.write('回帖时间：' + str(contentdict['topic_reply_time']) + '\n') f.writelines('回帖内容：' + contentdict['topic_reply_content'] + '\n') f.writelines('回帖人：' + contentdict['user_name'] + '\n\n\n') def spider(url): html = requests.get(url) selector = etree.HTML(html.text) content_field = selector.xpath('//div[@class="l_post j_l_post l_post_bright "]') item = {} for each in content_field: reply_info = json.loads(each.xpath('@data-field')[0].replace('"', '')) author = reply_info['author']['user_name'] content = each.xpath('div[@class="d_post_content_main"]/div/cc/div[@class="d_post_content j_d_post_content clearfix"]/text()')[0] reply_time = reply_info['content']['date'] # print(content) # print(reply_time) # print(author) item['user_name'] = author item['topic_reply_content'] = content item['topic_reply_time'] = reply_time writeTofile(item) if __name__ == '__main__': pool = ThreadPool(4) f = open('./content.txt', 'a+') page = ["http://tieba.baidu.com/p/3522395718?pn=1"] # page = [] # for i in range(1, 2): # newpage = 'http://tieba.baidu.com/p/3522395718?pn=' + str(i) # page.append(newpage) results = pool.map(spider, page) pool.close() pool.join() f.close() 报错信息： Traceback (most recent call last): File "D:/Learning/PycharmProjects/Python_Projects/jikexueyuan/XPath与多线程爬虫/Xpath_4.py", line 61, in <module> results = pool.map(spider, page) File "C:\Python36\Lib\multiprocessing\pool.py", line 266, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "C:\Python36\Lib\multiprocessing\pool.py", line 644, in get raise self._value File "C:\Python36\Lib\multiprocessing\pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "C:\Python36\Lib\multiprocessing\pool.py", line 44, in mapstar return list(map(*args)) File "D:/Learning/PycharmProjects/Python_Projects/jikexueyuan/XPath与多线程爬虫/Xpath_4.py", line 49, in spider writeTofile(item) File "D:/Learning/PycharmProjects/Python_Projects/jikexueyuan/XPath与多线程爬虫/Xpath_4.py", line 29, in writeTofile f.writelines('回帖内容：' + contentdict['topic_reply_content'] + '\n') UnicodeEncodeError: 'gbk' codec can't encode character '\uf618' in position 0: illegal multibyte sequence 通过Xpath获取到的数据，存到字典中，通过函数来将字典中的数据保存到文件中

[RainMan_123]

找到出错的位置了

[RainMan_123]

解决了，open文件的时候，将encoding设置为‘utf-8’ 然而，在pycharm 中居然要设置绝对路径才能写文件