pycharm配置scrapy在CMD创建报错

darkbjgiven 2020-01-28 06:28:57

使用pycharm做第一个scrapy项目：
-----------------------------------------------------------------------------------------------------------------------------------
报错：
D:\Program Files\JetBrains\PycharmProjects\testproject>scrapy startproject testspider
Traceback (most recent call last):
File "d:\program files\python\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "d:\program files\python\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Program Files\python\Scripts\scrapy.exe\__main__.py", line 5, in <module>
File "d:\program files\python\lib\site-packages\scrapy\cmdline.py", line 10, in <module>
from scrapy.crawler import CrawlerProcess
File "d:\program files\python\lib\site-packages\scrapy\crawler.py", line 16, in <module>
from scrapy.signalmanager import SignalManager
File "d:\program files\python\lib\site-packages\scrapy\signalmanager.py", line 2, in <module>
from pydispatch import dispatcher
ImportError: cannot import name 'dispatcher' from 'pydispatch' (d:\program files\python\lib\site-packages\pydispatch\__init__.py)

ImportError: cannot import name 'dispatcher' from 'pydispatch' (d:\program files\python\lib\site-packages\pydispatch\__init__.py)
在pydispatch里不能使用dispatcher（pydispatch这个安装成功了仍然报错）重名的话在下面也没看到啊。。。
我在用CMD的检查命令 scrapy -h 时也给出的这个报错

-----------------------------------------------------------------------------------------------------------------------------------
配置方面：
1、python3.7、windows10、pycharm2019
2、没有安装Anaconda（各种包都是自己下的）
3、twisted、wheel、pywin32、pydispatch均在pip3上安装成功
4、scrapy在CMD上检查不通过，使用pip install scrapy时候没报错，都满足了

-----------------------------------------------------------------------------------------------------------------------------------
请问各位有没有遇到过这个问题啊，一直没办法解决，绝望........................谢谢各位！！！！！

...全文

237 回复打赏收藏转发到动态举报

写回复

回复

切换为时间正序

请发表友善的回复…

发表回复

文件： import scrapy from demo1.items import Demo1Item import urllib from scrapy import log # BOSS直聘网站爬虫职位 class DemoSpider(scrapy.Spider): # 爬虫名，启动爬虫时需要的参数*必填 name = 'demo' # 爬取域范围，允许爬虫在这个域名下进行爬取（可选） allowed_domains = ['zhipin.com'] # 爬虫需要的url start_urls = ['https://www.zhipin.com/c101280600/h_101280600/?query=测试'] def parse(self, response): node_list = response.xpath("//div[@class='job-primary']") # 用来存储所有的item字段 # items = [] for node in node_list: item = Demo1Item() # extract() 将xpath对象转换为Unicode字符串 href = node.xpath("./div[@class='info-primary']//a/@href").extract() job_title = node.xpath("./div[@class='info-primary']//a/div[@class='job-title']/text()").extract() salary = node.xpath("./div[@class='info-primary']//a/span/text()").extract() working_place = node.xpath("./div[@class='info-primary']/p/text()").extract() company_name = node.xpath("./div[@class='info-company']//a/text()").extract() item['href'] = href[0] item['job_title'] = job_title[0] item['sa 报错： C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\python.exe "C:\Users\xieqianyun\PyCharm Community Edition 2019.2.5\helpers\pydev\pydevconsole.py" --mode=client --port=55825 import sys; print('Python %s on %s' % (sys.version, sys.platform)) sys.path.extend(['C:\\Users\\xieqianyun\\demo1', 'C:/Users/xieqianyun/demo1']) Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] Type 'copyright', 'credits' or 'license' for more information IPython 7.10.0 -- An enhanced Interactive Python. Type '?' for help. PyDev console: using IPython 7.10.0 Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32 runfile('C:/Users/xieqianyun/demo1/demo1/begin.py', wdir='C:/Users/xieqianyun/demo1/demo1') Traceback (most recent call last): File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\IPython\core\interactiveshell.py", line 3319, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('C:/Users/xieqianyun/demo1/demo1/begin.py', wdir='C:/Users/xieqianyun/demo1/demo1') File "C:\Users\xieqianyun\PyCharm Community Edition 2019.2.5\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "C:\Users\xieqianyun\PyCharm Community Edition 2019.2.5\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/Users/xieqianyun/demo1/demo1/begin.py", line 3, in cmdline.execute('scrapy crawl demo'.split()) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\cmdline.py", line 145, in execute cmd.crawler_process = CrawlerProcess(settings) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\crawler.py", line 267, in __init__ super(CrawlerProcess, self).__init__(settings) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\crawler.py", line 145, in __init__ self.spider_loader = _get_spider_loader(settings) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\crawler.py", line 347, in _get_spider_loader return loader_cls.from_settings(settings.frozencopy()) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\spiderloader.py", line 61, in from_settings return cls(settings) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\spiderloader.py", line 25, in __init__ self._load_all_spiders() File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\spiderloader.py", line 47, in _load_all_spiders for module in walk_modules(name): File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\site-packages\scrapy\utils\misc.py", line 73, in walk_modules submod = import_module(fullpath) File "C:\Users\xieqianyun\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 955, in _find_and_load_unlocked File "", line 665, in _load_unlocked File "", line 678, in exec_module File "", line 219, in _call_with_frames_removed File "C:\Users\xieqianyun\demo1\demo1\spiders\demo.py", line 4, in from scrapy import log ImportError: cannot import name 'log'

pycharm创建scrapy项目

Pycharm安装与使用课程目录01pycharm安装剪辑02pychram文件和工程的创建剪辑03pycharm文件的移动和重命名剪辑04文件运行和更改解释器剪辑05pycharme字体修改和主题剪辑06pycharme给文件添加默认信息剪辑07pychram快捷键设置剪辑08pychram中安装VIM插件剪辑09Pycharm中如何打断点剪辑10pycharm代码折叠和快速浏览剪辑

首先确保pycharm能正常使用(安装环境为win10 pycharm 2019.3.3 、python 3.6.8) 用 pycharm 安装scrapy框架很便捷只要你的pycharm配置好就能直接一键安装 scrapy需要安装的配套模块 -wheel -twisted -pywin32 -scrapy 在pycharm中Terminal输入对应的模块安装 -twisted对应的安装代码 -pip install Twisted -pywin32 对应安装代码 -pip install p

很多人在安装Scrapy库的时候会出现 error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/down loads/这样的保存信息，网上有很多解决方案，我找到一种很快速的解决方案分享给大家 1. 首先打开https://www.lfd.uci.edu/~gohlke/pythonlibs/网址ctrl+f搜索T

37,721

社区成员

34,239

社区内容

发帖

与我相关

我的任务

社区管理员

加入社区

近7日
近30日
至今

加载中

查看更多榜单

试试用AI创作助手写篇文章吧

+ 用AI写文章