scrapy运行过程中停住且无任何报错,请问该如何定位

汇讯猎头 2017-01-23 02:12:34
操作系统环境:mac
python版本:2.7

现象:
运行scrapy的过程中,突然之间停止,且没有任何报错,日志显示如下,请问哪位高人帮忙看看为何停止了,遇到这类问题该如何定位:谢谢!


……前面略

2017-01-23 13:42:39 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4720106
2017-01-23 13:42:39 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4720094
2017-01-23 13:42:39 [scrapy] DEBUG: Crawled (200) <GET http://nocutnews.co.kr/news/list?c1=254&c2=259> (referer: None)
2017-01-23 13:42:40 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60033/wd/hub/session {"requiredCapabilities": {}, "desiredCapabilities": {"platform": "ANY", "browserName": "phantomjs", "version": "", "javascriptEnabled": true}}
2017-01-23 13:42:40 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:42:40 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60033/wd/hub/session/c13fd4f0-e12e-11e6-a80e-87e393dbd863/url {"url": "http://nocutnews.co.kr/news/list?c1=254&c2=259", "sessionId": "c13fd4f0-e12e-11e6-a80e-87e393dbd863"}
2017-01-23 13:42:46 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:42:46 [selenium.webdriver.remote.remote_connection] DEBUG: GET http://127.0.0.1:60033/wd/hub/session/c13fd4f0-e12e-11e6-a80e-87e393dbd863/source {"sessionId": "c13fd4f0-e12e-11e6-a80e-87e393dbd863"}
2017-01-23 13:42:46 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:42:46 [selenium.webdriver.remote.remote_connection] DEBUG: DELETE http://127.0.0.1:60033/wd/hub/session/c13fd4f0-e12e-11e6-a80e-87e393dbd863 {"sessionId": "c13fd4f0-e12e-11e6-a80e-87e393dbd863"}
2017-01-23 13:42:46 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:42:46 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4708177
2017-01-23 13:42:47 [scrapy] DEBUG: Crawled (200) <GET http://nocutnews.co.kr/news/list?c1=254&c2=260> (referer: None)
2017-01-23 13:42:48 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60119/wd/hub/session {"requiredCapabilities": {}, "desiredCapabilities": {"platform": "ANY", "browserName": "phantomjs", "version": "", "javascriptEnabled": true}}
2017-01-23 13:42:48 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:42:48 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60119/wd/hub/session/c5b49430-e12e-11e6-a4d5-e100c4c26ce4/url {"url": "http://nocutnews.co.kr/news/list?c1=254&c2=260", "sessionId": "c5b49430-e12e-11e6-a4d5-e100c4c26ce4"}
2017-01-23 13:43:51 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:43:51 [selenium.webdriver.remote.remote_connection] DEBUG: GET http://127.0.0.1:60119/wd/hub/session/c5b49430-e12e-11e6-a4d5-e100c4c26ce4/source {"sessionId": "c5b49430-e12e-11e6-a4d5-e100c4c26ce4"}
2017-01-23 13:43:51 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:43:51 [selenium.webdriver.remote.remote_connection] DEBUG: DELETE http://127.0.0.1:60119/wd/hub/session/c5b49430-e12e-11e6-a4d5-e100c4c26ce4 {"sessionId": "c5b49430-e12e-11e6-a4d5-e100c4c26ce4"}
2017-01-23 13:43:51 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4722146
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4721697
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4721446
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4721364
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4718899
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4716210
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4715884
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4715666
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4715796
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4715192
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4712395
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4711118
2017-01-23 13:43:51 [root] DEBUG: bloomfilter follow link = http://nocutnews.co.kr/news/4710328
2017-01-23 13:43:51 [scrapy] INFO: Crawled 434 pages (at 3 pages/min), scraped 390 items (at 0 items/min)
2017-01-23 13:43:52 [scrapy] DEBUG: Crawled (200) <GET http://nocutnews.co.kr/news/list?c1=282> (referer: None)
2017-01-23 13:43:54 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60210/wd/hub/session {"requiredCapabilities": {}, "desiredCapabilities": {"platform": "ANY", "browserName": "phantomjs", "version": "", "javascriptEnabled": true}}
2017-01-23 13:43:54 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-23 13:43:54 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:60210/wd/hub/session/ed09b380-e12e-11e6-ba50-b128e0b318bc/url {"url": "http://nocutnews.co.kr/news/list?c1=282", "sessionId": "ed09b380-e12e-11e6-ba50-b128e0b318bc"}

...全文
838 4 打赏 收藏 转发到动态 举报
AI 作业
写回复
用AI写文章
4 条回复
切换为时间正序
请发表友善的回复…
发表回复
Adyan? 2021-01-19
  • 打赏
  • 举报
回复
我也是这个情况,什么鬼
浮沉独步 2017-02-24
  • 打赏
  • 举报
回复
我也遇到这样的情况,不知道因为什么
屎克螂 2017-02-10
  • 打赏
  • 举报
回复
请求要写超时
shiter 2017-02-04
  • 打赏
  • 举报
回复
是不是网页完了?还是网络断了

37,743

社区成员

发帖
与我相关
我的任务
社区描述
JavaScript,VBScript,AngleScript,ActionScript,Shell,Perl,Ruby,Lua,Tcl,Scala,MaxScript 等脚本语言交流。
社区管理员
  • 脚本语言(Perl/Python)社区
  • WuKongSecurity@BOB
加入社区
  • 近7日
  • 近30日
  • 至今

试试用AI创作助手写篇文章吧