37,740
社区成员
发帖
与我相关
我的任务
分享[爬虫资源]各大爬虫资源大汇总,做我们自己的awesome系列
Awesome-crawlerawesome
A collection of awesome web crawler,spider and resources in different language
Python
Scrapy - A fast high-level screen scraping and web crawling framework.
pyspider - A powerful spider system.
cola - A distributed crawling framework.
Demiurge - PyQuery-based scraping micro-framework.
feedparser - Universal feed parser.
Grab - Site scraping framework.
MechanicalSoup - A Python library for automating interaction with websites.
portia - Visual scraping for Scrapy.
crawley - Pythonic Crawling / Scraping Framework based on Non Blocking I/O operations.
RoboBrowser - A simple, Pythonic library for browsing the web without a standalone web browser.
MSpider - A simple ,easy spider using gevent and js render.
这是其中的一部分，还有其它相应语言的优秀爬虫框架在github里面，更多的请移步到github中
==>https://github.com/BruceDone/awesome-crawler<==
A collection of awesome web crawler,spider and resources in different language
Python
Scrapy - A fast high-level screen scraping and web crawling framework.
pyspider - A powerful spider system.
cola - A distributed crawling framework.
Demiurge - PyQuery-based scraping micro-framework.
feedparser - Universal feed parser.
Grab - Site scraping framework.
MechanicalSoup - A Python library for automating interaction with websites.
portia - Visual scraping for Scrapy.
crawley - Pythonic Crawling / Scraping Framework based on Non Blocking I/O operations.
RoboBrowser - A simple, Pythonic library for browsing the web without a standalone web browser.
MSpider - A simple ,easy spider using gevent and js render.
这是其中的一部分，还有其它相应语言的优秀爬虫框架在github里面，更多的请移步到github中
==>https://github.com/BruceDone/awesome-crawler<==
...全文
请发表友善的回复…
发表回复
人傻且呆但不萌 2016-10-11
- 打赏
- 举报
csdn的帖子竟然还不支持MD 我。。。
