scrapy框架爬取，为什么没有数据？

孤卷残梦饮一池恨 2020-06-03 02:22:34

爬取的url地址是https://careers.tencent.com/m/search.html

# -*- coding: utf-8 -*-

import scrapy





class HrSpider(scrapy.Spider):

    name = 'hr'

    allowed_domains = ['tencent.com']

    start_urls = ['https://careers.tencent.com/m/search.html']



    def parse(self, response):

        li_list = response.xpath('//li[@class="card-item"]')

        print(li_list)

...全文

623 4 打赏收藏转发到动态举报

写回复

用AI写文章

4 条回复

切换为时间正序

请发表友善的回复…

发表回复

小羽Jary 2020-06-04

打赏
举报

首先直接把response获取到的东西打印看一下，如果有东西，再检查你的xpath语句

孤卷残梦饮一池恨 2020-06-04

打赏
举报

引用 3 楼小羽Jary 的回复:

response.text的内容如下：需要的数据都在js里面，请问使用scrapy用什么解析数据。(不用这个框架可以有其他的方法爬取json)

<!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><meta name=keywords content=""><meta name=description content=""><meta name=apple-mobile-web-app-capable content=no><meta name=format-detection content="telephone=no"><title>搜索 | 腾讯招聘</title><link rel=stylesheet href=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/css/main.css></head><body><div id=app></div><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/careersmobilemlr/HeadFoot_zh-cn.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/careersmobilemlr/HostMsg_zh-cn.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/careersmobilemlr/Search_zh-cn.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/vendor/config.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/vendor/jquery.min.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/vendor/sona.min.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/vendor/report.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/manifest.build.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/vendor.build.js></script><script type=text/javascript src=https://cdn.multilingualres.hr.tencent.com/tencentcareermobile/static/js/m_zh-cn_search.build.js></script></body></html>

小羽Jary 2020-06-04