第四次试验-20222314陈振烨基于南理工学姐的百度动态图片爬虫系统以及对于自己交晚作业的深刻忏悔和表水

落影_ 2023-06-07 18:48:13

20221323陈振烨《Python程序设计》实验四报告

课程：《Python程序设计》
班级： 2223
姓名：陈振烨
学号：20222314
实验教师：王志强
实验日期：2023年5月10日
必修/选修：公选课

1.实验内容

根据南理工学姐的需求，编写网络爬虫，为了让南理工学姐能尽快看到自己喜欢看的最新图片，我们选择编写一个百度动态爬虫，为了减少南理工学姐端的操作难度，这里没有使用驱动，而是使用了构造的头来伪装浏览器，从而达到反爬，访问，以及发送下载请求的效果。

2.实验过程及结果

1.设计思路：

先通过微信询问南理工学姐喜欢看什么

然后运行自己写好的程序

2.部分核心代码思路：

# -*- coding: UTF-8 -*-"""
import requests
import tqdm


def configs(search, page, number):
    """

    :param search:
    :param page:
    :param number:
    :return:
    """
    url = 'https://image.baidu.com/search/acjson'
    params = {
        "tn": "resultjson_com",
        "logid": "11555092689241190059",
        "ipn": "rj",
        "ct": "201326592",
        "is": "",
        "fp": "result",
        "queryWord": search,
        "cl": "2",
        "lm": "-1",
        "ie": "utf-8",
        "oe": "utf-8",
        "adpicid": "",
        "st": "-1",
        "z": "",
        "ic": "0",
        "hd": "",
        "latest": "",
        "copyright": "",
        "word": search,
        "s": "",
        "se": "",
        "tab": "",
        "width": "",
        "height": "",
        "face": "0",
        "istype": "2",
        "qc": "",
        "nc": "1",
        "fr": "",
        "expermode": "",
        "force": "",
        "pn": str(60 * page),
        "rn": number,
        "gsm": "1e",
        "1617626956685": ""
    }
    return url, params


def loadpic(number, page):
    """

    :param number:
    :param page:
    :return:
    """
    while (True):
        if number == 0:
            break
        url, params = configs(search, page, number)
        result = requests.get(url, headers=header, params=params).json()
        url_list = []
        for data in result['data'][:-1]:
            url_list.append(data['thumbURL'])
        for i in range(len(url_list)):
            getImg(url_list[i], 60 * page + i, path)
            bar.update(1)
            number -= 1
            if number == 0:
                break
        page += 1
    print("\nfinish!")


def getImg(url, idx, path):
    """

    :param url:
    :param idx:
    :param path:
    :return:
    """
    img = requests.get(url, headers=header)
    file = open(path + 'maintenanceWorker_' + str(idx + 1) + '.jpg', 'wb')
    file.write(img.content)
    file.close()


if __name__ == '__main__':
    search = input("请输入搜索内容：")
    number = int(input("请输入需求数量："))
    path = 'E:\桌面\czynb\czy\南理工学姐喜欢的东西'
    header = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36'}

    bar = tqdm.tqdm(total=number)
    page = 0
    loadpic(number, page)

运行截图如下：