QNetworkAccessManager明显的内存泄漏

[hit1er]

while(1){
        QNetworkAccessManager *aa = new QNetworkAccessManager();
        QNetworkReply *bb;
        QNetworkRequest Request;
        Request.setUrl(QUrl("http://127.0.0.1"));

        bb= aa->get(Request);
        bb->abort();
        bb->close();
        bb->deleteLater();
        bb = NULL;

        delete aa;
        aa = NULL;
 }

我在做了个蜘蛛,用QNetworkAccessManager 取网页代码,很久才能看到问题,上面代码运行就能看到问题

一般情况下没人会发现,win7下用windows任务管理器,选项-选择列-选中句柄数,

内存确实增加不多,始终保持在30M左右

这句柄数一直上升,我服务器上大概句柄数跑到200万时候就崩溃,上面这段代码,几分钟就能跑到上百万的句柄数,

而且可以看出,new 了都有delete的


这是什么问题,QNetworkAccessManager 本身代码问题还是没有做什么设置才会这样

[hit1er]

上面代码只是演示而已,可以快速测试,实际代码肯定不是这么写的,要运行10几小时,甚至几天才会崩溃

[slientman]

嗯，有点意思啊。

[hit1er]

有意思个毛啊,这问题弄死我了

[zhu_xz]

你为啥要new一大堆QNetworkAccessManager呢？这个是可以用来发起多个请求的

[xiebin133]

你用内存池吧，你一直new delete，时间久了会产生内存碎片。这个是性能问题。

[cppbegginer]

你的代码是没有问题的，QT的库也是没有问题的，只是用法上需要完善，在while循环中的aa = NULL;后面加上一句a.processEvents()(a就是QApplication实例对象);处理所有事件即可解决泄露的问题。

有些资源是到event循环中才会被真正释放。

引用一段文档


void QObject::deleteLater () [slot]
Schedules this object for deletion.

The object will be deleted when control returns to the event loop. If the event loop is not running when this function is called (e.g. deleteLater() is called on an object before QCoreApplication::exec()), the object will be deleted once the event loop is started.

[addfourliu]

同情中。。。
我这两天仿照Demo中的例子写了个多tab的浏览器，不停的打开关闭tab几十次就会莫名其妙的崩溃，用内存泄露工具测了一下，也没有什么泄露，但是执行多了就会崩溃，我编译了一下Demo中的例子，同样操作几十次，同样的崩溃，不知道那里出了问题。。。。

[hit1er]

[Quote=引用 5 楼 的回复:]

你的代码是没有问题的，QT的库也是没有问题的，只是用法上需要完善，在while循环中的aa = NULL;后面加上一句a.processEvents()(a就是QApplication实例对象);处理所有事件即可解决泄露的问题。

有些资源是到event循环中才会被真正释放。

引用一段文档


void QObject::deleteLater () [slot]
Schedu……
[/Quote]

这样在主线程里确实好用啊,不过放线程里就不行

class abc : public QThread{
protected:
    void run(){
        QNetworkAccessManager *aa = new QNetworkAccessManager();
        QNetworkReply *bb;
        QNetworkRequest Request;
  
        Request.setUrl(QUrl("http://127.0.0.1/1.php"));

        bb= aa->get(Request);
        bb->abort();
        bb->close();
        delete aa;
        aa = NULL;

    }
};

void main(){
   QApplication a(argc, argv);

   for(int i=0;i<1000;i++){
        abc *bbb = new abc;
        bbb->start();
        bbb->wait();
        delete bbb;
        bbb = NULL;
    }
    return a.exec();

}

[cppbegginer]

如果你非要大量和反复使用，用完保存就可以了，不用删除吧。

[Quote=引用 8 楼 的回复:]

引用 5 楼 的回复:

你的代码是没有问题的，QT的库也是没有问题的，只是用法上需要完善，在while循环中的aa = NULL;后面加上一句a.processEvents()(a就是QApplication实例对象);处理所有事件即可解决泄露的问题。

有些资源是到event循环中才会被真正释放。

引用一段文档


void QObject::deleteLater (……
[/Quote]

[xiebin133]

用一个类的全局变量不就好了，一直new delete会产生大量的内存碎片，用久了肯定有问题的。

[hit1er]

downloadmanager.h

#ifndef DOWNLOADMANAGER_H
#define DOWNLOADMANAGER_H

#include <QApplication>
#include "baselib.h"
#include <QtNetwork/QHttpRequestHeader>
#include <QtNetwork/QNetworkAccessManager>
#include <QtNetwork/QNetworkRequest>
#include <QtNetwork/QNetworkReply>

class DownloadManager : public QObject
{
    Q_OBJECT
public:
    DownloadManager(QObject *parent=0);
    ~DownloadManager();
    void clear();
    void setCookie(QString);
    bool downloadFinished(bool bResult);
    void setHtmlData(QString strHtml);
    void setFaildRetryNum(int n);
    QString GetHtmlData();
    QString GetCookie();

public:
    bool HttpGetData(QString);
    bool HttpPostData(QString,QString);
    bool DownloadSrc(QString strUrl,QString strSavePath,QString strReferer = "");
    bool DownloadIsFinished();

    void HttpGetData_Asyn(QString);
    void HttpPostData_Asyn(QString,QString);
    void DownloadSrc_Asyn(QString strUrl,QString strSavePath,QString strReferer = "");
signals:
    void DownloadManagerProgress(int, int);
    void DownloadManagerFinished(bool);
    void DownloadManagerError(int);
private slots:
    void replyFinished();
    void slotError();
    void downloadProgress(qint64, qint64);
public:
    bool m_IsSuccess;
    bool m_IsCoverFile;
    bool m_IsFinished;

    int m_DownloadType;
    int m_DownloadFaildRetryNum;
    int m_DownloadFaildNum;
    int m_DownloadReceived;
    int m_DownloadTotal;
    int n_DownloadError;

    QString m_strUrl;
    QString m_strPath;
    QString m_strReferer;
    QString m_strCookie;
    QString m_strPostData;
    QString m_strHtmlData;
    QString m_strContentCodec;
private:
    QNetworkAccessManager *m_pManager;
    QEventLoop m_Event;
    QFile *m_pFile;
};


#endif // DOWNLOADMANAGER_H

[hit1er]

downloadmanager.cpp

#include "downloadmanager.h"

//==========清理
void DownloadManager::clear(){
    m_DownloadReceived = 0;
    m_DownloadTotal = 0;

    n_DownloadError = 0;
    m_DownloadType = 0;
    m_DownloadFaildRetryNum = 2;
    m_DownloadFaildNum = 0;
    m_IsCoverFile = true;
    m_IsSuccess = true;
    m_IsFinished = false;

    m_strUrl = "";
    m_strPath = "";
    m_strPostData = "";
    m_strCookie = "";
    m_strHtmlData = "";
    m_pFile = NULL;
}

DownloadManager::DownloadManager(QObject *parent)
    :QObject(parent)
{
    clear();
    m_pManager = new QNetworkAccessManager();
}

DownloadManager::~DownloadManager()
{
    m_pManager->deleteLater();
}

//==========添加cookie
void DownloadManager::setCookie(QString strCookie){
    m_strCookie = strCookie;
}

//==========下载http资源文件,异步
void DownloadManager::DownloadSrc_Asyn(QString strUrl,QString strSavePath,QString strReferer){
    m_DownloadType = 1;
    m_strUrl = strUrl;
    m_strPath = strSavePath;

    if(m_IsCoverFile == false){
        if(QFile::exists(strSavePath)) {
            n_DownloadError = -2;
            m_IsSuccess = false;
            return;
        }
    }
    if(m_pFile == NULL){
        m_pFile = new QFile(strSavePath);
        if (!m_pFile->open(QIODevice::WriteOnly)) {
            n_DownloadError = -3;
            m_IsSuccess = false;
            delete m_pFile;
            return;
        }
    }

    if(strReferer.length() == 0){
        strReferer = strUrl;
    }
    m_strReferer = strReferer;

    QNetworkRequest Request;
    Request.setUrl(QUrl(strUrl));
    Request.setRawHeader("Referer",strReferer.toStdString().c_str());
    Request.setRawHeader("User-Agent","Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)");


    if(m_strCookie != ""){
        Request.setRawHeader("Cookie",m_strCookie.toStdString().c_str());
    }
    QNetworkReply *pReply = m_pManager->get(Request);

    connect(pReply,SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(slotError()));
    connect(pReply,SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(downloadProgress(qint64,qint64)));
    connect(pReply,SIGNAL(finished()),this,SLOT(replyFinished()));
}

//==========下载http资源文件
bool DownloadManager::DownloadSrc(QString strUrl,QString strSavePath,QString strReferer)
{
    DownloadSrc_Asyn(strUrl,strSavePath,strReferer);
    if(m_DownloadFaildNum == 0){
        if(m_Event.isRunning() == false){
            m_Event.exec();
        }
    }

    return m_IsSuccess;
}

//==========http Get方式获取网页源码,异步
void DownloadManager::HttpGetData_Asyn(QString strUrl){
    //qDebug("Start A HTTP Get Task:%s",strUrl.toStdString().c_str());
    m_DownloadType = 2;
    m_strUrl = strUrl;


    QNetworkRequest Request;
    Request.setUrl(QUrl(strUrl.toUtf8()));
    Request.setRawHeader("Referer",strUrl.toStdString().c_str());
    Request.setRawHeader("User-Agent","Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)");
    Request.setRawHeader("Accept-Encoding","gzip");
    if(m_strCookie != ""){
        Request.setRawHeader("Cookie",m_strCookie.toStdString().c_str());
    }

    QNetworkReply *pReply = m_pManager->get(Request);

    connect(pReply,SIGNAL(error(QNetworkReply::NetworkError)),this,SLOT(slotError()));
    connect(pReply,SIGNAL(downloadProgress(qint64,qint64)),this,SLOT(downloadProgress(qint64,qint64)));
    connect(pReply,SIGNAL(finished()),this,SLOT(replyFinished()));

}
//==========http Get方式获取网页源码
bool DownloadManager::HttpGetData(QString strUrl){
    HttpGetData_Asyn(strUrl);

    if(m_DownloadFaildNum == 0){
        if(m_Event.isRunning() == false){
            m_Event.exec();
        }
    }

    return m_IsSuccess;
}

//==========http Post方式获取网页源码,异步
void DownloadManager::HttpPostData_Asyn(QString strUrl,QString strData){
    //qDebug("Start A HTTP Post Task");

    m_DownloadType = 3;
    m_strUrl = strUrl;
    m_strPostData = strData;

    QNetworkRequest Request;
    Request.setUrl(QUrl(strUrl));
    Request.setRawHeader("Referer",strUrl.toStdString().c_str());
    Request.setRawHeader("User-Agent","Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)");
    Request.setRawHeader("Accept-Encoding","gzip");
    if(m_strCookie != ""){
        Request.setRawHeader("Cookie",m_strCookie.toStdString().c_str());
    }
    QNetworkReply *pReply = m_pManager->post(Request,strData.toLocal8Bit());

    connect(pReply,SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(slotError()));
    connect(pReply,SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(downloadProgress(qint64,qint64)));
    connect(pReply,SIGNAL(finished()),this,SLOT(replyFinished()));
}

//==========http Post方式获取网页源码
bool DownloadManager::HttpPostData(QString strUrl,QString strData){

    HttpPostData_Asyn(strUrl,strData);

    if(m_DownloadFaildNum == 0){
        if(m_Event.isRunning() == false){
            m_Event.exec();
        }
    }
    return m_IsSuccess;
}


//==========任务完成信号
void DownloadManager::replyFinished()
{
    QNetworkReply* pReply = (QNetworkReply*)sender();

    if(pReply->hasRawHeader("Location")){
        QString strLocation = pReply->header(QNetworkRequest::LocationHeader).toString();
        switch(m_DownloadType){
        case 1:
            DownloadSrc(strLocation,m_strPath,m_strReferer);
            break;
        case 2:
            HttpGetData(strLocation);
            break;
        case 3:
            HttpPostData(strLocation,m_strPostData);
            break;
        }
        return;
    }

    if(n_DownloadError == 0){
        switch(m_DownloadType){
        case 1:{
            if (m_pFile){
                m_pFile->write(pReply->readAll());
                m_pFile->close();
                delete m_pFile;
                m_pFile = NULL;
            }else{
                n_DownloadError = -4;
            }
        }
            break;
        case 2:
        case 3:
        {
            QByteArray strHttpData;
            //-----------GZIP解压
            if(pReply->rawHeader("Content-Encoding").toUpper() == "GZIP"){
                strHttpData = BaseLib::gzipDecompress(pReply->readAll());
            }else{
                strHttpData = pReply->readAll();
            }

            //-----------判断网页编码
            QString m_strContentCodec = pReply->rawHeader("Content-Type");
            if(BaseLib::subStr_Str(m_strContentCodec,"charset=","\0",m_strContentCodec) == false){
                m_strContentCodec = BaseLib::GetHtmlCharset(&strHttpData);
            }
            //-----------转换为unicode qstring
            QTextCodec *codec = QTextCodec::codecForName(m_strContentCodec.toStdString().c_str());
            //-----------如果不存在这个编码,那就让他默认为utf8
            if(codec == NULL){
                codec = QTextCodec::codecForName("utf8");
            }
            m_strHtmlData = codec->toUnicode(strHttpData.data());
            strHttpData.clear();
        }
            break;
        }
        //qDebug("任务完成:%s",m_strUrl.toStdString().c_str());
        downloadFinished(true);
    }else{
        if(n_DownloadError == 203){
            qDebug("Page 404,HttpGetFaild");
            downloadFinished(false);
        }else{
            if(m_DownloadFaildNum < m_DownloadFaildRetryNum){
                m_DownloadFaildNum++;

                n_DownloadError = 0;
                qDebug("%dth/%d failed,retry",m_DownloadFaildNum,m_DownloadFaildRetryNum);

                switch(m_DownloadType){
                case 1:
                    DownloadSrc(m_strUrl,m_strPath);
                    break;
                case 2:
                    HttpGetData(m_strUrl);
                    break;
                case 3:
                    HttpPostData(m_strUrl,m_strPostData);
                    break;
                }
            }else{
                qDebug("Already try %d times,still fails,quit\r\nquit url:%s",m_DownloadFaildRetryNum,m_strUrl.toStdString().c_str());
                downloadFinished(false);
            }
        }
    }
    pReply->deleteLater();
}
//==========下载完成
bool DownloadManager::downloadFinished(bool bResult)
{
    m_IsFinished = true;
    m_IsSuccess = bResult;
    m_Event.exit(0);
    emit DownloadManagerFinished(m_IsSuccess);
    return bResult;
}
//==========发生错误信号
void DownloadManager::slotError(){
    QNetworkReply* pReply = (QNetworkReply*)sender();
    n_DownloadError = pReply->error();
    emit DownloadManagerError(n_DownloadError);
    pReply->deleteLater();

    qDebug("DownloadManager slotError:%d\r\n",n_DownloadError);
}

//==========详细进度
void DownloadManager::downloadProgress(qint64 bytesReceived, qint64 bytesTotal){
    m_DownloadReceived = bytesReceived;
    m_DownloadTotal    = bytesTotal;
    emit DownloadManagerProgress(m_DownloadReceived,m_DownloadTotal);
}

//==========获取get 或 post 得到的html
QString DownloadManager::GetHtmlData()
{
    return m_strHtmlData;
}

//==========设置html
void DownloadManager::setHtmlData(QString strHtml)
{
    m_strHtmlData = strHtml;
}

//==========获取cookie
QString DownloadManager::GetCookie()
{
    return m_strCookie;
}

//==========设置重试次数
void DownloadManager::setFaildRetryNum(int n)
{
    m_DownloadFaildRetryNum = n;
}

//==========下载是否完成
bool DownloadManager::DownloadIsFinished()
{
    return m_IsFinished;
}

[hit1er]

我怎么找也找不出那个地方没释放的,这个类我主要是放在线程里面调用,我是循环同时开启30个线程抓取网页,30个都完了之后,继续开启30个抓取另一批网页,内存看上去没有增加,就是这个句柄数不断增加

我在线程里放加入其他代码,怎么执行也没问题,就是一旦有了

m_pManager->get(Request);

这句,这句柄数就不会减少

[kinzess123]

[Quote=引用 13 楼 的回复:]

我怎么找也找不出那个地方没释放的,这个类我主要是放在线程里面调用,我是循环同时开启30个线程抓取网页,30个都完了之后,继续开启30个抓取另一批网页,内存看上去没有增加,就是这个句柄数不断增加

我在线程里放加入其他代码,怎么执行也没问题,就是一旦有了

m_pManager->get(Request);

这句,这句柄数就不会减少
[/Quote]

线程里没有event loop,而deleteLate需要进入,
在线程代码后面加入exec();
然后..想办法退出线程...

记得没错的话就是这样..

[yyxmm]

很久的帖子了，我也遇到了这个问题，我的代码可能更规范些，但是还是很严重啊，用一个按钮，点一次，发送一次，就能很明显的看出来，内存不断增大

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QDebug>

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    manager = new QNetworkAccessManager;
    connect(manager, SIGNAL(finished(QNetworkReply*)), this, SLOT(onfinished(QNetworkReply*)));
}

MainWindow::~MainWindow()
{
    delete ui;
}

void MainWindow::onfinished(QNetworkReply *reply)
{
    qDebug() << reply->readAll();
    reply->abort();
    reply->close();
    reply->deleteLater();
}

void MainWindow::on_pushButtonLogin_clicked()
{
    isNeedVcode();
}

int MainWindow::isNeedVcode()
{
    QNetworkRequest request;
    //QString strurl = QString("http://www.baidu.com");
    QString strurl = QString("http://check.ptlogin2.qq.com/check?&uin=870603663");
    request.setUrl(QUrl(strurl));
    //
    request.setRawHeader("User-Agent", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)");
    //
    manager->get(request);

    return 0;
}