C# 多线程调用WebClient速度变慢的问题

[zhihuiqian]

不使用线程速度很快，创建多个线程序调用时就变得很慢，不知什么地方出问题了，麻烦高手帮下！
代码如下
private string getHtml(string url)
{
string restr = "";
WebClient myWebClient = new WebClient();
try
{
byte[] myDataBuffer = myWebClient.DownloadData(url);
restr = Encoding.Default.GetString(myDataBuffer);
myDataBuffer = null;
}
}
catch (Exception ex)
{
restr = "";
ErrlogIt("getHtml err:" + ex.Message);
}
finally
{
myWebClient.Dispose();
}
return restr;
}

private void timerMO_Tick(object sender, EventArgs e)
{
this.timerMO.Enabled = false;

if (putmoBool)
{
thMO = new Thread(putMO);
thMO.Name = "thMO";
thMO.Start();
frmMain.CheckForIllegalCrossThreadCalls = false;
}
this.timerMO.Interval = cInterval;
this.timerMO.Enabled = true;
}

private void putMO()
{
putmoBool = false;

DataSet dsMoUrl = new DataSet();
try
{
string sqlUrl = "select corpid,subnumber,mourl,moflag from corpurl where len(mourl)>10 and moflag<=0 order by moflag desc";
dsMoUrl = Query(sqlUrl);
}
catch (Exception ex)
{
ErrlogIt(ex.Message);
}

if (dsMoUrl == null)
{
putmoBool = true;
return;
}
if (dsMoUrl.Tables.Count == 0)
{
putmoBool = true;
return;
}

try
{
actionMo(dsMoUrl);
}
catch (Exception ex)
{
ErrlogIt(ex.Message);
}
finally
{

dsMoUrl.Dispose();
putmoBool = true;
}
}
private void actionMo(DataSet dsMoUrl)
{
bool canGo = true;
int ct = 0;
int i = 0;
ct = dsMoUrl.Tables[0].Rows.Count;
while (i < ct && canGo)
{
DataRow dr = dsMoUrl.Tables[0].Rows[i];
string corpid = dr["corpid"].ToString();
string subnumber = dr["subnumber"].ToString();
string mourl = dr["mourl"].ToString();
int moflag = int.Parse(dr["moflag"].ToString());

DataSet dsMoList = new DataSet();
if (canGo)
{
try
{
string QuerySql = "select top " + pushSpeed.ToString() + " * from mo.motocorp where corpID=" + corpid + " and subnumber=" + subnumber + " order by moflag asc,moid asc";
dsMoList = Query(QuerySql);
}
catch (Exception ex)
{
canGo = false;
ErrlogIt(ex.Message);
}
}

if (canGo)
{
int j = 0;
int listCt = dsMoList.Tables[0].Rows.Count;
string urlResp = "";
string respValue = "";

while (j < listCt && canGo)
{
DataRow listDr = dsMoList.Tables[0].Rows[j];
string moid = listDr["moid"].ToString();

string sUrl = mourl + "?msgFormat=1&moID=" + moid

try
{
urlResp = getHtml(sUrl);
}
catch
{
urlResp = "";
logIt("失败!");
this.labErr.Text = (int.Parse(this.labErr.Text.ToString()) + 1).ToString();
}

try
{
if (urlResp.Length < 3)
urlResp = urlResp + " ";

respValue = urlResp.Substring(0, 3);
if (respValue.Trim() == "100")
{
logIt("moOK!");
}
else
{
logIt("moNO!");
canGo = false;
}
}
catch (Exception ex)
{
ErrlogIt(ex.Message);
}

j = j + 1;
}

}
dsMoList.Dispose();
i = i + 1;
}
}

[zhihuiqian]

程序有多个TIMER，并且都有调用getHtml(string url) 方法，如果不使用线程速度很快，使用超过两个线程就变慢了

[jerry_zhang99]

程序有点长，没有找到重点，是不是因为请求同一个url的问题啊

[zhihuiqian]

URL没有问题，不使用线程getHTML速度很快

[_see_you_again_]

this.timerMO.Interval = cInterval;
this.timerMO.Enabled = true; //重新开始计时

[卧_槽]

你家带宽不够用楽

[zhihuiqian]

byte[] myDataBuffer = myWebClient.DownloadData(url);
监控全部卡在这个地方，带宽8M的，不使用线程很快的

[_see_you_again_]

if (putmoBool)//第一个线程已经在putMO()设置为false，故本次触发跳过等待下次触发。
{
thMO = new Thread(putMO);

[_see_you_again_]

[Quote=引用 6 楼 zhihuiqian 的回复:]
byte[] myDataBuffer = myWebClient.DownloadData(url);
监控全部卡在这个地方，带宽8M的，不使用线程很快的
[/Quote]
如果同时下载数据会将所有任务时间延迟，就像你用迅雷同时下载多个文件，一共就那么点带宽，它只能平均分配。

[_see_you_again_]

用多线程下载文件虽然利用了99%的CPU使用，但是它涉及到占用所有的带宽，所以同时下载几个文件加起来的带宽是一样的，跟多线程没关系。比如同时下载两个都是1M的文件,这样就相当于要下载2M,当然比下载一个1M的慢了。

[zhihuiqian]

那应该如何解决这个问题！！

[mywisest]

ServicePointManager.DefaultConnectionLimit = 512;

[charles_y]

When a connection is closed, on the side that is closing the connection the 5 tuple { Protocol, Local IP, Local Port, Remote IP, Remote Port} goes into a TIME_WAIT state for 240 seconds by default. In this case, the protocol is fixed - TCP the local IP, remote IP and remote PORT are also typically fixed. So the variable is the local port. What happens is that when you don't bind, a port in the range 1024-5000 is used. So roughly you have 4000 ports. If you use all of them in 4 minutes - meaning roughly you make 16 web service calls per second for 4 minutes you will exhaust all the ports. That is the cause of this exception.

OK now how can this be fixed?

One of the ways is to increase the dynamic port range. The max by default is 5000. You can set this up to 65534. HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort is the key to use.

The second thing you can do is once the connection does get into an TIME_WAIT state you can reduce the time it is in that state, Default is 4 minutes, but you can set this to 30 seconds HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelay is the key to use. Set this to 30 seconds

[charles_y]

ServicePointManager.DefaultConnectionLimit = 512;

很对，一般这个值缺省为2