利用httpwebrequest登陆csdn出现的问题

2010-06-04 06:05:32 2.7k 阅读 AI 写同款 · GEO 优化AI 选题数据监测一键分发 ›

Unity 升级引擎版本后构建报错：script class layout is incompatible between the editor and the player Unity升级引擎版本后，项目报错：script class layout is incompatible between the editor and the player。阅读详情

我利用httpwebrequest登陆老是实现不了。不知道为什么。
我说一下我的流程。

登陆userlogin.aspx页面，将页面的cookie保存，然后使用保存的cookie到验证码页面将验证码下载下来

接着继续利用cookie post数据到表单。
但是没有成功。不知道是什么原因。
代码如下。

        /// <summary>

        /// 通过get方式请求页面，传递一个实例化的cookieContainer

        /// </summary>

        /// <param name="postUrl"></param>

        /// <param name="cookie"></param>

        /// <returns></returns>

        public static ArrayList GetHtmlData(string postUrl, CookieContainer cookie)

        {

            HttpWebRequest request;

            HttpWebResponse response;

            ArrayList list = new ArrayList();

            request = WebRequest.Create(postUrl) as HttpWebRequest;

            request.Method = "GET";

            request.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2; CIBA; .NET4.0C; .NET4.0E)";

            request.CookieContainer = cookie;

            request.KeepAlive = true;

            request.Accept = "image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";

            try

            {

                //获取服务器返回的资源

                using (response = (HttpWebResponse)request.GetResponse())

                {

                    using (StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.Default))

                    {

                        if (response.Cookies.Count > 0)

                            cookie.Add(response.Cookies);

                        //保存Cookies

                        list.Add(cookie);

                        list.Add(reader.ReadToEnd());

                        list.Add(Guid.NewGuid().ToString());//图片名

                    }

                }

            }

            catch (WebException ex)

            {

                list.Clear();

                list.Add("发生异常\n\r");

                WebResponse wr = ex.Response;

                using (Stream st = wr.GetResponseStream())

                {

                    using (StreamReader sr = new StreamReader(st, System.Text.Encoding.Default))

                    { 

                        list.Add(sr.ReadToEnd());

                    }

                }

            }

            catch (Exception ex)

            {

                list.Clear();

                list.Add("5");

                list.Add("发生异常：" + ex.Message);

            }

            return list;

        }

/// <summary>

        /// 下载验证码图片并保存到本地

        /// </summary>

        /// <param name="Url">验证码URL</param>

        /// <param name="cookCon">Cookies值</param>

        /// <param name="savePath">保存位置/文件名</param>

        public static bool DowloadCheckImg(string Url, CookieContainer cookCon, string savePath)

        {

            bool bol = true;

            HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(Url);

            webRequest.Referer = "http://passport.csdn.net/UserLogin.aspx";

            webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.2; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2; CIBA; .NET4.0C; .NET4.0E)";

            webRequest.KeepAlive = true;

            webRequest.Accept = "*/*";

            webRequest.CookieContainer = cookCon;

            try

            {

                //获取服务器返回的资源

                using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())

                {

                    using (Stream sream = webResponse.GetResponseStream())

                    {

                        List<byte> list = new List<byte>();

                        while (true)

                        {

                            int data = sream.ReadByte();

                            if (data == -1)

                                break;

                            list.Add((byte)data);

                        }

                        File.WriteAllBytes(savePath, list.ToArray());

                    }

                }

            }

            catch (WebException ex)

            {

                bol = false;

            }

            catch (Exception ex)

            {

                bol = false;

            }

            return bol;

        }

        /// <summary>

        /// 发送相关数据至页面

        /// 进行登录操作

        /// 并保存cookie

        /// </summary>

        /// <param name="postData"></param>

        /// <param name="postUrl"></param>

        /// <param name="cookie"></param>

        /// <returns></returns>

        public static ArrayList PostData(string postData, string postUrl, CookieContainer cookie)

        {

            ArrayList list = new ArrayList();

            HttpWebRequest request;

            HttpWebResponse response;

            ASCIIEncoding encoding = new ASCIIEncoding();

            request = WebRequest.Create(postUrl) as HttpWebRequest;

            byte[] b = encoding.GetBytes(postData);

            request.Method = "POST";

            request.CookieContainer = cookie;

            request.ContentLength = b.Length;

            using (Stream stream = request.GetRequestStream())

            {

                stream.Write(b, 0, b.Length);

            }

            try

            {

                //获取服务器返回的资源

                using (response = request.GetResponse() as HttpWebResponse)

                {

                    using (StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8))

                    {

                        if (response.Cookies.Count > 0)

                            cookie.Add(response.Cookies);

                        list.Add(cookie);

                        list.Add(reader.ReadToEnd());

                    }

                }

            }

            catch (WebException wex)

            {

                WebResponse wr = wex.Response;

                using (Stream st = wr.GetResponseStream())

                {

                    using (StreamReader sr = new StreamReader(st, System.Text.Encoding.Default))

                    {

                        list.Add(sr.ReadToEnd());

                    }

                }

            }

            catch (Exception ex)

            {

                list.Add("发生异常\n\r"+ex.Message);

            }

            return list;

        }

主要的3个方法。不需要登陆的话可以抓取到数据，不知道为什么这样就不可以。
求解。

...全文

别再乱按F8了！Win10系统修复的正确姿势：用管理员CMD运行sfc /scannow 本文详细介绍了Windows 10系统修复的正确方法，重点推荐使用管理员CMD运行`sfc /scannow`命令。相比传统的F8键和重装系统，这一方法更高效且安全，能解决大多数系统文件损坏问题。文章还提供了管理员CMD的启动方式、sfc命令的完整参数解析以及实战故障排除流程图，帮助用户快速恢复系统健康。阅读详情

vip__888

.NET社区 62,272 成员 668,968 内容

+ 关注 + 加入社区

切换为时间正序

jce195447 2013-01-25

打赏
举报

Mark一下以作备用。

hylaw 2012-11-05

打赏
举报

引用 16 楼 minyskirt 的回复:

楼主，根据你的意思，我自己用代码写了一个CSDN的登录器，发给你参考一下，下面说说具体的思路. 首先，我们先定义一些变量： C# code 123456789 private const string NET_SESSIONID= "ASP.NET_SessionId="; private const string CLIENTK……

好人！

fsdr3h5 2012-10-31

打赏
举报

[Quote=引用 16 楼的回复:]
楼主，根据你的意思，我自己用代码写了一个CSDN的登录器，发给你参考一下，下面说说具体的思路.

首先，我们先定义一些变量：
C# code

private const string NET_SESSIONID= "ASP.NET_SessionId=";
private const string CLIENTKEY = "ClientKey=";

……
[/Quote]
Login_Load(object sender, EventArgs e)方法是什么意思？自定义的？

ak_ljd 2011-07-20

打赏
举报

你好，问一下怎样通过这种方式实现IE中打开登录后的跳转页？在asp.net web中

jianshao810 2011-02-16

打赏
举报

是搞掂没有啊？楼主

liaoyukun111 2010-10-20

打赏
举报

学习下

ziyan688999 2010-07-25

打赏
举报

操啊。根本不行啊。

xupeihuagudulei 2010-06-05

打赏
举报

去codeproject上找些文章对比下就清楚鸟。

minyskirt 2010-06-05

打赏
举报

楼主，根据你的意思，我自己用代码写了一个CSDN的登录器，发给你参考一下，下面说说具体的思路.

首先，我们先定义一些变量：



        private const string NET_SESSIONID= "ASP.NET_SessionId=";

        private const string CLIENTKEY = "ClientKey=";



        string aspcookie = "";

        private string html = "";

        private string sessionId  = "";

        private string clientKey = "";

        private string viewState = "";

一、利用HttpWebRequest获取CSDN的登录信息，如ASP.NET_SessionId、ClientKey、__VIEWSTATE。这步可以放在Login_Load中，下面是代码：

 

        private void Login_Load(object sender, EventArgs e)

        {

            HttpWebRequest request = WebRequest.Create("http://passport.csdn.net/UserLogin.aspx") as HttpWebRequest;

            request.Credentials = CredentialCache.DefaultCredentials;

            request.Accept = "*/*";

            request.Referer = "http://passport.csdn.net/UserLogin.aspx";

            request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)";

            request.Method = "GET";

            request.KeepAlive = true;

            request.Headers.Set("Accept-Language", "zh-CN");

            request.Headers.Set("Accept-Encoding", "gzip, deflate");



            HttpWebResponse response = request.GetResponse() as HttpWebResponse;

            System.IO.Stream responseStream = response.GetResponseStream();

            System.IO.StreamReader reader = new System.IO.StreamReader(responseStream, Encoding.GetEncoding("utf-8"));

            html = reader.ReadToEnd();



            string viewStateFlag = "id=\"__VIEWSTATE\" value=\"";

            int i = html.IndexOf(viewStateFlag) + viewStateFlag.Length;

            int j = html.IndexOf("\"", i);

            viewState = html.Substring(i, j - i);



            aspcookie = response.Headers.Get("Set-Cookie");

            sessionId = GetSessionId();

            clientKey = GetClientKey();

            // 获取图片验证码

            GetCode();

        }

         /// <summary>

        /// 获取ClientKey

        /// </summary>

        /// <returns></returns>

        private string GetClientKey()

        {

            // 这个仅仅是用来获取ClientKey值的长度

            string id = "bb32434c-3bb3-4e44-92c3-8952f631ca87";

            int index = aspcookie.IndexOf(CLIENTKEY) + CLIENTKEY.Length;

            String str = aspcookie.Substring(index, id.Length);

            return str;

        }



        /// <summary>

        /// 获取ASP.NET_SessionId=

        /// </summary>

        /// <returns></returns>

        private string GetSessionId()

        {

            string id = "5mhl0tvbw5shlpnhxgwnck45";

            int index = aspcookie.IndexOf(NET_SESSIONID) + NET_SESSIONID.Length;

            String str = aspcookie.Substring(index, id.Length);

            return str;

        }

二、获取图片验证码，在这一步中需要注意的是，因为我们用的是另外一个HttpWebRequest去获取，所以为了保持获取的验证码与登录页面的Session一致，我们需要把Session等信息放入Header中去，下面是代码：



 /// <summary>

        /// 产生新的验证码

        /// </summary>

        private void GetCode()

        {

            try

            {

                CookieCollection cookies = new CookieCollection();

                cookies.Add(new Cookie("ASP.NET_SessionId",sessionId));

                cookies.Add(new Cookie("ClientKey",clientKey));



                HttpWebRequest httpWebRequest;

                HttpWebResponse webResponse;

                byte[] byteRequest = { };

                httpWebRequest = (HttpWebRequest)HttpWebRequest.Create("http://passport.csdn.net/ShowExPwd.aspx");

                CookieContainer co = new CookieContainer();

                co.Add(new Uri("http://passport.csdn.net"), cookies);

                httpWebRequest.CookieContainer = co;

                httpWebRequest.Accept = "*/*";

                httpWebRequest.Referer = "http://passport.csdn.net/UserLogin.aspx";

                httpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)";

                httpWebRequest.Method = "GET";

                httpWebRequest.Headers.Set("Accept-Encoding", "gzip, deflate");

                httpWebRequest.Headers.Set("Accept-Language", "zh-CN");

                httpWebRequest.KeepAlive = true;



                webResponse = (HttpWebResponse)httpWebRequest.GetResponse();

                Stream responseStream = webResponse.GetResponseStream();

                Image original = Image.FromStream(responseStream);

                Bitmap bitMap = new Bitmap(original);

                this.pictureBox1.Image = bitMap;

                responseStream.Close();

            }

            catch (Exception exception)

            {

                MessageBox.Show("ERROR:" + exception.Message);

            }

        }

三、最后一步是登录事件，登录很简单，把相关的信息放入请求数据中就行了。



private void btnLogin_Click(object sender, EventArgs e)

        {

            HttpWebRequest httpWebRequest;

            HttpWebResponse webResponse;



            string randnum = txtRandnum.Text;

            string password = txtPassword.Text;

            string loginname = txtUserName.Text;



            string postData = "ctl00$CPH_Content$tb_LoginNameOrLoginEmail={0}&ctl00$CPH_Content$tb_Password={1}&ctl00$CPH_Content$tb_ExPwd={2}&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE={3}&ClientKey={4}&from={5}&MailParameters=&PrePage=&ctl00$CPH_Content$Image_Login.x=46&ctl00$CPH_Content$Image_Login.y=9";

            postData = string.Format(postData, System.Web.HttpUtility.UrlEncode(loginname),System.Web.HttpUtility.UrlEncode(password),System.Web.HttpUtility.UrlEncode(randnum), System.Web.HttpUtility.UrlEncode(viewState), clientKey, System.Web.HttpUtility.UrlEncode("http://hi.csdn.net/"));

            byte[] byteRequest = Encoding.UTF8.GetBytes(postData);



            CookieCollection cookies = new CookieCollection();

            cookies.Add(new Cookie("ASP.NET_SessionId", sessionId));

            cookies.Add(new Cookie("ClientKey", clientKey));

            cookies.Add(new Cookie("UN",loginname));



            httpWebRequest = (HttpWebRequest)HttpWebRequest.Create("http://passport.csdn.net/UserLogin.aspx");

            CookieContainer co = new CookieContainer();

            co.Add(new Uri("http://passport.csdn.net"), cookies);

            httpWebRequest.CookieContainer = co;

            httpWebRequest.ContentType = "application/x-www-form-urlencoded";

            httpWebRequest.Accept = "image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*";

            httpWebRequest.Referer = "http://passport.csdn.net/UserLogin.aspx";

            httpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)";

            httpWebRequest.Method = "POST";

            httpWebRequest.KeepAlive = true;

            httpWebRequest.Headers.Set("Cache-Control", "no-cache");

            httpWebRequest.Headers.Set("Accept-Encoding", "gzip, deflate");

            httpWebRequest.Headers.Set("Accept-Language", "zh-CN");



            httpWebRequest.ContentLength = byteRequest.Length;

            Stream stream;

            stream = httpWebRequest.GetRequestStream();

            stream.Write(byteRequest, 0, byteRequest.Length);

            stream.Close();

            webResponse = (HttpWebResponse)httpWebRequest.GetResponse();

            System.IO.Stream responseStream = webResponse.GetResponseStream();



            String header = webResponse.Headers.ToString();

            Stream getStream = webResponse.GetResponseStream();

            long contentLength = webResponse.ContentLength;



            byte[] outBytes = new byte[contentLength];

            outBytes = ReadFully(getStream);

            getStream.Close();



            getStream = new MemoryStream(outBytes);

            StreamReader streamReader = new StreamReader(getStream, Encoding.UTF8);

            string getString = streamReader.ReadToEnd();

            streamReader.Close();

            getStream.Close();



        }



        static byte[] ReadFully(Stream stream)

        {

            byte[] buffer = new byte[128];

            using (MemoryStream ms = new MemoryStream())

            {

                while (true)

                {

                    int read = stream.Read(buffer, 0, buffer.Length);

                    if (read <= 0)

                        return ms.ToArray();

                    ms.Write(buffer, 0, read);

                }

            }

        }

其中，在如果把数据以及需要把哪些数据Post到服务器中花费了不少时间，后来利用HttpAnalyzer来抓取浏览器中登录的数据包与WinForm登录的数据包进行了对比，才找出问题所在。

vip__888 2010-06-05

打赏
举报

Justin-Liu 2010-06-04

打赏
举报

vip__888 2010-06-04

打赏
举报

[Quote=引用 12 楼 syeerzy 的回复:]
身在外，没Windows～不能帮你试一下，你自己试试吧，应该没问题。
[/Quote]
呵呵，谢谢

syeerzy 2010-06-04

打赏
举报

if (response.Cookies.Count > 0)
cookie.Add(response.Cookies);

这个地方有问题吧，你监视一下，这样可能会让cookie里面出现不该出现的反复叠加。
CookieContainer本身已经有收集Cookie的作用了，你只要保持这个对象作为上下文就可以了，应该不需要自己去手动赋值吧？

其实有更容易的方式，对你这样的应用你可以写个类继承自 WebClient ，然后让CookieContainer作为这个类的一个属性，然后重写GetRequest （也许不是叫这个名字，太久没碰aspnet了。。反正这个类可以重写的方法不多）在里面把CookieContainer赋值进去给这个Request（记忆中貌似要转一下类型）。然后就在任何地方都不要管跟Cookie相关的事情了。。。一切都做好了。

身在外，没Windows～不能帮你试一下，你自己试试吧，应该没问题。

syeerzy 2010-06-04

打赏
举报

1 是不是 postdata 这个字符串跟真正的内容有区别
2 是不是页面有 gzip

建议用Httpwatcher 之类的工具监视一下看看真正浏览器操作时到底发生了什么事情，跟你模拟的情况有什么出入。

vip__888 2010-06-04

打赏
举报

[Quote=引用 9 楼 sxldfang 的回复:]
public static ArrayList PostData(...)

不知道你返回的ArrayList用什么用，CookieContainer 保存有登陆后的信息，页面HTML中也可能含有以后访问的相关信息。
[/Quote]
我用集合保存了好几个信息
具体可以看代码

sxldfang 2010-06-04

打赏
举报

public static ArrayList PostData(...)

不知道你返回的ArrayList用什么用，CookieContainer 保存有登陆后的信息，页面HTML中也可能含有以后访问的相关信息。

vip__888 2010-06-04

打赏
举报

[Quote=引用 7 楼 wuyq11 的回复:]

这种方式登录其他网站是否成功
[/Quote]
也没有成功
就是不知道问题出在哪

wuyq11 2010-06-04

打赏
举报

这种方式登录其他网站是否成功

vip__888 2010-06-04

打赏
举报

vip__888 2010-06-04

打赏
举报

如梦怎么还没来

加载更多回复（3）

利用httpwebrequest登陆csdn出现的问题

相关推荐