如何用Java提取网页中的信息？

[孟祥龙]

Perl能够很好提取网页中的信息，那么Java也能象它一样强大吗？
如果给定一个URL，是不是Java会有特定的方法直接就获得到页面的内容了？
使用java会很麻烦吗？
希望各位多多指点

[全粘架构师]

/*
* 创建日期 2005-5-31
*
*/
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;

import javax.swing.text.html.HTMLEditorKit.ParserCallback;
import javax.swing.text.html.parser.ParserDelegator;

/**
* @author Raito
*
*/
public class Scan {
private URL currentUrl;
private ParserCallback parse=new ParserCallback();

public Scan(String url){
ParserDelegator pd=new ParserDelegator();
try {
currentUrl=new URL(url);
URLConnection con=currentUrl.openConnection();
HttpURLConnection httpcon=(HttpURLConnection) con;
if(httpcon.getResponseCode()==HttpURLConnection.HTTP_OK){
if(httpcon.getContentType().startsWith("text/html")){
InputStreamReader in=new InputStreamReader(con.getInputStream());
BufferedReader buf=new BufferedReader(in);
//这个循环可以用while(true)
for(int i=0;i<50;i++){
String s=buf.readLine();
System.out.println(s);
}
//pd.parse(buf,parse=new ParserCallback(),false);
buf.close();
in.close();
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String [] args){
Scan s=new Scan("http://www.163.com");
}
}

[孟祥龙]

多谢hellwindy(夜神·月) ！让我看到了希望阿，呵呵！

不过我运行的时候，走到httpcon.getResponseCode()的时候出错了

java.net.UnknownHostException: www.163.com
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at src.Scan.<init>(Scan.java:52)
at src.Scan.main(Scan.java:74)

您知道这是什么原因吗？谢！

[onkyo1]

可以用 html parser 库
http://htmlparser.sourceforge.net/

[孟祥龙]

刚才那个错应该是我用代理服务器上网的原因，不过等我设置代理之后不出错了，但是httpcon.getResponseCode()取的值是-1，不等于HttpURLConnection.HTTP_OK，这又是为什么？

谢谢onkyo1(onkyo1)，别笑我太弱了，我都不知道应该怎么用html parser库，能说说吗？