用jsoup爬虫出现点状况

kangxinya 2019-03-22 12:02:38

本来想用jsoup爬一下题库，但是爬到的内容总是重复，有朋友能帮我看看问题出在哪吗

public class MainActivity extends AppCompatActivity {

    private static final String TAG = "k";



    @Override

    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_main);



        new Thread(new Runnable() {

            @Override

            public void run() {

                jsoupdata();

            }

        }).start();



    }



    private void jsoupdata() {

        try {

            Document doc = Jsoup.connect("https://mp.weixin.qq.com/s/fFOGqYmsrzNKlEE7cXcYFA").get();

            Elements titleAndPic = doc.select("title");

            Elements hdate=doc.select("div#js_content").select("span");

            Log.d(TAG,"html:"+titleAndPic.text());

            Log.d(TAG,"html"+hdate.text());//《-就是这个

        } catch (Exception e) {

            Log.i("mytag", e.toString());

        }

    }

}

这是爬的span标签的一部分内容，出现了很多重复，如“包含（”，“同心同德和融共生”

2019-03-22 10:43:53.820 6503-6527/com.example.myapplication D/k: html1.北京公交集团企业文化的 “同行”内涵 包括（ 包括（ A ） ） A. 同心同德 和融共生 同心同德 和融共生  行无止境 追求卓越 行无止境 追求卓越

这是爬到的html出错那段的代码

<span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;">1.北京公交集团企业文化的</span>

    <span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;">“同行”内涵</span>

    <span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">包括（</span></span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">包括（</span>

    <span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;">A</span>

    <span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">）</span></span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">）</span>

    <span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;">A.</span>

    <span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">同心同德</span>  <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">和融共生</span></span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">同心同德</span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">和融共生</span>

    <span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;">  </span>

    <span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">行无止境</span>  <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">追求卓越</span></span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">行无止境</span>

    <span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">追求卓越</span>

这是用浏览器相关的一段代码

<strong style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;">1.北京公交集团企业文化的</span></strong><strong style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;">“同行”内涵</span></strong><strong style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">包括（</span></span></strong><strong style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;font-family: 宋体;color: rgb(255, 0, 0);box-sizing: border-box !important;word-wrap: break-word !important;">A</span></strong><strong style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;font-family: 宋体;box-sizing: border-box !important;word-wrap: break-word !important;"><span style="max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">）</span></span></strong>

高手们看看是怎么回事，我在网上找了好长时间也没找到是哪出的问题

...全文