jdbc读取大表优化

[老程序员]

大家好，
我使用jdbc读取上几千W的数据表到文本文件。数据库是sql server的。请问有没有什么优化办法。或者从哪一些方面来优化。

[跑步的小伙子]

1.新建File
2.新建FileOutPutStream
3.循环取出小量的数据，分批写入数据库，直到数据库数据取完为止
4.然后再关闭FileOutPutStream之类的对象.
5.最好是异步的处理数据。

[hchjjun]

文本文件太大操作也比较慢，可以分多个文本文件存储。

[laorer]

分析瓶颈，如果查询比较复杂，就用存储过程，由sql server本身来优化，
还有考虑块操作，多少条数据写到内存之后再写到文件中，而是每条数据来了就直接写文件中

暂时想到的就这么多

[lewis219]

可以采用分批的方法，比如先取一万条数据，取完直接写入文本文件，然后再取下一万条再写入第二个文本文件，以此类推，这个取多少条数据可以根据情况自己定

[sunnylyy]

最好的办法就是不用jdbc,用数据库本身提供的功能

[老程序员]

因为不在windows下运行，所以，sql server 自带的工具没有办法实现
导出的是放在一个文件中。

[lewis219]

分批读分批写，这样后面读写的时候内存就可以回收前面读写时占用的内存，不至于内存溢出

[sunnylyy]

[Quote=引用 5 楼 rovige 的回复:]
因为不在windows下运行，所以，sql server 自带的工具没有办法实现
导出的是放在一个文件中。
[/Quote]

用bcp命令

[老程序员]

bcp能在liunx下运行吗？

[junyun414]

超级新手，进来围观一下。

[老程序员]

Work Report
Hello everyone! I'm glad to have the chance to make a presentation in such a great meeting! Although we are a new team, I can feel a lot of passion while I'm working in this team.We have worked together for 1 months, it was a little tired, but it is worthy to have my abilities improved in such short time.
Currently, we are studying business of the CCBM system, and I joined every training course. I think study has become a happy thing here. It will be great if I can make full use of what I learned in the future. Next, let me introduce a bussiness called Family and Friends Number service which I learned recently.
What is Family and Friends Number service? If you make phone calls to your parents or friends quite often, you may need this service. By using this service you cost on phone calls can be significantly decreased. Therefore, a lot of people will like this service.
What need to do before join the service? The system will check whether your number is blacklisted, or you have payment dued and so on. If everything is ok, the number will be added in a Family group. We can also config how many members can be added in a family group. If a family group has reached its limit, customer will need to pay if they want to add more numbers into it.
Thanks!