hive做insert时卡在map阶段很久

allenpony 2014-02-26 09:58:32

在HIVE内执行一个很简单的操作insert into b select id from a ，发现会卡在map阶段很久
2014-02-26 06:05:56,875 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:06:57,200 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:07:57,440 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:08:57,681 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:09:57,921 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:10:58,160 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:11:58,399 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:12:58,635 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:13:58,877 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:14:59,112 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:15:59,344 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:16:59,560 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:17:59,773 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:18:59,984 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:20:00,192 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:21:00,397 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:22:00,597 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:23:00,797 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:24:00,994 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:25:01,192 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:26:01,380 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:27:01,564 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:28:01,752 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:29:01,938 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:30:02,132 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:31:02,378 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:32:02,566 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:33:02,750 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:34:02,939 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:35:03,124 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:36:03,324 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:37:03,524 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:38:03,714 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:39:03,894 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:40:04,078 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:41:04,254 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:42:04,431 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:43:04,640 Stage-1 map = 0%, reduce = 0%
2014-02-26 06:43:16,708 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 16.36 sec
2014-02-26 06:43:17,720 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 16.36 sec
MapReduce Total cumulative CPU time: 16 seconds 360 msec

检查日志后发现，DN1在2014-02-26 06:05:55,821 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName hadoop for UID 501 from the native implementation这个时间点就执行完成了，但是在DN2上，一直等到06:43才真正开始执行，jobtracker的日志也是从6.05直接就跳到了6.43，日志没有任何错误，这是问什么啊？资源不够？我的几个机器同时还在跑其他的任务。

...全文