oozie 调度 spark on yarn 阻塞问题

sinat_32638197 2017-12-07 10:10:33
在用 oozie 调度 spark on yarn 时候出现阻塞情况,tsts1任务进度条一直到如图红色那边就不动了,oozie 的 mr 任务也在90%左右不动,希望有大神提点一下

mr任务的log如下循环

Heart beat
2017-12-06 01:10:40,295 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:41,299 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:42,301 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:43,304 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:44,305 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:45,307 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:45,619 [communication thread] INFO org.apache.hadoop.mapred.TaskAttemptListenerImpl - Progress of TaskAttempt attempt_1512464778631_0020_m_000000_0 is : 1.0
2017-12-06 01:10:46,311 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:47,315 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:48,322 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:49,326 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:50,330 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:51,333 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:52,335 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:53,337 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:54,342 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:55,347 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:56,352 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:57,357 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:58,362 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:10:59,366 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:00,369 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:01,371 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:02,373 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:03,376 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:04,378 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:05,383 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:06,385 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:07,386 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:08,390 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
2017-12-06 01:11:09,391 [uber-SubtaskRunner] INFO org.apache.spark.deploy.yarn.Client - Application report for application_1512464778631_0021 (state: ACCEPTED)
Heart beat
...全文
281 回复 打赏 收藏 转发到动态 举报
写回复
用AI写文章
回复
切换为时间正序
请发表友善的回复…
发表回复

50,526

社区成员

发帖
与我相关
我的任务
社区描述
Java相关技术讨论
javaspring bootspring cloud 技术论坛(原bbs)
社区管理员
  • Java相关社区
  • 小虚竹
  • 谙忆
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧