local模式成功,YARN模式下执行wordcount失败,求解

wzwdev1980 2018-06-30 02:32:22
部署HDFS和YARN集群,hdfs和YARN自动切换均成功,HDFS上传文件成功,通过本地分布式计算也成功,在yarn模式下执行失败

节点配置如下:


NodeManager报的警告:

2018-04-27 05:16:37,791 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: couldn't find app application_1530182275331_0001 while processing FINISH_CONTAINERS event



YARN执行时控制台显示的:

2018-06-28 21:44:01,950 INFO mapreduce.Job: Job job_1530191876883_0001 failed with state FAILED due to: Application application_1530191876883_0001 failed 2 times due to Error launching appattempt_1530191876883_0001_000002. Got exception: java.net.ConnectException: Call From webf.com/127.0.0.1 to webf.com:45668 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
...全文
699 3 打赏 收藏 转发到动态 举报
写回复
用AI写文章
3 条回复
切换为时间正序
请发表友善的回复…
发表回复
qq_31882211 2021-05-20
  • 打赏
  • 举报
回复
请问这个问题解决了吗,我刚好也是遇到这个问题,动态添加yarn nodemanager,任务调度不过来
wzwdev1980 2018-07-10
  • 打赏
  • 举报
回复
不行,还是错:


NodeManager报的警告:

2018-04-27 05:16:37,791 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: couldn't find app application_1530182275331_0001 while processing FINISH_CONTAINERS event



YARN执行时控制台显示的:

2018-06-28 21:44:01,950 INFO mapreduce.Job: Job job_1530191876883_0001 failed with state FAILED due to: Application application_1530191876883_0001 failed 2 times due to Error launching appattempt_1530191876883_0001_000002. Got exception: java.net.ConnectException: Call From webf.com/127.0.0.1 to webf.com:45668 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
kxiaozhuk 2018-07-06
  • 打赏
  • 举报
回复
1.检查一下yarn-site.xml是否加到工程conf下
2.贴一下你的执行命令,看是否指定--master yarn-client/cluster

917

社区成员

发帖
与我相关
我的任务
社区描述
华为云计算论坛,提供全面深入的云计算前景分析、丰富的技术干货、程序样例,分享华为云前沿资讯动态,方便开发者快速成长与发展,欢迎提问、互动,多方位了解云计算!
社区管理员
  • 华为云计算社区
  • 海洋 之心
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧