ubuntu-12.04 hadoop-2.6.0 执行mapreduce时错误，大侠帮忙看看！

sharewitheveryone 2015-01-20 12:11:20
两台机器分别编译，伪分布式下，一台可以正常运行，一台出现错误，下面将错误帖出来。
执行命令如下：bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar -input /input -output /output/tmp -mapper /usr/local/hadoop/mapper -reducer /usr/local/hadoop/reducer
错误如下：
packageJobJar: [/tmp/hadoop-unjar8869563477289590789/] [] /tmp/streamjob8490408200979643516.jar tmpDir=null
15/01/20 11:52:04 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/01/20 11:52:05 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/01/20 11:52:05 INFO mapred.FileInputFormat: Total input paths to process : 1
15/01/20 11:52:05 INFO mapreduce.JobSubmitter: number of splits:2
15/01/20 11:52:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1421725699816_0003
15/01/20 11:52:06 INFO impl.YarnClientImpl: Submitted application application_1421725699816_0003
15/01/20 11:52:06 INFO mapreduce.Job: The url to track the job: http://bruce-ubuntu-12.04:8088/proxy/application_1421725699816_0003/
15/01/20 11:52:06 INFO mapreduce.Job: Running job: job_1421725699816_0003
15/01/20 11:52:13 INFO mapreduce.Job: Job job_1421725699816_0003 running in uber mode : false
15/01/20 11:52:13 INFO mapreduce.Job: map 0% reduce 0%
15/01/20 11:52:14 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000000_0, Status : FAILED
Container launch failed for container_1421725699816_0003_01_000002 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

15/01/20 11:52:14 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000001_0, Status : FAILED
Container launch failed for container_1421725699816_0003_01_000003 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

15/01/20 11:52:16 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000000_1, Status : FAILED
Container launch failed for container_1421725699816_0003_01_000004 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244)
at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

...全文
730 4 打赏 收藏 转发到动态 举报
写回复
用AI写文章
4 条回复
切换为时间正序
请发表友善的回复…
发表回复
sharewitheveryone 2015-01-20
  • 打赏
  • 举报
 回复
15/01/20 11:52:17 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000001_1, Status : FAILED Container launch failed for container_1421725699816_0003_01_000005 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406 at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 15/01/20 11:52:19 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000000_2, Status : FAILED Container launch failed for container_1421725699816_0003_01_000006 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406 at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 15/01/20 11:52:20 INFO mapreduce.Job: Task Id : attempt_1421725699816_0003_m_000001_2, Status : FAILED Container launch failed for container_1421725699816_0003_01_000007 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406 at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:258) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:244) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:129) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 15/01/20 11:52:22 INFO mapreduce.Job: map 100% reduce 100% 15/01/20 11:52:22 INFO mapreduce.Job: Job job_1421725699816_0003 failed with state FAILED due to: Task failed task_1421725699816_0003_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 15/01/20 11:52:22 INFO mapreduce.Job: Counters: 4 Job Counters Other local map tasks=5 Data-local map tasks=2 Total time spent by all maps in occupied slots (ms)=0 Total time spent by all reduces in occupied slots (ms)=0 15/01/20 11:52:22 ERROR streaming.StreamJob: Job not successful! Streaming Command Failed!
人生偌只如初见 2015-01-20
  • 打赏
  • 举报
 回复
java.lang.IllegalArgumentException: Does not contain a valid host:port authority: bruce-ubuntu-12.04:45406 发现很多问题都来自于hosts文件配置
sharewitheveryone 2015-01-20
  • 打赏
  • 举报
 回复
原因查找出来，是主机名的问题，在/etc/hosts中127.0.0.1对应两个主机名，其中一个有.符号
人工智能：深度学习入门到精通实战
第十二章：经典网络 12.01-卷积神经网络 12.02-实验23-LeNet模型 12.03-深度卷积神经网络 12.04-实验24-AlexNet模型 12.05-使用重复元素的网络 12.06-实验25-VGG模型 12.07-网络中的网络 12.08-实验26-NiN模型 ...
Ubuntu12.04下hadoop-2.6.0单机配置和伪分布式配置
需要重新编译的教程httpblogcsdnnetggz631047367articledetails42460589在Ubuntu下创建hadoop用户组和用户在Ubuntu下安装JDK安装ssh服务 建立ssh无密码登录本机 安装hadoop 测试Hadoop伪分布式配置 需要重新编译的...
在ubuntu12.04上安装hadoop2.6.0
步骤： 一.安装JDK，及配置环境变量 二.配置hosts文件 三.配置SSH(这一步不是必须的，但可以免去每次开启或关闭集群时输入节点...chmod 600 ~/.ssh/authorized_keys ssh-add ~/.ssh/id_rsa ...四.下载并配置hadoop2.6.0
Hadoop安装教程_单机/伪分布式配置_Hadoop2.6.0/Ubuntu14.04
当开始着手实践 Hadoop 时，安装 Hadoop 往往会成为新手的一道门槛。尽管安装其实很简单，书上有写到，官方网站也有 Hadoop 安装配置教程，但由于对 Linux 环境不熟悉，书上跟官网上简略的安装步骤新手往往 Hold ...
【Hadoop】Hadoop安装教程（单机/伪分布式）与配置_Hadoop2.6.0/Ubuntu14.04
Hadoop安装教程（单机/伪分布式）与配置_Hadoop2.6.0/Ubuntu14.04 参考：厦门大学数据库实验室 环境 本教程使用 Ubuntu 14.04 64位 作为系统环境（Ubuntu 12.04，Ubuntu16.04 也行，32位、64位均可），请自行...
Hadoop生态社区

20,808

社区成员

4,690

社区内容

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区，致力于有Hadoop，hive，Spark，Hbase，Flink，ClickHouse，Kafka，数据仓库，大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今

加载中

查看更多榜单
社区公告
暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章