hadoop2.7.0 统计字符数量,报“The auxService:mapreduce_shuffle does not exist”异常

军大_j 2015-06-02 03:59:10
下面内容是我执行命令输出的内容:
-----------------------------------------------------------------------------
[root@ikaixin01 hadoop-2.7.0]# hadoop jar /cloud/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount /words /count
Java HotSpot(TM) Client VM warning: You have loaded library /cloud/hadoop-2.7.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/06/02 02:19:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/02 02:19:14 INFO client.RMProxy: Connecting to ResourceManager at ikaixin01/192.168.137.88:8032
15/06/02 02:19:16 INFO input.FileInputFormat: Total input paths to process : 1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: number of splits:1
15/06/02 02:19:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433170139587_0005
15/06/02 02:19:17 INFO impl.YarnClientImpl: Submitted application application_1433170139587_0005
15/06/02 02:19:17 INFO mapreduce.Job: The url to track the job: http://ikaixin01:8088/proxy/application_1433170139587_0005/
15/06/02 02:19:17 INFO mapreduce.Job: Running job: job_1433170139587_0005
15/06/02 02:19:28 INFO mapreduce.Job: Job job_1433170139587_0005 running in uber mode : false
15/06/02 02:19:28 INFO mapreduce.Job: map 0% reduce 0%
15/06/02 02:19:29 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_0, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000002 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

15/06/02 02:19:31 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_1, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000003 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

15/06/02 02:19:33 INFO mapreduce.Job: Task Id : attempt_1433170139587_0005_m_000000_2, Status : FAILED
Container launch failed for container_1433170139587_0005_01_000004 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

15/06/02 02:19:37 INFO mapreduce.Job: map 100% reduce 100%
15/06/02 02:19:38 INFO mapreduce.Job: Job job_1433170139587_0005 failed with state FAILED due to: Task failed task_1433170139587_0005_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

15/06/02 02:19:38 INFO mapreduce.Job: Counters: 4
Job Counters
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=0
Total time spent by all reduces in occupied slots (ms)=0
[root@ikaixin01 hadoop-2.7.0]#
------------------------------------------------------
网上查,都描述的是yarn-site.xml 缺少了aux-service的配置。
我的配置:
<property>
<name>yarn.nodemanager.aux-service</name>
<value>mapreduce_shuffle</value>
</property>

麻烦大拿们帮忙看看是什么问题导致。
...全文
4305 7 打赏 收藏 转发到动态 举报
写回复
用AI写文章
7 条回复
切换为时间正序
请发表友善的回复…
发表回复
qq_35382930 2016-06-21
  • 打赏
  • 举报
回复
请问楼主你在修改yarn配置之后再怎么做呢,直接又运行一次hadoop jar /cloud/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount /words /count 呢,还是需要再重新格式阿虎集群呢
MamyA 2015-07-14
  • 打赏
  • 举报
回复
请问大神你是如何配置好的,我用之前旧的版本的搭建方式换到hadoop2.7上,各种不成功呢。请问主要要配置哪些文件?求解。。。。
学战到底 2015-06-25
  • 打赏
  • 举报
回复
大神,我的配置没问题,也出现这个错误,请教。。。。
q1066186643 2015-06-04
  • 打赏
  • 举报
回复
谢谢分享..........
qq_28741651 2015-06-04
  • 打赏
  • 举报
回复
编码要认真啊。
军大_j 2015-06-02
  • 打赏
  • 举报
回复
<name>yarn.nodemanager.aux-service</name>这个写错了,应该是: <name>yarn.nodemanager.aux-services</name> 编码要认真啊。

932

社区成员

发帖
与我相关
我的任务
社区描述
云计算 云存储相关讨论
社区管理员
  • 云存储
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧