运行hadoop mapreduce程序报错

kjajf8 2017-09-06 03:41:16
与代码无关,应该是电脑配置问题
java.lang.Exception: java.lang.NoSuchMethodError: org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getRssMemorySize()J

全部过程
2017-09-06 15:08:13,406 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2017-09-06 15:08:13,409 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1460)) - Starting flush of map output
2017-09-06 15:08:13,410 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1482)) - Spilling map output
2017-09-06 15:08:13,410 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1483)) - bufstart = 0; bufend = 108; bufvoid = 104857600
2017-09-06 15:08:13,410 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:flush(1485)) - kvstart = 26214396(104857584); kvend = 26214352(104857408); length = 45/6553600
2017-09-06 15:08:13,421 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:sortAndSpill(1667)) - Finished spill 0
2017-09-06 15:08:13,425 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:done(1038)) - Task:attempt_local1766184705_0001_m_000000_0 is done. And is in the process of committing
2017-09-06 15:08:13,431 INFO [Thread-19] mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2017-09-06 15:08:13,443 WARN [Thread-19] mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local1766184705_0001
java.lang.Exception: java.lang.NoSuchMethodError: org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getRssMemorySize()J
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getRssMemorySize()J
at org.apache.hadoop.mapred.Task.updateResourceCounters(Task.java:872)
at org.apache.hadoop.mapred.Task.updateCounters(Task.java:1021)
at org.apache.hadoop.mapred.Task.done(Task.java:1040)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:345)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-09-06 15:08:14,077 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1360)) - Job job_local1766184705_0001 running in uber mode : false
2017-09-06 15:08:14,078 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1367)) - map 0% reduce 0%
2017-09-06 15:08:14,080 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Job job_local1766184705_0001 failed with state FAILED due to: NA
2017-09-06 15:08:14,090 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1385)) - Counters: 10
Map-Reduce Framework
Map input records=12
Map output records=12
Map output bytes=108
Map output materialized bytes=0
Input split bytes=104
Combine input records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
File Input Format Counters
Bytes Read=132
Finished
...全文
1478 回复 打赏 收藏 转发到动态 举报
写回复
用AI写文章
回复
切换为时间正序
请发表友善的回复…
发表回复

2,245

社区成员

发帖
与我相关
我的任务
社区描述
OpenAPI
社区管理员
  • OpenAPI社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧