mahout测试canopy时出现 file output does not exist!

weimengfeiwu 2018-03-20 11:20:33
hdfs中明明创建了output 文件夹但是在运行
hadoop jar mahout-example-job.jar org.apache.mahout.clustering.syntheticcontrol.canopy.Job
命令时出现如下错误:(如下是我实验将output的path改为 /ouput 之后的结果还是报错)
18/03/20 10:45:07 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to rem edy this.
18/03/20 10:45:08 INFO input.FileInputFormat: Total input paths to process : 1
18/03/20 10:45:08 INFO mapreduce.JobSubmitter: number of splits:1
18/03/20 10:45:08 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1521419494657_ 0933
18/03/20 10:45:08 INFO impl.YarnClientImpl: Submitted application application_1521419494657_ 0933
18/03/20 10:45:08 INFO mapreduce.Job: The url to track the job: http://hdp00:8088/proxy/appl ication_1521419494657_0933/
18/03/20 10:45:08 INFO mapreduce.Job: Running job: job_1521419494657_0933
18/03/20 10:45:19 INFO mapreduce.Job: Job job_1521419494657_0933 running in uber mode : fals e
18/03/20 10:45:19 INFO mapreduce.Job: map 0% reduce 0%
18/03/20 10:45:30 INFO mapreduce.Job: Task Id : attempt_1521419494657_0933_m_000000_0, Statu s : FAILED
Error: java.io.FileNotFoundException: File /ouput does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:444)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:676)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.finalClustersPa th(ClusterClassificationMapper.java:158)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.setup(ClusterCl assificationMapper.java:75)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:18 65)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

18/03/20 10:45:59 INFO mapreduce.Job: Task Id : attempt_1521419494657_0933_m_000000_1, Statu s : FAILED
Error: java.io.FileNotFoundException: File /ouput does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:444)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:676)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.finalClustersPa th(ClusterClassificationMapper.java:158)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.setup(ClusterCl assificationMapper.java:75)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:18 65)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

18/03/20 10:46:18 INFO mapreduce.Job: Task Id : attempt_1521419494657_0933_m_000000_2, Statu s : FAILED
Error: java.io.FileNotFoundException: File /ouput does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:444)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:676)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1538)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1578)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.finalClustersPa th(ClusterClassificationMapper.java:158)
at org.apache.mahout.clustering.classify.ClusterClassificationMapper.setup(ClusterCl assificationMapper.java:75)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:18 65)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

18/03/20 10:46:22 INFO mapreduce.Job: map 100% reduce 0%
18/03/20 10:46:22 INFO mapreduce.Job: Job job_1521419494657_0933 failed with state FAILED du e to: Task failed task_1521419494657_0933_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

18/03/20 10:46:22 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=11032
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=11032
Total vcore-milliseconds taken by all map tasks=11032
Total megabyte-milliseconds taken by all map tasks=16945152
Exception in thread "main" java.lang.InterruptedException: Cluster Classification Driver Job failed processing output/data
at org.apache.mahout.clustering.classify.ClusterClassificationDriver.classifyCluster MR(ClusterClassificationDriver.java:298)
at org.apache.mahout.clustering.classify.ClusterClassificationDriver.run(ClusterClas sificationDriver.java:308)
at org.apache.mahout.clustering.canopy.CanopyDriver.clusterData(CanopyDriver.java:37 4)
at org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:157)
at org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:168)
at mathout.CanopyJob.run(CanopyJob.java:52)
at mathout.CanopyJob.main(CanopyJob.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java :43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
...全文
352 回复 打赏 收藏 转发到动态 举报
写回复
用AI写文章
回复
切换为时间正序
请发表友善的回复…
发表回复

20,808

社区成员

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区,致力于有Hadoop,hive,Spark,Hbase,Flink,ClickHouse,Kafka,数据仓库,大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧