20,808
社区成员
发帖
与我相关
我的任务
分享
hadoop jar /home/hadoop/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /test/input /test/output
自带的wordcount源码如下:
Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
if (otherArgs.length < 2) {
System.err.println("Usage: wordcount <in> [<in>...] <out>");
System.exit(2);
}
Job job = Job.getInstance(conf, "word count");
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
在代码中并没有看到本地模式的设置啊,并没有找到下面的代码啊
conf.set("mapreduce.framework.name", "local");
conf.set("fs.defaultFS", "file:///");
conf.set("mapreduce.framework.name", "local");
conf.set("yarn.resourcemanager.hostname", "local");
你的意思是说自带的wordcount例子本来就只能以本地模式运行,我要想以集群方式运行的话,要自己修改wordcount的源码吗?Configuration conf = new Configuration();
conf.set("mapreduce.framework.name", "local");
conf.set("fs.defaultFS", "file:///");
conf.set("mapreduce.framework.name", "local");
conf.set("yarn.resourcemanager.hostname", "local");
/*conf.set("mapreduce.framework.name", "yarn");
conf.set("yarn.resourcemanager.hostname", "mini1");*/
Job job = Job.getInstance(conf);