关于map传递静态变量的问题，着急，在线等，谢谢。

[jiashilong_syit]

小弟初学HADOOP, 学M/R遇到在mapper的setup()函数中传递参数的问题，模拟如下：

import java.io.IOException;
import java.util.StringTokenizer;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.Mapper.Context;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;

public class WordCount {

private static String s = null;

public static class TokenizerMapper
extends Mapper<Object, Text, Text, IntWritable>{

private final static IntWritable one = new IntWritable(1);
private Text word = new Text();

[color=#FF0000] protected void setup(Context context
) throws IOException, InterruptedException {
super.setup(context);
// 这里为什么还是为空呢，主函数中已经赋值了呀
if(s == null) {
throw new RuntimeException("fuck.");
}
}[/color]

protected void cleanup(Context context
) throws IOException, InterruptedException {
super.cleanup(context);
}

public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
StringTokenizer itr = new StringTokenizer(value.toString());
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
context.write(word, one);
}
}
}

public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWritable> {
private IntWritable result = new IntWritable();

public void reduce(Text key, Iterable<IntWritable> values,
Context context
) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
result.set(sum);
context.write(key, result);
}
}

public static void main(String[] args) throws Exception {
s = "123";
Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
if (otherArgs.length != 2) {
System.err.println("Usage: wordcount <in> <out>");
System.exit(2);
}
Job job = new Job(conf, "word count");
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}


s在M/R之前已经赋值为“123”了，为什么在setup()函数中还是为空，导致异常？

请求大神指点，谢谢，分数不多，见谅。

[撸大湿]

public static class TokenizerMapper 
       extends Mapper<Object, Text, Text, IntWritable>{
    
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();
    
    protected void setup(Context context
    ) throws IOException, InterruptedException {
    	String s = context.getConfiguration().get("s");
        // 通过context.getConfiguration().get()接受值
    	if(s == null) {
    	   throw new RuntimeException("Fuck.");
    	}else(s.equals("123")){
           throw new RuntimeException("Yeah.");
}

public static void main(String[] args) throws Exception {

    Configuration conf = new Configuration();
    conf .set("s", ""123"");
    //用CONF实例传值
    //........
    //........
}

[撸大湿]

引用 楼主 jiashilong_syit 的回复:

小弟初学HADOOP, 学M/R遇到在mapper的setup()函数中传递参数的问题，模拟如下：

引用 1 楼 tntzbzc 的回复:

//用CONF实例传值

利用好Context是关键

[holdup20080808]

因为执行的main函数是在作业发布的客户端的JVM进程里对s赋的值，而你要取的s值则是在另外一个JVM里，即TASK运行的MAP这JVM进程里，要从发布作业客户端将参数传递到各个运行任务的服务端节点正如“撸大湿”提供的方法即可。