Spark运行wordcount遇到如下问题，求解决

[solene1314]

17/08/03 13:20:23 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.ClassCastException: java.util.Arrays$ArrayList cannot be cast to java.util.Iterator
at Spark.SparkApp.core.WordCount$1.call(WordCount.java:28)
at Spark.SparkApp.core.WordCount$1.call(WordCount.java:1)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:125)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:125)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:191)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
这个的问题是什么，以下是我的代码

[solene1314]

package Spark.SparkApp.core; import java.util.Arrays; import java.util.Iterator; import java.util.List; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaPairRDD; import org.apache.spark.api.java.JavaRDD; import org.apache.spark.api.java.JavaSparkContext; import org.apache.spark.api.java.function.FlatMapFunction; import org.apache.spark.api.java.function.Function2; import org.apache.spark.api.java.function.PairFunction; import org.apache.spark.api.java.function.VoidFunction; import scala.Tuple2; public class WordCount { public static void main(String[] args) { System.setProperty("hadoop.home.dir", "G:/hadoop-2.8.0/hadoop-2.8.0"); SparkConf conf = new SparkConf().setAppName("Spark WordCount").setMaster("local"); JavaSparkContext sc = new JavaSparkContext(conf); JavaRDD<String> lines = sc.textFile("G:/CodeProject/SparkApp/pom.xml"); JavaRDD<String> words = lines.flatMap(new FlatMapFunction<String,String>() { public Iterator<String> call(String t) throws Exception { // TODO Auto-generated method stub return (Iterator<String>) Arrays.asList(t.split(" ")); } }); JavaPairRDD<String, Integer> pairs = words.mapToPair(new PairFunction<String, String,Integer>() { public Tuple2<String, Integer> call(String word) throws Exception { // TODO Auto-generated method stub return new Tuple2<String, Integer>(word, 1); } }); JavaPairRDD<String, Integer> wordsCount = pairs.reduceByKey(new Function2<Integer, Integer, Integer>() { public Integer call(Integer v1, Integer v2) throws Exception { // TODO Auto-generated method stub return v1+v2; } }); wordsCount.foreach(new VoidFunction<Tuple2<String,Integer>>() { public void call(Tuple2<String, Integer> pairs) throws Exception { // TODO Auto-generated method stub System.out.println(pairs._1+":"+pairs._2); } }); sc.stop(); } }

[廖某]

根据报错日志： java.lang.ClassCastException: java.util.Arrays$ArrayList cannot be cast to java.util.Iterator 可知是类型转化的问题，应该是你这条语句引起的： return (Iterator<String>) Arrays.asList(t.split(" "));

[LinkSe7en]

public Iterator<String> call(String t) throws Exception { // TODO Auto-generated method stub return (Iterator<String>) Arrays.asList(t.split(" ")); } 改为return Arrays.asList(t.split(" ")).iterator()

[卡奥斯道]

public Iterator<String> call(String t) throws Exception { // TODO Auto-generated method stub return (Iterator<String>) Arrays.asList(t.split(" ")); } 正常的都是如下： public Iterable<String> call(String s) throws Exception { return Arrays.asList(s.split(" ")); } 你这方法返回值类型都错了。将方法返回值类型改为Iterable。OK？