1,261
社区成员
发帖
与我相关
我的任务
分享
val windowWordCounts = sc.parallelize(List("aaa"->"aa","aaa"->"aa","aaa"->"cc","bbb"->"dd","bbb"->"dd","bbb"->"dd"))
val mapedRdd = windowWordCounts.map(rdd=>{
val key = rdd._1+"_"+rdd._2
(key,1L)
})
val reduceRDD = mapedRdd.reduceByKey(_+_)
val res = reduceRDD.collect
结果为
Array((bbb_dd,3), (aaa_aa,2), (aaa_cc,1))
val mapedRdd = windowWordCounts.map(rdd=>{
val key = rdd._1+"_"+rdd._2
(key,1L)
})
val reduceRDD = mapedRdd.reduceByKey(_+_)
reduceRDD 这个RDD中的数据 就是 (aaa_aa 2)(aaa_cc 1)(bbb_bb 3)