intellij idea 连接spark报错,spark是cloudera编译安装的

Kumquatlemon 2017-11-16 05:54:02
Error:scalac: Error: bad symbolic reference. A signature in SparkConf.class refers to term avro
in package org.apache which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling SparkConf.class.
scala.reflect.internal.Types$TypeError: bad symbolic reference. A signature in SparkConf.class refers to term avro
in package org.apache which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling SparkConf.class.

at scala.reflect.internal.pickling.UnPickler$Scan.toTypeError(UnPickler.scala:847)

at scala.reflect.internal.pickling.UnPickler$Scan$LazyTypeRef.complete(UnPickler.scala:854)

at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1231)

at scala.reflect.internal.Symbols$Symbol.tpe(Symbols.scala:1202)

at scala.tools.nsc.transform.SpecializeTypes$$anonfun$specializedTypeVars$5.apply(SpecializeTypes.scala:428)

at scala.tools.nsc.transform.SpecializeTypes$$anonfun$specializedTypeVars$5.apply(SpecializeTypes.scala:428)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at scala.collection.immutable.List.foreach(List.scala:318)

at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)

at scala.collection.AbstractTraversable.map(Traversable.scala:105)


...全文
578 2 打赏 收藏 转发到动态 举报
写回复
用AI写文章
2 条回复
切换为时间正序
请发表友善的回复…
发表回复
善若止水 2017-11-18
  • 打赏
  • 举报
回复
第一步: 先安装scala的jdk,能保证idea中能用local的方式测试scala代码 第二步: 引用spark的"spark-assembly-1.6.1-hadoop2.6.0.jar"包,这样才可以
曹宇飞丶 2017-11-16
  • 打赏
  • 举报
回复
package org.apache which is not available It may be completely missing from the current classpath, or the version on the classpath might be incompatible with the version used

20,808

社区成员

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区,致力于有Hadoop,hive,Spark,Hbase,Flink,ClickHouse,Kafka,数据仓库,大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧