linux下,java开发kafka问题!

前台美工 2016-08-14 10:38:51
建立maven工程,报错内容如下:
[etl@et01 cuiminghaoTest]$ hadoop jar mvn.jar
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
16/08/14 10:40:37 INFO utils.VerifiableProperties: Verifying properties
16/08/14 10:40:37 INFO utils.VerifiableProperties: Property key.serializer.class is overridden to kafka.serializer.StringEncode
16/08/14 10:40:37 INFO utils.VerifiableProperties: Property metadata.broker.list is overridden to 10.161.4.176:9055
16/08/14 10:40:37 INFO utils.VerifiableProperties: Property request.required.acks is overridden to -1
16/08/14 10:40:37 INFO utils.VerifiableProperties: Property serializer.class is overridden to kafka.serializer.StringEncoder
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
at kafka.utils.Pool.<init>(Pool.scala:28)
at kafka.producer.ProducerStatsRegistry$.<init>(ProducerStats.scala:34)
at kafka.producer.ProducerStatsRegistry$.<clinit>(ProducerStats.scala)
at kafka.producer.async.DefaultEventHandler.<init>(DefaultEventHandler.scala:48)
at kafka.producer.Producer.<init>(Producer.scala:59)
at kafka.javaapi.producer.Producer.<init>(Producer.scala:26)
at KProducer.<init>(KProducer.java:33)
at KProducer.main(KProducer.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: scala.collection.GenTraversableOnce$class
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 14 more


pom.xml配置如下:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>KPTestGroup</groupId>
<artifactId>ProducerTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>KProducerTest</name>
<description>测试</description>
<dependencies>
<dependency>
<groupId>KPTestGroup</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.10.0.0</version>
</dependency>
<dependency>
<groupId>KPTestGroup</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.8</version>
</dependency>
<dependency>
<groupId>KPTestGroup</groupId>
<artifactId>scala-parser-combinators_2.11</artifactId>
<version>1.0.4</version>
</dependency>

</dependencies>
</project>


java代码如下:



import java.util.Properties;
import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;

/**
* Hello world!
*
*/
public class KProducer
{
public final Producer<String, String> producer;
public final static String TOPIC = "test";

private KProducer(){
Properties props = new Properties();
//此处配置的是kafka的端口
props.put("metadata.broker.list", "10.161.4.176:9055");

//配置value的序列化类
props.put("serializer.class", "kafka.serializer.StringEncoder");
//配置key的序列化类
props.put("key.serializer.class", "kafka.serializer.StringEncoder");

//request.required.acks
//0, which means that the producer never waits for an acknowledgement from the broker (the same behavior as 0.7). This option provides the lowest latency but the weakest durability guarantees (some data will be lost when a server fails).
//1, which means that the producer gets an acknowledgement after the leader replica has received the data. This option provides better durability as the client waits until the server acknowledges the request as successful (only messages that were written to the now-dead leader but not yet replicated will be lost).
//-1, which means that the producer gets an acknowledgement after all in-sync replicas have received the data. This option provides the best durability, we guarantee that no messages will be lost as long as at least one in sync replica remains.
props.put("request.required.acks","-1");

producer = new Producer<String, String>(new ProducerConfig(props));
}

void produce() {
int messageNo = 1000;
final int COUNT = 10000;

while (messageNo < COUNT) {
String key = String.valueOf(messageNo);
String data = "hello kafka message " + key;
System.out.println(data);
producer.send(new KeyedMessage<String, String>(TOPIC, key ,data));
messageNo ++;
}
}

public static void main( String[] args )
{
System.out.println("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa");
new KProducer().produce();

}
}


33行代码如下:
producer = new Producer<String, String>(new ProducerConfig(props));

使用的jar包如下:
C:\Users\Administrator\.m2\repository\KPTestGroup\kafka_2.11\0.10.0.0\kafka_2.11-0.10.0.0.jar
C:\Users\Administrator\.m2\repository\KPTestGroup\scala-library\2.11.8\scala-library-2.11.8.jar
C:\Users\Administrator\.m2\repository\KPTestGroup\scala-parser-combinators_2.11\1.0.4\scala-parser-combinators_2.11-1.0.4.jar

kafka服务器正常使用命令可以发送接收消息,使用如下指令接收消息:
kafka-console-producer.sh --broker-list hnn01:9055 --sync --topic test
kafka-console-consumer.sh --zookeeper hnn01:2181 --topic test --from-beginning

使用java打包放到服务器上就报错,执行jar包的命令是hadoop jar mvn.jar ,从提示上可以看出已经进入main主方法了,kafka的包也使用了,但是在send()的时候就报错了,求大神指教:::::::::::::
...全文
823 2 打赏 收藏 转发到动态 举报
写回复
用AI写文章
2 条回复
切换为时间正序
请发表友善的回复…
发表回复
前台美工 2016-08-15
  • 打赏
  • 举报
回复
求助大神。不要沉,顶顶顶顶顶顶顶顶顶顶顶顶顶顶顶顶顶顶顶
yany8060 2016-08-14
  • 打赏
  • 举报
回复
我和你报一样的错误。。。 没办法不知道怎么办

18,771

社区成员

发帖
与我相关
我的任务
社区描述
Linux/Unix社区 专题技术讨论区
社区管理员
  • 专题技术讨论区社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧