storm log4j冲突的问题

hellohhhello 2016-05-07 11:56:59

这个问题困扰了我好久，网上查了好多，大都是那几种方法（修改pom.xml，添加依赖排除），试验过后，还是冲突，因此，来求助大神啦

程序kafka--storm---hbase/hdfs 均为集群模式

pom.xml文件如下：
<groupId>production</groupId>
<artifactId>Kafka2Storm2Hbase</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<name>Kafka2Storm2Hbase</name>
<url>http://maven.apache.org</url>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>0.9.2-incubating</version>


<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>

<version>0.9.3</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-hbase</artifactId>

<version>0.9.3</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.5.0-cdh5.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>0.98.6-cdh5.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-hdfs</artifactId>
<version>0.9.3</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.5.0-cdh5.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.9.2</artifactId>
<version>0.8.1.1</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>jmxri</artifactId>
<groupId>com.sun.jmx</groupId>
</exclusion>
<exclusion>
<artifactId>jms</artifactId>
<groupId>javax.jms</groupId>
</exclusion>
<exclusion>
<artifactId>jmxtools</artifactId>
<groupId>com.sun.jdmk</groupId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>

</exclusions>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>create-executable-jar</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>
jar-with-dependencies
</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>test.Kafka2Storm2Hbase.StormTopologyExample</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

发布：
./storm jar /tmp/Kafka2Storm2Hbase-0.0.1-SNAPSHOT-jar-with-dependencies.jar test.Kafka2Storm2Hbase.StormTopologyExample test1

异常如下：

Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.log4j.Logger.getLogger(Logger.java:39)
at org.apache.log4j.Logger.getLogger(Logger.java:43)
at test.Kafka2Storm2Hbase.writeToHbaseBolt.<clinit>(writeToHbaseBolt.java:39)
at test.Kafka2Storm2Hbase.StormTopologyExample.main(StormTopologyExample.java:74)
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
at org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:49)
... 4 more

后来我干脆尝试不写日志，将writeToHbaseBolt中的 private static final Logger LOG = Logger.getLogger(writeToHbaseBolt.class);去掉，虽然能发布了，但是依旧是日志的报错。

可在storm ui上看见：
java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory at org.apache.log4j.Logger.getLogger(Logger.java:39) at kafka.utils.Logging$class.logger(Logging.scala:24) at kafka.network.BlockingChannel.logger$lzycompute(BlockingChannel.scala:35) at kafka.network.BlockingChannel.logger(BlockingChannel.scala:35) at kafka.utils.Logging$class.debug(Logging.scala:51) at kafka.network.BlockingChannel.debug(BlockingChannel.scala:35) at kafka.network.BlockingChannel.connect(BlockingChannel.scala:64) at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44) at kafka.consumer.SimpleConsumer.getOrMakeConnection(SimpleConsumer.scala:142) at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:69) at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:124) at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79) at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77) at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67) at storm.kafka.PartitionManager.<init>(PartitionManager.java:83) at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:138) at backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617.invoke(executor.clj:563) at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)

或者是我有什么办法可以继续尝试吗？谢谢~~

...全文