flume的avro配置的问题

jiayou55 2014-04-23 01:53:16

我的疑问是端口8888是怎么来的，启动后为什么会报错，会avro方式配置的麻烦留个qq，谢谢
以下是我的其中一种配置
#############
agent1057.sources.searchStatusSrc.type=exec
agent1057.sources.searchStatusSrc.command = tail -F /app/thrift-solr4/logs/log.log |grep status:

agent1057.sinks.searchStatusSink.type = avro
agent1057.sinks.searchStatusSink.hostname = hadoop0
agent1057.sinks.searchStatusSink.port = 8888

agent1057.sources.searchStatusSrc.channels = searchStatusChanne
agent1057.sinks.searchStatusSink.channel = searchStatusChanne
agent1057.channels.searchStatusChanne.type = memory
agent1057.channels.searchStatusChanne.capacity = 1000
agent1057.channels.searchStatusChanne.transactionCapactiy = 1000

###########

然后在此台服务器上启动flume
后报错：

2014-04-23 13:41:23,054 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.sink.AbstractRpcSink.destroyConnection(AbstractRpcSink.java:246)] Rpc sink searchStatusSink closing Rpc client: NettyAvroRpcClient { host: hadoop0, port: 8888}
2014-04-23 13:41:23,056 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop0, port: 8888}: Failed to send batch
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:294)
at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:366)
... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop0, port: 8888}: Exception thrown from remote handler
at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:376)
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:353)
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:282)
... 4 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: NettyTransceiver closed
at org.apache.avro.ipc.CallFuture.get(CallFuture.java:128)
at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:368)
... 6 more
Caused by: java.io.IOException: NettyTransceiver closed
at org.apache.avro.ipc.NettyTransceiver.disconnect(NettyTransceiver.java:338)
at org.apache.avro.ipc.NettyTransceiver.access$200(NettyTransceiver.java:59)
at org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.handleUpstream(NettyTransceiver.java:496)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:348)
at org.jboss.netty.handler.codec.frame.FrameDecoder.channelClosed(FrameDecoder.java:236)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:93)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:476)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:623)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:101)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
... 1 more

...全文