kafka服务器Topic日志被占满

[qq_28630583]

网上找的开源的kafka，版本是kafka_2.10-0.10.0.1，在使用kafka自带的日志清理策略之后，日志没有被清理掉，查看server.log中好像报了这样的错，能不能问下各位大神，这种日志文件被kafka本身占用，怎么处理

[2017-02-08 19:33:58,586] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-02-08 19:34:27,086] INFO Scheduling log segment 9163 for log topic.alarmMatchFace-0 for deletion. (kafka.log.Log)
[2017-02-08 19:34:27,087] ERROR Uncaught exception in scheduled task 'kafka-log-retention' (kafka.utils.KafkaScheduler)
kafka.common.KafkaStorageException: Failed to change the log file suffix from to .deleted for log segment 9163
at kafka.log.LogSegment.kafkaStorageException$1(LogSegment.scala:263)
at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:265)
at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:832)
at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:823)
at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:579)
at kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:579)
at scala.collection.immutable.List.foreach(List.scala:318)
at kafka.log.Log.deleteOldSegments(Log.scala:579)
at kafka.log.LogManager.kafka$log$LogManager$$cleanupExpiredSegments(LogManager.scala:427)
at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:458)
at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:456)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
at kafka.log.LogManager.cleanupLogs(LogManager.scala:456)
at kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:192)
at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.nio.file.FileSystemException: \tmp\kafka-logs\topic.alarmMatchFace-0\00000000000000009163.log -> \tmp\kafka-logs\topic.alarmMatchFace-0\00000000000000009163.log.deleted: 另一个程序正在使用此文件，进程无法访问。

at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
at java.nio.file.Files.move(Unknown Source)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:670)
at kafka.log.FileMessageSet.renameTo(FileMessageSet.scala:370)
... 27 more
Suppressed: java.nio.file.FileSystemException: \tmp\kafka-logs\topic.alarmMatchFace-0\00000000000000009163.log -> \tmp\kafka-logs\topic.alarmMatchFace-0\00000000000000009163.log.deleted: 另一个程序正在使用此文件，进程无法访问。

at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
at java.nio.file.Files.move(Unknown Source)
at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:667)
... 28 more

[qq_28630583]

自己顶一下，有没有大神能解答这个问题

[qq_28630583]

有人有类似的经历的吗？怎么解决的？

[CapitalZ]

我也遇到了类似的问题，报错如下： [2017-02-22 09:06:03,164] ERROR Uncaught exception in scheduled task 'kafka-lo retention' (kafka.utils.KafkaScheduler) kafka.common.KafkaStorageException: Failed to change the log file suffix from o .deleted for log segment 0 正在努力解决。。。

[liuhsgz]

log cleaner 清理有条件的，最后一个活动的segment不能被清理，有个配置log.cleaner.min.compaction.lag.ms，默认对所有segment进行清理

[weixin_38706940]

同样问题，楼主解决了吗？

[gxlwl]

同问如何解决，好像说是window下的kafka有bug？

[liwei6797]

我也遇到，windows server 2012 系统