azkaban 执行kettle作业15分钟未执行完成就会报错

zhty0417 2019-11-29 05:09:15

问题描述：
设计：kettle作业是利用等待文件插件扫描文件，有文件就加载到数据库，没有文件则继续等等待，等待到设置的时间最大值2个小时没有文件则退出，调度工具用的是azkaban,azkaban调shell ,shell里调起kettle脚本。
目前发现azkaban调起之后，如果超过15分钟没有文件则会报错，或者有文件在15分钟内有数据文件，但在15分钟内未执行完，kettle显示执行成功，但azkaban返回却是错误的。
第一种报错如下：
2019/11/29 15:20:54.273 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2019/11/29 15:21:12.398 +0800 INFO [ExecutorServlet] [Azkaban] User null has called action log on 8
2019/11/29 15:21:42.441 +0800 INFO [ExecutorServlet] [Azkaban] User null has called action log on 8
java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:94)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:516)
at azkaban.execapp.JobRunner.run(JobRunner.java:436)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:98)
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:88)
... 7 more
2019/11/29 15:22:01.260 +0800 INFO [MOVE_FILES] [Azkaban] Job O_LA_CM_GROUP finished with status FAILED in 1186 seconds
2019/11/29 15:22:01.260 +0800 INFO [MOVE_FILES] [Azkaban] Setting MOVE_FILES to FAILED_FINISHING
2019/11/29 15:22:01.273 +0800 INFO [MOVE_FILES] [Azkaban] No attachment file for job O_LA_CM_GROUP written.
Uploading flowId MOVE_FILES
第二种报错如下：
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:22 - 文本文件输入.0 - linenr 48450000
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:23 - 文本文件输入.0 - linenr 48500000
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:23 - 表输出.0 - linenr 24250000
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:23 - 表输出.1 - linenr 24250000
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:23 - 文本文件输入.0 - linenr 48550000
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - 文本文件输入.0 - Finished processing (I=48571912, O=0, R=0, W=48571912, U=1, E=0)
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - oracle - Connection to database closed!
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - 表输出.0 - Finished processing (I=0, O=24285956, R=24285956, W=24285956, U=0, E=0)
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - oracle - Connection to database closed!
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - 表输出.1 - Finished processing (I=0, O=24285956, R=24285956, W=24285956, U=0, E=0)
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - Starting entry [成功]
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - exec(3, 0, 成功.0)
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - 成功 - Starting job entry
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - Finished job entry [成功] (result=[true])
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - Finished job entry [O_LA_PTY_INDIV_REL] (result=[true])
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - Finished job entry [等待文件] (result=[true])
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - O_LA_PTY_INDIV_REL - Job execution finished
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - Kitchen - Finished!
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - Kitchen - Start=2019/11/29 15:02:22.164, Stop=2019/11/29 15:41:30.877
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - 2019/11/29 15:41:30 - Kitchen - Processing ended after 39 minutes and 8 seconds (2348 seconds total).
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - Process completed unsuccessfully in 2356 seconds.
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL ERROR - Job run failed!
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL ERROR - azkaban.jobExecutor.utils.process.ProcessFailureExceptionazkaban.jobExecutor.utils.process.ProcessFailureException
29-11-2019 15:41:31 CST O_LA_PTY_INDIV_REL INFO - Finishing job O_LA_PTY_INDIV_REL at 1575013291347 with status FAILED
请问各位大侠有知道原因的吗

...全文