hive on tez 报错 无法使用

下面是我的配置方法


select * from predict_als ,正常

select count(*) from predict_als; 报错

下面是诊断信息
User: hadoop
Name: HIVE-78752bfe-7a07-402a-b6e4-15044a862316
Application Type: TEZ
Application Tags:
YarnApplicationState: FAILED
FinalStatus Reported by AM: FAILED
Started: 21-Jun-2019 18:20:54
Elapsed: 44sec
Tracking URL: History
Diagnostics: Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
# hive详细日志
2019-06-21T18:20:54,800 INFO [main] client.TezClient: Session mode. Starting session.
2019-06-21T18:20:54,800 INFO [main] client.TezClientUtils: Using tez.lib.uris value from configuration: hdfs://HN3/apps/tez-0.8.5.tar.gz
2019-06-21T18:20:54,800 INFO [main] client.TezClientUtils: Using tez.lib.uris.classpath value from configuration: null
2019-06-21T18:20:54,823 INFO [main] client.TezClient: Tez system stage directory hdfs://HN3/tmp/hive/hadoop/_tez_session_dir/78752bfe-7a07-402a-b6e4-15044a862316/.tez/application_1561112432699_0001 doesn't exist and is created
2019-06-21T18:20:55,153 INFO [main] impl.YarnClientImpl: Submitted application application_1561112432699_0001
2019-06-21T18:20:55,155 INFO [main] client.TezClient: The url to track the Tez Session: http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/
2019-06-21T18:21:39,812 INFO [main] client.TezClient: App did not succeed. Diagnostics: Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2019-06-21T18:21:39,812 ERROR [main] exec.Task: Failed to execute tez graph.
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
相关推荐
hive on tez 常见报错问题收集
1)Failing because I am unlikely to write too. 2)Caused by: java.lang.OutOfMemoryError: Java heap space 3)Current usage: 1.0 GB of 1 GB physical memory used;2.7 GB of 2.1 GB virtual memory used. Killing container 4)java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: null 5)hive on tez 最终insert的表如果使用到union all 时会导致直接查询结果表数据为空的
hive on tez 配置好之后报错
注意: 我的kryo复制过去了. 报错内容 Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1624615458877_1974_1_00, diagnostics=[Vertex vertex_1624615458877_1974_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.a..
cdh继承tez引擎 cdh添加tez引擎 hive引擎增加
包括pom修改的每个截图和配置,该文档从有道云笔记到处略加修改,其中pom内容格式需要你手动调整一下. 富含 大量截图帮助定位和配置. 含hive on tez 常见报错的解决方法. tez比spark更省心. spark容易出现内存问题.
Hive_1.2.1_Tez_0.9.1安装包.rar
资源中包含了Hive1.2.1和Tez0.9.1安装包,安装配置说明可参考我的博客,如果安装过程中遇到问题可与我联系解决。
大数据Hive on MR/TEZ与hadoop的整合应用
本课程主要介绍hive的架构与应用,首先介绍了hive的实现结构,然后介绍了Hive Metastore三种模式,接着重点介绍hive的安装与配置,以及hive sql的基本操作,后面又介绍了hive beeline、hive on tez的使用,并测试了hive on MR和TEZ的性能优缺。 本课程以apache hadoop发行版本为例,介绍如何实现hadoop与hive的整合,hadoop环境采用手动安装的hadoop3.2.1环境,hive版本同样采用apache hive3.1.2发行版本。以3台主机为例作为hive安装环境,采用Centos7.7操作系统。 课程大纲如下: 1、hive的架构与应用场景 2、Hive Metastore三种运行模式 3、hive安装以及与hadoop整合 4、hive的安装与配置 5、启动metastore与hiveserver2服务 6、hive常用SQL操作 7、beeline的使用 8、tez的编译以及tez与hive整合步骤 9、测试tez功能以及与MR性能比较


