hive on tez 报错 无法使用

qq_33246475 qq_33246475 墨衍会员 · AI 创作全网分发 墨衍智能分发 通过墨衍一键同步至各平台 可同步平台 CSDN微信公众号微博知乎掘金百家号博客园抖音小红书 你的文章也可以这样分发 墨衍会员 ¥399起/年,写一次发全网 我也要 + 加入社区
首先是软件版本信息


下面是我的配置方法




select * from predict_als ,正常


select count(*) from predict_als; 报错



下面是诊断信息
User: hadoop
Name: HIVE-78752bfe-7a07-402a-b6e4-15044a862316
Application Type: TEZ
Application Tags:
YarnApplicationState: FAILED
FinalStatus Reported by AM: FAILED
Started: 21-Jun-2019 18:20:54
Elapsed: 44sec
Tracking URL: History
Diagnostics: Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.

# hive详细日志
2019-06-21T18:20:54,800 INFO [main] client.TezClient: Session mode. Starting session.
2019-06-21T18:20:54,800 INFO [main] client.TezClientUtils: Using tez.lib.uris value from configuration: hdfs://HN3/apps/tez-0.8.5.tar.gz
2019-06-21T18:20:54,800 INFO [main] client.TezClientUtils: Using tez.lib.uris.classpath value from configuration: null
2019-06-21T18:20:54,823 INFO [main] client.TezClient: Tez system stage directory hdfs://HN3/tmp/hive/hadoop/_tez_session_dir/78752bfe-7a07-402a-b6e4-15044a862316/.tez/application_1561112432699_0001 doesn't exist and is created
2019-06-21T18:20:55,153 INFO [main] impl.YarnClientImpl: Submitted application application_1561112432699_0001
2019-06-21T18:20:55,155 INFO [main] client.TezClient: The url to track the Tez Session: http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/
2019-06-21T18:21:39,812 INFO [main] client.TezClient: App did not succeed. Diagnostics: Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2019-06-21T18:21:39,812 ERROR [main] exec.Task: Failed to execute tez graph.
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1561112432699_0001 failed 2 times due to AM Container for appattempt_1561112432699_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://IPTV-Recommend1:8088/proxy/application_1561112432699_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1561112432699_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:575)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

...全文

相关推荐

hive on tez 常见报错问题收集

1)Failing because I am unlikely to write too. 2)Caused by: java.lang.OutOfMemoryError: Java heap space 3)Current usage: 1.0 GB of 1 GB physical memory used;2.7 GB of 2.1 GB virtual memory used. Killing container 4)java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: null 5)hive on tez 最终insert的表如果使用到union all 时会导致直接查询结果表数据为空的

lin0207180

hive on tez 配置好之后报错

注意: 我的kryo复制过去了. 报错内容 Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1624615458877_1974_1_00, diagnostics=[Vertex vertex_1624615458877_1974_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.a..

Thomas2143

cdh继承tez引擎 cdh添加tez引擎 hive引擎增加

包括pom修改的每个截图和配置,该文档从有道云笔记到处略加修改,其中pom内容格式需要你手动调整一下. 富含 大量截图帮助定位和配置. 含hive on tez 常见报错的解决方法. tez比spark更省心. spark容易出现内存问题.

Thomas2143

Hive_1.2.1_Tez_0.9.1安装包.rar

资源中包含了Hive1.2.1和Tez0.9.1安装包,安装配置说明可参考我的博客,如果安装过程中遇到问题可与我联系解决。

剁椒鱼不要头

大数据Hive on MR/TEZ与hadoop的整合应用

本课程主要介绍hive的架构与应用,首先介绍了hive的实现结构,然后介绍了Hive Metastore三种模式,接着重点介绍hive的安装与配置,以及hive sql的基本操作,后面又介绍了hive beeline、hive on tez的使用,并测试了hive on MR和TEZ的性能优缺。 本课程以apache hadoop发行版本为例,介绍如何实现hadoop与hive的整合,hadoop环境采用手动安装的hadoop3.2.1环境,hive版本同样采用apache hive3.1.2发行版本。以3台主机为例作为hive安装环境,采用Centos7.7操作系统。 课程大纲如下:     1、hive的架构与应用场景 2、Hive Metastore三种运行模式 3、hive安装以及与hadoop整合 4、hive的安装与配置 5、启动metastore与hiveserver2服务   6、hive常用SQL操作 7、beeline的使用 8、tez的编译以及tez与hive整合步骤 9、测试tez功能以及与MR性能比较

weixin_38732519
评论 1
收藏 已收藏
分享
返回顶部
qq_33246475
Hadoop生态社区 20,842 成员 4,695 内容