Hive报Error communicating with the metastore

cxf4029210 2018-02-27 05:37:26
Hadoop集群运行大约1到2周会出现Error communicating with the metastore的情况,重启metastore后恢复正常。
看日志似乎是因为心跳超时中止了事务,不知道为啥会心跳超时?求助
2018-02-27T00:16:24,877  INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.TxnHandler: 'HouseKeeper' locked by 'cplcdn3'
2018-02-27T00:16:24,905 INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.TxnHandler: Deleted 818 ext locks from HIVE_LOCKS due to timeout (vs. 4 found. List: [612320, 612324, 612330, 612344]) maxHeartbeatTime=1519661483775
2018-02-27T00:16:24,930 INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.TxnHandler: Aborted the following transactions due to timeout: [52959, 52960, 52967, 52968, 52969, 52970, 52971, 52972, 52973, 52974]
2018-02-27T00:16:24,930 INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.TxnHandler: Aborted 10 transactions due to timeout
2018-02-27T00:16:24,933 INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.AcidHouseKeeperService: timeout reaper ran for 0seconds. isAliveCounter=-2147482203
2018-02-27T00:16:24,949 INFO [org.apache.hadoop.hive.ql.txn.AcidHouseKeeperService-0] txn.TxnHandler: 'HouseKeeper' unlocked by 'cplcdn3'

2018-02-27T00:20:19,110 ERROR [pool-4-thread-130] metastore.RetryingHMSHandler: TxnAbortedException(message:Transaction txnid:52968 already aborted)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.ensureValidTxn(TxnHandler.java:2705)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.enqueueLockWithRetry(TxnHandler.java:855)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:789)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5972)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
at com.sun.proxy.$Proxy21.lock(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:13828)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:13812)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


...全文
2352 2 打赏 收藏 转发到动态 举报
写回复
用AI写文章
2 条回复
切换为时间正序
请发表友善的回复…
发表回复
五哥 2018-07-25
  • 打赏
  • 举报
回复
添加Hive元数据(使用mysql存储)

INSERT INTO NEXT_LOCK_ID VALUES(1);
INSERT INTO NEXT_COMPACTION_QUEUE_ID VALUES(1);
INSERT INTO NEXT_TXN_ID VALUES(1);
COMMIT;

说明:初始时这三个表没有数据,如果不添加数据,会报以下错误:
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager FAILED: Error in acquiring locks: Error communicating with the metastore

20,808

社区成员

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区,致力于有Hadoop,hive,Spark,Hbase,Flink,ClickHouse,Kafka,数据仓库,大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧