hive与hbase整合报错Failed with exception java.io.IOException:java.lang.NullPointerExc

ramontop1 ramontop1 墨衍会员 · AI 创作全网分发 墨衍智能分发 通过墨衍一键同步至各平台 可同步平台 CSDN微信公众号微博知乎掘金百家号博客园抖音小红书 你的文章也可以这样分发 墨衍会员 ¥399起/年,写一次发全网 我也要 + 加入社区
刚开始接触hadoop一段时间,以后要做数据挖掘的项目。整合hive、hbase就卡那里了。分开运行还顺利,hive建一个hbase能访问的表就比较O疼了。导入数据也没问题,就是查询不可以。

建表:

hive> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz");
OK
Time taken: 1.603 seconds


导数据:

hive> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=86;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1409219405692_0016, Tracking URL = http://cluster01:8888/proxy/application_1409219405692_0016/
Kill Command = /home/hadoop/hadoop-2.2.0/bin/hadoop job -kill job_1409219405692_0016
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2014-08-29 15:40:20,961 Stage-0 map = 0%, reduce = 0%
2014-08-29 15:40:29,406 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 3.89 sec
MapReduce Total cumulative CPU time: 3 seconds 890 msec
Ended Job = job_1409219405692_0016
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 3.89 sec HDFS Read: 6016 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 3 seconds 890 msec
OK
Time taken: 21.2 seconds


然后。。。Failed with exception java.io.IOException:java.lang.NullPointerException

hive> SELECT * FROM hbase_table_1;
OK
Failed with exception java.io.IOException:java.lang.NullPointerException
Time taken: 0.131 seconds


hive.log:

2014-08-29 15:25:42,121 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.run start=1409297141731 end=1409297142121 duration=390 from=org.apache.hadoop.hive.ql.Driver>
2014-08-29 15:25:42,204 ERROR [main]: CliDriver (SessionState.java:printError(545)) - Failed with exception java.io.IOException:java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:636)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:534)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:137)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1519)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:285)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.net.DNS.reverseDns(DNS.java:92)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:218)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:184)
at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits(HiveHBaseTableInputFormat.java:479)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:418)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:561)
... 14 more

2014-08-29 15:25:42,204 INFO [main]: exec.TableScanOperator (Operator.java:close(574)) - 0 finished. closing...


hive-site.xml:

<configuration>

<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://cluster01:9000/hive/warehouse</value>
</property>

<property>
<name>hive.exec.scratchdir</name>
<value>hdfs://cluster01:9000/hive/scratchdir</value>
</property>

<property>
<name>hive.querylog.location</name>
<value>file:///var/hadoop/hive/logs</value>
</property>

<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://cluster01:3306/hive?createDatabaseIfNotExist=true</value>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>

<property>
<name>hive.aux.jars.path</name>
<value>file:///home/hadoop/hive-0.13.1/lib/hive-hbase-handler-0.13.1.jar,file:///home/hadoop/hive-0.13.1/lib/protobuf-java-2.5.0.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-client-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-common-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-protocol-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-server-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/zookeeper-3.4.6.jar,file:///home/hadoop/hive-0.13.1/lib/guava-11.0.2.jar</value>
</property>

<property>
<name>hive.zookeeper.quorum</name>
<value>cluster01,cluster02,cluster03,cluster04,cluster05,cluster06,cluster07</value>
</property>

</configuration>


其他:
hadoop@cluster01:~$ java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

hadoop@cluster01:~$ uname -a
Linux cluster01 3.13.0-34-generic #60-Ubuntu SMP Wed Aug 13 15:45:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

hadoop@cluster01:~$ ls | grep -
hadoop-2.2.0
hbase-0.96.2
hive-0.13.1
mahout-0.9
zookeeper-3.4.6
...全文

相关推荐

hive查询报错:java.io.IOException:org.apache.parquet.io.ParquetDecodingException

这个异常是用spark sql将oracle(不知道mysql中有没有该问题,大家可以自己测试一下)中表数据查询出来然后写入hive表中,之后在hive命令行执行查询语句时产生的,下面先具体看一下如何产生这个异常的。...

董可伦

Hive中数据报错:Failed with exception java.io.IOException: rename for src path: hdfs://xx returned false.

1.现象 执行hive中shell脚本的时候报如下错误 Logging initialized using configuration in file:/etc/hive/1.1.0.0-0131/0/hive-log4j.properties Query ID = root_20200423155947_dfd65c44-2baa-44e2-853d-3fd8e58a7701 Total ...

莫叫石榴姐

集团供应链SCM计划管理现状及计划指标分析报.pptx

集团供应链SCM计划管理现状及计划指标分析报.pptx

公众号:优享智库

【永磁同步电机的通量链接模型】使用有限元分析得到的磁通链接图来建立PMSM模型(Simulink仿真实现)

内容概要:本文介绍了如何利用有限元分析(FEA)得到的磁通链接图来建立永磁同步电机(PMSM)的高精度数学模型,并在Simulink环境中实现系统级仿真。该方法通过精确计算电机内部磁场分布,提取不同工况下的非线性磁通特性,构建能够反映饱和效应、空间谐波等实际物理现象的动态模型,从而提升电机控制系统仿真的真实性与准确性。文中详述了从有限元建模、磁通数据提取与处理,到将静态数据映射为动态方程并集成至Simulink的全流程技术方案,为高性能电机控制算法的研发提供了可靠的仿真基础。; 适合人群:具备电机学基础知识和Simulink仿真经验的电气工程及相关专业的研究生、科研人员,以及从事电机控制算法开发的工程师; 使用场景及目标:①用于高精度永磁同步电机控制系统的设计与验证;②支撑先进控制策略(如模型预测控制、自适应控制)的研究与性能优化;③替代实物实验进行多工况快速仿真测试,降低研发成本并缩短开发周期; 阅读建议:建议读者结合ANSYS Maxwell、JMAG等有限元软件与Matlab/Simulink开展联合仿真实践,重点关注磁通数据的准确提取、插值处理与动态映射方法,确保模型精度;同时可进一步拓展至温度场、损耗等多物理场耦合的完整热模型构建。

优化算法侠_科研

事业单位考试资料20261121

事业单位考试资料20261121

sensmayday
评论 4
收藏 已收藏
分享
返回顶部
ramontop1
Hadoop生态社区 20,843 成员 4,695 内容