hive与hbase整合报错Failed with exception java.io.IOException:java.lang.NullPointerExc
建表:
hive> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz");
OK
Time taken: 1.603 seconds
导数据:
hive> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=86;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1409219405692_0016, Tracking URL = http://cluster01:8888/proxy/application_1409219405692_0016/
Kill Command = /home/hadoop/hadoop-2.2.0/bin/hadoop job -kill job_1409219405692_0016
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2014-08-29 15:40:20,961 Stage-0 map = 0%, reduce = 0%
2014-08-29 15:40:29,406 Stage-0 map = 100%, reduce = 0%, Cumulative CPU 3.89 sec
MapReduce Total cumulative CPU time: 3 seconds 890 msec
Ended Job = job_1409219405692_0016
MapReduce Jobs Launched:
Job 0: Map: 1 Cumulative CPU: 3.89 sec HDFS Read: 6016 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 3 seconds 890 msec
OK
Time taken: 21.2 seconds
然后。。。Failed with exception java.io.IOException:java.lang.NullPointerException
hive> SELECT * FROM hbase_table_1;
OK
Failed with exception java.io.IOException:java.lang.NullPointerException
Time taken: 0.131 seconds
hive.log:
2014-08-29 15:25:42,121 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.run start=1409297141731 end=1409297142121 duration=390 from=org.apache.hadoop.hive.ql.Driver>
2014-08-29 15:25:42,204 ERROR [main]: CliDriver (SessionState.java:printError(545)) - Failed with exception java.io.IOException:java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:636)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:534)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:137)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1519)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:285)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.net.DNS.reverseDns(DNS.java:92)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:218)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:184)
at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits(HiveHBaseTableInputFormat.java:479)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:418)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:561)
... 14 more
2014-08-29 15:25:42,204 INFO [main]: exec.TableScanOperator (Operator.java:close(574)) - 0 finished. closing...
hive-site.xml:
<configuration>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://cluster01:9000/hive/warehouse</value>
</property>
<property>
<name>hive.exec.scratchdir</name>
<value>hdfs://cluster01:9000/hive/scratchdir</value>
</property>
<property>
<name>hive.querylog.location</name>
<value>file:///var/hadoop/hive/logs</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://cluster01:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<property>
<name>hive.aux.jars.path</name>
<value>file:///home/hadoop/hive-0.13.1/lib/hive-hbase-handler-0.13.1.jar,file:///home/hadoop/hive-0.13.1/lib/protobuf-java-2.5.0.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-client-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-common-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-protocol-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/hbase-server-0.96.2-hadoop2.jar,file:///home/hadoop/hive-0.13.1/lib/zookeeper-3.4.6.jar,file:///home/hadoop/hive-0.13.1/lib/guava-11.0.2.jar</value>
</property>
<property>
<name>hive.zookeeper.quorum</name>
<value>cluster01,cluster02,cluster03,cluster04,cluster05,cluster06,cluster07</value>
</property>
</configuration>
其他:
hadoop@cluster01:~$ java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
hadoop@cluster01:~$ uname -a
Linux cluster01 3.13.0-34-generic #60-Ubuntu SMP Wed Aug 13 15:45:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
hadoop@cluster01:~$ ls | grep -
hadoop-2.2.0
hbase-0.96.2
hive-0.13.1
mahout-0.9
zookeeper-3.4.6
相关推荐
hive查询报错:java.io.IOException:org.apache.parquet.io.ParquetDecodingException
这个异常是用spark sql将oracle(不知道mysql中有没有该问题,大家可以自己测试一下)中表数据查询出来然后写入hive表中,之后在hive命令行执行查询语句时产生的,下面先具体看一下如何产生这个异常的。...
Hive中数据报错:Failed with exception java.io.IOException: rename for src path: hdfs://xx returned false.
1.现象 执行hive中shell脚本的时候报如下错误 Logging initialized using configuration in file:/etc/hive/1.1.0.0-0131/0/hive-log4j.properties Query ID = root_20200423155947_dfd65c44-2baa-44e2-853d-3fd8e58a7701 Total ...
集团供应链SCM计划管理现状及计划指标分析报.pptx
集团供应链SCM计划管理现状及计划指标分析报.pptx
【永磁同步电机的通量链接模型】使用有限元分析得到的磁通链接图来建立PMSM模型(Simulink仿真实现)
内容概要:本文介绍了如何利用有限元分析(FEA)得到的磁通链接图来建立永磁同步电机(PMSM)的高精度数学模型,并在Simulink环境中实现系统级仿真。该方法通过精确计算电机内部磁场分布,提取不同工况下的非线性磁通特性,构建能够反映饱和效应、空间谐波等实际物理现象的动态模型,从而提升电机控制系统仿真的真实性与准确性。文中详述了从有限元建模、磁通数据提取与处理,到将静态数据映射为动态方程并集成至Simulink的全流程技术方案,为高性能电机控制算法的研发提供了可靠的仿真基础。; 适合人群:具备电机学基础知识和Simulink仿真经验的电气工程及相关专业的研究生、科研人员,以及从事电机控制算法开发的工程师; 使用场景及目标:①用于高精度永磁同步电机控制系统的设计与验证;②支撑先进控制策略(如模型预测控制、自适应控制)的研究与性能优化;③替代实物实验进行多工况快速仿真测试,降低研发成本并缩短开发周期; 阅读建议:建议读者结合ANSYS Maxwell、JMAG等有限元软件与Matlab/Simulink开展联合仿真实践,重点关注磁通数据的准确提取、插值处理与动态映射方法,确保模型精度;同时可进一步拓展至温度场、损耗等多物理场耦合的完整热模型构建。
事业单位考试资料20261121
事业单位考试资料20261121


