kettle二次开发连接HIVE,找不到HIVE类型
这是我的连接xml信息:
public static final String[] databaseXML = {
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<connection>" +
"<name>bjdt</name>" +
"<server>202.115.161.209</server>" +
"<type>MYSQL</type>" +
"<access>Native</access>" +
"<database>isearch5_beta_0.1</database>" +
"<port>3306</port>" +
"<username>root</username>" +
"<password>cs.swust</password>" +
"</connection>",
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<connection>" +
"<name>kettle</name>" +
"<server>10.11.0.65</server>" +
"<type>HIVE2</type>" +
"<access>Native</access>" +
"<database>default</database>" +
"<port>10000</port>" +
"<username/>" +
"<password>Encrypted</password>" +
"</connection>"
};
这是报错:
Unable to load database connection info from XML node
Unable to create new database interface
database type with plugin id [HIVE2] couldn't be found!
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:1007)
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:952)
at DataAnalysis.service.createKtrService.generateMyOwnTrans(createKtrService.java:91)
at DataAnalysis.service.createKtrService.doit(createKtrService.java:66)
at DataAnalysis.controller.createKtrController.createKtr(createKtrController.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
之后我再kettle-core.jar里面查看kettle-database-types.xml:<database-types>
<database-type id="AS/400">
<description>AS/400</description>
<classname>org.pentaho.di.core.database.AS400DatabaseMeta</classname>
</database-type>
<database-type id="CACHE">
<description>Intersystems Cache</description>
<classname>org.pentaho.di.core.database.CacheDatabaseMeta</classname>
</database-type>
<database-type id="DB2">
<description>IBM DB2</description>
<classname>org.pentaho.di.core.database.DB2DatabaseMeta</classname>
</database-type>
<database-type id="DBASE">
<description>dBase III, IV or 5</description>
<classname>org.pentaho.di.core.database.DbaseDatabaseMeta</classname>
</database-type>
........后面还有很多数据库的类型,
但是发现没有HIVE类型,那么问题来了,使用kettle的spoon却可以连接HIVE,这是为什么呢?我又应该怎么才可以用代码连接呢?
public static final String[] databaseXML = {
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<connection>" +
"<name>bjdt</name>" +
"<server>202.115.161.209</server>" +
"<type>MYSQL</type>" +
"<access>Native</access>" +
"<database>isearch5_beta_0.1</database>" +
"<port>3306</port>" +
"<username>root</username>" +
"<password>cs.swust</password>" +
"</connection>",
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
"<connection>" +
"<name>kettle</name>" +
"<server>10.11.0.65</server>" +
"<type>HIVE2</type>" +
"<access>Native</access>" +
"<database>default</database>" +
"<port>10000</port>" +
"<username/>" +
"<password>Encrypted</password>" +
"</connection>"
};
这是报错:
Unable to load database connection info from XML node
Unable to create new database interface
database type with plugin id [HIVE2] couldn't be found!
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:1007)
at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:952)
at DataAnalysis.service.createKtrService.generateMyOwnTrans(createKtrService.java:91)
at DataAnalysis.service.createKtrService.doit(createKtrService.java:66)
at DataAnalysis.controller.createKtrController.createKtr(createKtrController.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
之后我再kettle-core.jar里面查看kettle-database-types.xml:<database-types>
<database-type id="AS/400">
<description>AS/400</description>
<classname>org.pentaho.di.core.database.AS400DatabaseMeta</classname>
</database-type>
<database-type id="CACHE">
<description>Intersystems Cache</description>
<classname>org.pentaho.di.core.database.CacheDatabaseMeta</classname>
</database-type>
<database-type id="DB2">
<description>IBM DB2</description>
<classname>org.pentaho.di.core.database.DB2DatabaseMeta</classname>
</database-type>
<database-type id="DBASE">
<description>dBase III, IV or 5</description>
<classname>org.pentaho.di.core.database.DbaseDatabaseMeta</classname>
</database-type>
........后面还有很多数据库的类型,
但是发现没有HIVE类型,那么问题来了,使用kettle的spoon却可以连接HIVE,这是为什么呢?我又应该怎么才可以用代码连接呢?
...全文
相关推荐
kettle8.1Hive连接包
kettle8.1连接Hive、sqlserver、oracle数据库连接包。
kettle 从oracle数据库导数据到hive 表
由于公司要把oracle 数据库迁移到hive ,hbase 环境,特建议使用kettle做数据迁移,调试3个小时,终于调试成功,顺手写了个配置文档。
kettle8.2连接hive驱动包.zip
hive,jdbc,kettle,驱动包, kettle8.2连接hive驱动包: hive-jdbc-1.1.0-cdh5.14.2.jar
kettle批量插入hive2,解决表输出速度慢问题
解决kettle使用表输出到hive2时,批量提交不起作用导致速度非常慢的问题,重新编译big-data-plugin插件使表输出支持对hive2批量提交,速度由几十条每秒提升到三千条每秒。
使用kettle将mysql中的数据导入到hive中
先说说思路:1是直接从mysql输出至hive中,另外一个是先将数据搞到hdfs中,再load一下,当然这只是textfile存储格式而言,若是ORC等其他列式存储的话建议先搞一个textfile格式的中间表,再insert into table select 插进去 太学浅,还是就第一种在人理解上最好的一种先记录一下 1.连接hive对外服务的组件hiveserver2 点击测试,OK之后会显示测试成功 测试直接将mysql中的stdcode表导入一下 抽取数据,插入的目标库中有没有对应为表的话点击下方的SQL,会默认执行DDL语句,create一下表,接下来NEXT 成功!当然具体的


