使用zeppelin连接带kerberos的hive的时候出错

appleYQL 2019-07-24 04:45:37
各位大佬,公司打算使用zeppelin作为sql界面查询hive,我这边开发环境是使用cdh6.2.0,zeppelin使用的是0.8.1版本,jdk是jdk1.8.0_131.连接的hive使用了kerberos作为认证,我关于hive interpreter配置如下图

keytab文件我试过了能使用kinit登录认证,url我也用beeline尝试过,也正常。但是我使用hive写一个简单的查询sql的时候出现下面的错误:
INFO [2019-07-23 23:05:46,233] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:115) - Job 20190722-223013_769575118 finished by scheduler org.apache.zeppelin.jd
bc.JDBCInterpreter490505797
INFO [2019-07-23 23:07:14,230] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:109) - Job 20190722-223013_769575118 started by scheduler org.apache.zeppelin.jdbc
.JDBCInterpreter490505797
WARN [2019-07-23 23:07:14,237] ({pool-2-thread-2} JDBCInterpreter.java[appendProxyUserToURL]:494) - User impersonation for hive has changed please refer: http://zeppe
lin.apache.org/docs/latest/interpreter/jdbc.html#apache-hive
INFO [2019-07-23 23:07:14,248] ({pool-2-thread-2} JDBCSecurityImpl.java[createSecureConfiguration]:60) - The user has already logged in using Keytab and principal, no
action required
INFO [2019-07-23 23:07:14,250] ({pool-2-thread-2} Utils.java[parseURL]:324) - Supplied authorities: sdwsmn1:10000
INFO [2019-07-23 23:07:14,250] ({pool-2-thread-2} Utils.java[parseURL]:443) - Resolved authority: sdwsmn1:10000
ERROR [2019-07-23 23:07:14,252] ({pool-2-thread-2} TSaslTransport.java[open]:313) - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:229)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:184)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:410)
at org.apache.zeppelin.jdbc.JDBCInterpreter.access$000(JDBCInterpreter.java:91)
at org.apache.zeppelin.jdbc.JDBCInterpreter$2.run(JDBCInterpreter.java:459)
at org.apache.zeppelin.jdbc.JDBCInterpreter$2.run(JDBCInterpreter.java:456)
at java.security.AccessController.doPrivileged(Native Method)
-bash-4.2$ vim logs/zeppelin-interpreter-hive-hdfs-sdwsdn2.log
INFO [2019-07-24 03:50:53,424] ({pool-1-thread-1} RemoteInterpreterServer.java[shutdown]:208) - Shutting down...
INFO [2019-07-24 03:51:02,448] ({main} RemoteInterpreterServer.java[main]:260) - URL:jar:file:/usr/local/zeppelin-0.8.0-bin-all/lib/interpreter/zeppelin-interpreter-0
.8.0.jar!/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.class
INFO [2019-07-24 03:51:02,519] ({main} RemoteInterpreterServer.java[<init>]:161) - Launching ThriftServer at 25.10.6.4:44330
INFO [2019-07-24 03:51:02,527] ({main} RemoteInterpreterServer.java[<init>]:165) - Starting remote interpreter server on port 44330
INFO [2019-07-24 03:51:02,530] ({Thread-0} RemoteInterpreterServer.java[run]:202) - Starting remote interpreter server on port 44330
INFO [2019-07-24 03:51:03,538] ({Thread-1} RemoteInterpreterUtils.java[registerInterpreter]:165) - callbackHost: 25.10.6.4, callbackPort: 37339, callbackInfo: Callbac
kInfo(host:25.10.6.4, port:44330)
INFO [2019-07-24 03:51:03,685] ({pool-1-thread-1} RemoteInterpreterServer.java[createInterpreter]:310) - Instantiate interpreter org.apache.zeppelin.jdbc.JDBCInterpre
ter


出错后我也分析过,是没有提供可用的票据,但是我看日志和zeppelin的源码发现kerberos认证是正常的啊,认证的源码如下:
public static void createSecureConfiguration(Properties properties,
AuthenticationMethod authType) {
switch (authType) {
case KERBEROS:
Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", KERBEROS.toString());
UserGroupInformation.setConfiguration(conf);
try {
// Check TGT before calling login
// Ref: https://github.com/apache/hadoop/blob/release-3.0.1-RC1/hadoop-common-project/
// hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1232
if (!UserGroupInformation.isSecurityEnabled()
|| UserGroupInformation.getCurrentUser().getAuthenticationMethod() != KERBEROS
|| !UserGroupInformation.isLoginKeytabBased()) {
UserGroupInformation.loginUserFromKeytab(
properties.getProperty("zeppelin.jdbc.principal"),
properties.getProperty("zeppelin.jdbc.keytab.location"));
} else {
LOGGER.info("The user has already logged in using Keytab and principal, " +
"no action required");
}
} catch (IOException e) {
LOGGER.error("Failed to get either keytab location or principal name in the " +
"interpreter", e);
}
}
}

出现这个问题后我在https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html找到个方法,但是按照这样操作会出现
LoginException: Cannot read from System.in
这个错误。
因为公司的网络不太好,源码编译出问题了,没法直接调试,各位大佬有遇到过这个问题吗?
...全文
501 1 打赏 收藏 转发到动态 举报
写回复
用AI写文章
1 条回复
切换为时间正序
请发表友善的回复…
发表回复
haifeng112612 2022-03-30
  • 打赏
  • 举报
回复

我也遇到了认证过去的问题,(我用的是hdp集群)最后是KDC节点重新认证zeppelin,OK了

2,209

社区成员

发帖
与我相关
我的任务
社区描述
其他数据库开发 其他数据库
社区管理员
  • 其他数据库社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧