使用zeppelin连接带kerberos的hive的时候出错

appleYQL 2019-07-24 04:45:37
各位大佬,公司打算使用zeppelin作为sql界面查询hive,我这边开发环境是使用cdh6.2.0,zeppelin使用的是0.8.1版本,jdk是jdk1.8.0_131.连接的hive使用了kerberos作为认证,我关于hive interpreter配置如下图

keytab文件我试过了能使用kinit登录认证,url我也用beeline尝试过,也正常。但是我使用hive写一个简单的查询sql的时候出现下面的错误:
INFO [2019-07-23 23:05:46,233] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:115) - Job 20190722-223013_769575118 finished by scheduler org.apache.zeppelin.jd
bc.JDBCInterpreter490505797
INFO [2019-07-23 23:07:14,230] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:109) - Job 20190722-223013_769575118 started by scheduler org.apache.zeppelin.jdbc
.JDBCInterpreter490505797
WARN [2019-07-23 23:07:14,237] ({pool-2-thread-2} JDBCInterpreter.java[appendProxyUserToURL]:494) - User impersonation for hive has changed please refer: http://zeppe
lin.apache.org/docs/latest/interpreter/jdbc.html#apache-hive
INFO [2019-07-23 23:07:14,248] ({pool-2-thread-2} JDBCSecurityImpl.java[createSecureConfiguration]:60) - The user has already logged in using Keytab and principal, no
action required
INFO [2019-07-23 23:07:14,250] ({pool-2-thread-2} Utils.java[parseURL]:324) - Supplied authorities: sdwsmn1:10000
INFO [2019-07-23 23:07:14,250] ({pool-2-thread-2} Utils.java[parseURL]:443) - Resolved authority: sdwsmn1:10000
ERROR [2019-07-23 23:07:14,252] ({pool-2-thread-2} TSaslTransport.java[open]:313) - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:229)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:184)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:410)
at org.apache.zeppelin.jdbc.JDBCInterpreter.access$000(JDBCInterpreter.java:91)
at org.apache.zeppelin.jdbc.JDBCInterpreter$2.run(JDBCInterpreter.java:459)
at org.apache.zeppelin.jdbc.JDBCInterpreter$2.run(JDBCInterpreter.java:456)
at java.security.AccessController.doPrivileged(Native Method)
-bash-4.2$ vim logs/zeppelin-interpreter-hive-hdfs-sdwsdn2.log
INFO [2019-07-24 03:50:53,424] ({pool-1-thread-1} RemoteInterpreterServer.java[shutdown]:208) - Shutting down...
INFO [2019-07-24 03:51:02,448] ({main} RemoteInterpreterServer.java[main]:260) - URL:jar:file:/usr/local/zeppelin-0.8.0-bin-all/lib/interpreter/zeppelin-interpreter-0
.8.0.jar!/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.class
INFO [2019-07-24 03:51:02,519] ({main} RemoteInterpreterServer.java[<init>]:161) - Launching ThriftServer at 25.10.6.4:44330
INFO [2019-07-24 03:51:02,527] ({main} RemoteInterpreterServer.java[<init>]:165) - Starting remote interpreter server on port 44330
INFO [2019-07-24 03:51:02,530] ({Thread-0} RemoteInterpreterServer.java[run]:202) - Starting remote interpreter server on port 44330
INFO [2019-07-24 03:51:03,538] ({Thread-1} RemoteInterpreterUtils.java[registerInterpreter]:165) - callbackHost: 25.10.6.4, callbackPort: 37339, callbackInfo: Callbac
kInfo(host:25.10.6.4, port:44330)
INFO [2019-07-24 03:51:03,685] ({pool-1-thread-1} RemoteInterpreterServer.java[createInterpreter]:310) - Instantiate interpreter org.apache.zeppelin.jdbc.JDBCInterpre
ter


出错后我也分析过,是没有提供可用的票据,但是我看日志和zeppelin的源码发现kerberos认证是正常的啊,认证的源码如下:
public static void createSecureConfiguration(Properties properties,
AuthenticationMethod authType) {
switch (authType) {
case KERBEROS:
Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", KERBEROS.toString());
UserGroupInformation.setConfiguration(conf);
try {
// Check TGT before calling login
// Ref: https://github.com/apache/hadoop/blob/release-3.0.1-RC1/hadoop-common-project/
// hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1232
if (!UserGroupInformation.isSecurityEnabled()
|| UserGroupInformation.getCurrentUser().getAuthenticationMethod() != KERBEROS
|| !UserGroupInformation.isLoginKeytabBased()) {
UserGroupInformation.loginUserFromKeytab(
properties.getProperty("zeppelin.jdbc.principal"),
properties.getProperty("zeppelin.jdbc.keytab.location"));
} else {
LOGGER.info("The user has already logged in using Keytab and principal, " +
"no action required");
}
} catch (IOException e) {
LOGGER.error("Failed to get either keytab location or principal name in the " +
"interpreter", e);
}
}
}

出现这个问题后我在https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html找到个方法,但是按照这样操作会出现
LoginException: Cannot read from System.in
这个错误。
因为公司的网络不太好,源码编译出问题了,没法直接调试,各位大佬有遇到过这个问题吗?
...全文
696 1 打赏 收藏 转发到动态 举报
写回复
用AI写文章
1 条回复
切换为时间正序
请发表友善的回复…
发表回复
haifeng112612 2022-03-30
  • 打赏
  • 举报
回复

我也遇到了认证过去的问题,(我用的是hdp集群)最后是KDC节点重新认证zeppelin,OK了

内容概要:本文聚焦于“空地多无人平台协同路径规划技术”的研究与复现,基于Matlab代码实现对无人机(UAV)与无人地面车辆(UGV)等异构多智能体系统的协同路径规划方法进行深入探讨。研究涵盖了多种智能优化算法,如蚁群算法、A*算法、RRT算法及蜣螂优化算法等,在三维复杂动态环境中实现多平台的安全、高效路径规划。重点解决了动态障碍规避、地形威胁应对与多任务约束下的协同作业问题,强调算法的工程可操作性与科研创新性。文档配套提供完整的Matlab仿真代码与网盘资源,支持算法验证、性能对比与二次开发,有助于推动相关技术在实际科研与工程项目中的落地应用。; 适合人群:具备一定编程基础和自动化、控制科学、计算机或机器人等相关专业背景的科研人员,特别适用于从事无人机系统、智能优化算法、路径规划及多智能体协同控制研究的硕士、博士研究生及高校研究人员。; 使用场景及目标:① 复现高水平论文中的空地协同路径规划算法;② 构建三维仿真环境开展多无人平台协同任务验证;③ 对比分析不同智能算法在路径规划中的收敛速度、避障能力与路径最优性;④ 支持科研项目申报、实验验证与学术成果产出; 阅读建议:建议读者结合提供的Matlab代码与网盘资料,按照模块逐步实践,重点关注算法实现细节、环境建模过程与参数调优策略,深入理解协同机制与优化目标的设计思路,从而提升科研创新能力与工程实现水平。

2,209

社区成员

发帖
与我相关
我的任务
社区描述
其他数据库开发 其他数据库
社区管理员
  • 其他数据库社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧