sqoop在hadoop上搭建问题,server版本无法显示,同时Connection refused

江南浙里 2015-02-03 03:52:50
[size=14px]首先说下我的hadoop是2.4.0,sqoop是1.99.3,sqoop2的搭建了许多遍,总是在soop --version时不成功,sqoop 客户端的版本可以显示,但是sqoop的server版本无法显示,也就是意味着sqoop不能连接到mysql数据库,如下图所示:[/size]

sqoop:000> show version
client version:
Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f
Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014
sqoop:000> show version --server
Exception has occurred during processing command
Exception: com.sun.jersey.api.client.ClientHandlerException Message: java.net.ConnectException: Connection refused

catalina.out日志如下:
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop/lib], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-hdfs], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-hdfs/lib], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-mapreduce], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-mapreduce/lib], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-yarn], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/lib/hadoop-yarn/lib], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/hadoop/share/hadoop/commom], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/hadoop/share/hadoop/commom/lib], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:20 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/hadoop/share/hadoop/yarn/*jar], exists: [false], isDirectory: [false], canRead: [false]
Feb 03, 2015 3:23:21 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/i386:/lib:/usr/lib
Feb 03, 2015 3:23:22 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-12000
Feb 03, 2015 3:23:22 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1206 ms
Feb 03, 2015 3:23:22 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Feb 03, 2015 3:23:22 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.37
Feb 03, 2015 3:23:22 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive sqoop.war
2015-02-03 15:23:24,257 INFO [main] core.SqoopServer (SqoopServer.java:initialize(47)) - Booting up Sqoop server
2015-02-03 15:23:24,299 INFO [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize(96)) - Starting config file poller thread
log4j: Parsing for [root] with value=[WARN, file].
log4j: Level token is [WARN].
log4j: Category root set to WARN
log4j: Parsing appender named "file".
log4j: Parsing layout options for "file".
log4j: Setting property [conversionPattern] to [%d{ISO8601} %-5p %c{2} [%l] %m%n].
log4j: End of parsing for "file".
log4j: Setting property [file] to [@LOGDIR@/sqoop.log].
log4j: Setting property [maxBackupIndex] to [5].
log4j: Setting property [maxFileSize] to [25MB].
log4j: setFile called: @LOGDIR@/sqoop.log, true
log4j: setFile ended
log4j: Parsed "file" options.
log4j: Parsing for [org.apache.sqoop] with value=[DEBUG].
log4j: Level token is [DEBUG].
log4j: Category org.apache.sqoop set to DEBUG
log4j: Handling log4j.additivity.org.apache.sqoop=[null]
log4j: Parsing for [org.apache.derby] with value=[INFO].
log4j: Level token is [INFO].
log4j: Category org.apache.derby set to INFO
log4j: Handling log4j.additivity.org.apache.derby=[null]
log4j: Finished configuring.
log4j: Could not find root logger information. Is this OK?
log4j: Parsing for [default] with value=[INFO,defaultAppender].
log4j: Level token is [INFO].
log4j: Category default set to INFO
log4j: Parsing appender named "defaultAppender".
log4j: Parsing layout options for "defaultAppender".
log4j: Setting property [conversionPattern] to [%d %-5p %c: %m%n].
log4j: End of parsing for "defaultAppender".
log4j: Setting property [file] to [@LOGDIR@/default.audit].
log4j: setFile called: @LOGDIR@/default.audit, true
log4j: setFile ended
log4j: Parsed "defaultAppender" options.
log4j: Handling log4j.additivity.default=[null]
log4j: Finished configuring.
Sqoop failed to load:
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:79)
at org.apache.sqoop.driver.JobManager.initialize(JobManager.java:223)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)
at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:38)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4705)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
... 28 more
...全文
2622 11 打赏 收藏 转发到动态 举报
AI 作业
写回复
用AI写文章
11 条回复
切换为时间正序
请发表友善的回复…
发表回复
一木一石 2016-10-18
  • 打赏
  • 举报
 回复
引用 10 楼 wendaowangqi 的回复:
遇到这样得问题，一般都是没有搞清楚 在sqoop1.99.*得版本中，是server、cient模式，这样你使用sqoop2-shell启动client时，如果没有指定server是谁，那肯定会报错。 我的解决办法： 类似于其他得框架，在home/目录下，创建 '.sqoop.rc' 文件，指定好server是谁就行了： ~/.sqoop2rc ： 
# Configure our Sqoop 2 server automatically
set server --host hadoop1

# Run in verbose mode by default
set option --name verbose --value true
或者可以在sqoop2-shell命令行界面 指定：sqoop:000>set server --host your.host.com --port 12000 --webapp sqoop
一木一石 2016-10-18
  • 打赏
  • 举报
 回复
遇到这样得问题，一般都是没有搞清楚 在sqoop1.99.*得版本中，是server、cient模式，这样你使用sqoop2-shell启动client时，如果没有指定server是谁，那肯定会报错。 我的解决办法： 类似于其他得框架，在home/目录下，创建 '.sqoop.rc' 文件，指定好server是谁就行了： ~/.sqoop2rc ： 
# Configure our Sqoop 2 server automatically
set server --host hadoop1

# Run in verbose mode by default
set option --name verbose --value true
qq_33905389 2016-04-14
  • 打赏
  • 举报
 回复
楼主是怎么解决的 啊。跪求解决方案
vfgvi6 2016-03-22
  • 打赏
  • 举报
 回复
楼主怎么配置的啊？我也是这个原因，但是路径已经配置了啊
小没良心的雨一直下 2015-12-17
  • 打赏
  • 举报
 回复
我也遇到楼主你这样的问题，纠结了好久了，以前一直没成功过，请问下你的common.loader咋配置
bingqiling0204 2015-04-23
  • 打赏
  • 举报
 回复
同样的问题，不知道楼主catalina.properties中的common.loader咋配置的呢？谢谢
江南浙里 2015-04-14
  • 打赏
  • 举报
 回复
引用 4 楼 u014522765 的回复:
楼主问题解决了吗
问题已经解决,在配置的时候出了一点差错,将catalina.properties中的common.loader配置错了,配好之后,如下所示: sqoop:000> show version --all client version: Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014 server version: Sqoop 1.99.4 source revision 2475a76ef70a0660f381c75c3d47d0d24f00b57f Compiled by gshapira on Sun Nov 16 02:50:00 PST 2014 API versions: [v1] sqoop:000>
不懂寂寞林 2015-04-09
  • 打赏
  • 举报
 回复
楼主问题解决了吗
江南浙里 2015-03-23
  • 打赏
  • 举报
 回复
前段时间忙别的事情,sqoop项目就放下了,今天又开始配置,还是无法连接服务器,不知什么原因,求大神指教!!!
江南浙里 2015-03-23
  • 打赏
  • 举报
 回复
还是没人回答啊...现在还没解答.
江南浙里 2015-02-03
  • 打赏
  • 举报
 回复
不知道有没有人遇到类似的问题,看网上大家配置sqoop2都很成功,但是我这里一直过不去,希望有人指教,谢谢!
记一次hadoop Connection refused: no further information排错
前言 music： 这两天在研究hadoop的hdfs，用本地java api远程调用hdfs时出现了连接被拒绝的问题，具体异常如下： Caused by: java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConne...
sqoop Communications link failure、Connection refused解决
这两天在准备sqoop，按照各种教程先后使用了sqoop2 sqoop、又重装了两次mysql，都一直报错，现已解决，记录一下。 The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnec...
Sqoop导入数据时异常java.net.ConnectException: Connection refused
异常信息 java.net.ConnectException: Call From node4/192.168.179.143 to node4:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.o...
sqoop从mysql导入数据到hdfs报错：Connection refused 无法连接10020端口
sqoop从mysql导入数据到hdfs，报错 java.net.ConnectException: Call From bigdata001/192.168.254.128 to bigdata001:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more det
【Sqoop一】Sqoop1.4.5和Hadoop2.5.2数据集成入门
Sqoop安装与配置 Sqoop的运行需要依赖于Hadoop，因此需要首先安装Hadoop2.5.2，并且Sqoop的启动前需要首先安装Hadoop。   本文使用如下版本 sqoop-1.4.5.bin_hadoop-2.0.4-alpha版本， Hadoop使用2.5.2版本   Sqoop安装和配置的步骤   1. 从如下地址下载Sqoop   http://...
Hadoop生态社区

20,848

社区成员

4,696

社区内容

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区，致力于有Hadoop，hive，Spark，Hbase，Flink，ClickHouse，Kafka，数据仓库，大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今

加载中

查看更多榜单
社区公告
暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章