oozie 调用 sqoop,出现这个错误 Failing Oozie Launcher

hfmbook 2015-12-27 01:09:15
Sqoop command arguments :
import
--connect
jdbc:mysql://cdh5.hadoop.com:3306/test
--username
root
--password
123456
--table
epc_partgroup
--num-mappers
1
--fields-terminated-by
"\t"
--delete-target-dir
--hive-database
hive_gary
--hive-import
--hive-table
epc_partgroup
=================================================================

>>> Invoking Sqoop command line now >>>

4924 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
4971 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.5-cdh5.3.6
5000 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your password on the command-line is insecure. Consider using -P instead.
5025 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
5172 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing to use a MySQL streaming resultset.
5180 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code generation
5864 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `epc_partgroup` AS t LIMIT 1
5958 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `epc_partgroup` AS t LIMIT 1
5966 [main] INFO org.apache.sqoop.orm.CompilationManager - HADOOP_MAPRED_HOME is /opt/soft/cdh5/hadoop-2.5.0-cdh5.3.6
13088 [main] INFO org.apache.sqoop.orm.CompilationManager - Writing jar file: /tmp/sqoop-gary/compile/7eb5d287f10f01ef33f03384f9dcf6b5/epc_partgroup.jar
13167 [main] INFO org.apache.sqoop.tool.ImportTool - Destination directory epc_partgroup deleted.
13167 [main] WARN org.apache.sqoop.manager.MySQLManager - It looks like you are importing from mysql.
13168 [main] WARN org.apache.sqoop.manager.MySQLManager - This transfer can be faster! Use the --direct
13168 [main] WARN org.apache.sqoop.manager.MySQLManager - option to exercise a MySQL-specific fast path.
13168 [main] INFO org.apache.sqoop.manager.MySQLManager - Setting zero DATETIME behavior to convertToNull (mysql)
13172 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase - Beginning import of epc_partgroup
13225 [main] WARN org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
14223 [main] INFO org.apache.sqoop.mapreduce.db.DBInputFormat - Using read commited transaction isolation
Heart beat
48069 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase - Transferred 2.6914 KB in 34.8306 seconds (79.1259 bytes/sec)
48113 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase - Retrieved 102 records.
48220 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `epc_partgroup` AS t LIMIT 1
48323 [main] INFO org.apache.sqoop.hive.HiveImport - Loading uploaded data into Hive
Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://cdh5.hadoop.com:8020/user/gary/oozie-gary/0000007-151227120318126-oozie-gary-W/sqoop-node--sqoop/action-data.seq

Oozie Launcher ends

我看到 两个压yarn上运行的进程已经成功,最后导入hive的时候出现了这个错误
...全文
3168 10 打赏 收藏 转发到动态 举报
写回复
用AI写文章
10 条回复
切换为时间正序
请发表友善的回复…
发表回复
weixin_42392100 2018-10-18
  • 打赏
  • 举报
回复
使用oozie调要sqoop从关系型数据库到数据到hive,分两步:第一步是将数据导入到hdfs 的账户目录下/user/***/tablename;第二步是将该文件load到对应的表里面。你这个应该是第二步出了问题,原因在于你的Hue账户不是OS账户,没有权限移动文件。
hfmbook 2018-01-28
  • 打赏
  • 举报
回复
$SQOOP_CONF_DIR has not been set in the environment. 没有设置这个
随风心境 2018-01-26
  • 打赏
  • 举报
回复
引用 6 楼 Kumquatlemon 的回复:
同样的错误 84723 [uber-SubtaskRunner] INFO org.apache.sqoop.hive.HiveImport - Loading uploaded data into Hive Intercepting System.exit(1) <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] Oozie Launcher failed, finishing Hadoop job gracefully
请问解决了吗
随风心境 2018-01-26
  • 打赏
  • 举报
回复
我也遇到这个错误,请问解决了吗,能不能指点一下,谢谢非常感谢
Kumquatlemon 2017-11-21
  • 打赏
  • 举报
回复
同样的错误 84723 [uber-SubtaskRunner] INFO org.apache.sqoop.hive.HiveImport - Loading uploaded data into Hive Intercepting System.exit(1) <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] Oozie Launcher failed, finishing Hadoop job gracefully
hfmbook 2017-10-23
  • 打赏
  • 举报
回复
$SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
cd20121111yt 2017-09-17
  • 打赏
  • 举报
回复
你好 我也遇到同样问题,请问你是怎么解决的后来。
悠悠-我心 2015-12-28
  • 打赏
  • 举报
回复
不知道你的具体环境是什么,找不到具体原因
hfmbook 2015-12-28
  • 打赏
  • 举报
回复
引用 1 楼 liu_yulong 的回复:
不知道你的具体环境是什么,找不到具体原因
oozie 调用sqoop导入mysql数据到 hive [code] import --connect jdbc:mysql://cdh5.hadoop.com:3306/test --username root --password 123456 --table epc_partgroup --num-mappers 1 --fields-terminated-by "\t" --delete-target-dir --hive-database hive_gary --hive-import --hive-table epc_partgroup [code]

20,808

社区成员

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区,致力于有Hadoop,hive,Spark,Hbase,Flink,ClickHouse,Kafka,数据仓库,大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧