请教一个Oracle golden gate的使用问题(从Oracle同步到kafka)

BAStriver 2018-12-07 06:42:55

弄了两天的Oracle到Kafka的同步，找遍了所有csdn，哪位大神有经验可以处理一下吗？

目录如下：

错误：
Error occurred during initialization of VM
java/lang/NoClassDefFoundError: java/lang/Object

以下是报错信息：



[root@bigdata13 ogg] ./replicat parmafile dirprm/r2kafka.prm 

Usage: REPLICAT PARAMFILE <param file> [REPORTFILE <report file>]

                [INITIALDATALOAD] [PROCESSID <PID>]

                [USESUBDIRS] [NOUSESUBDIRS]

                [PAUSEATEND] [NOPAUSEATEND]

                [SKIPTRANSACTION [THREADS(<thread list>)]]

                [ATCSN <CSN> [THREADS(<thread list>)]]

                [AFTERCSN <CSN> [THREADS(<thread list>)]]

                [FILTERDUPTRANSACTIONS|NOFILTERDUPTRANSACTIONS]



Source Context :

  SourceModule            : [er.init]

  SourceID                : [/scratch/aime/adestore/views/aime_adc4150560/oggcore/OpenSys/src/app/er/init.cpp]

  SourceMethod            : [usage]

  SourceLine              : [7785]

  ThreadBacktrace         : [11] elements

                          : [/etc/ogg/libgglog.so(CMessageContext::AddThreadContext()+0x1e) [0x7f98f856f0ae]]

                          : [/etc/ogg/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x6ac) [0x7f98f855f9bc]]

                          : [/etc/ogg/libgglog.so(_MSG_String(CSourceContext*, int, char const*, CMessageFactory::MessageDisposition)+0x39) [0x7f98f854dd19]]

                          : [./replicat(init_functions(int, char**, ggs::gglib::ggapp::ReplicationContextParams&, ggs::gglib::ggdatasource::DataSourceParams&, ggs::gglib::ggdatatarget::DataTargetParams&)+0x316b) [0x5ca8db]]

                          : [./replicat() [0x6e3f92]]

                          : [./replicat(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x5e) [0x7e2d8e]]

                          : [./replicat(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x173) [0x7e7153]]

                          : [./replicat(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0x140) [0x7e79c0]]

                          : [./replicat(main+0x3b) [0x6e7e0b]]

                          : [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f98f0bffc05]]

                          : [./replicat() [0x550831]]



2018-12-07 18:31:16  ERROR   OGG-00012  Command line error:invalid startup syntax: parmafile.



2018-12-07 18:31:16  ERROR   OGG-01668  PROCESS ABENDING.

[root@bigdata13 ogg]# ./replicat paramfile dirprm/r2kafka.prm 



***********************************************************************

                    Oracle GoldenGate for Big Data

                    Version 12.3.2.1.1 (Build 005)

                                   

                      Oracle GoldenGate Delivery

  Version 12.3.0.1.2 OGGCORE_OGGADP.12.3.0.1.2_PLATFORMS_180712.2305

    Linux, x64, 64bit (optimized), Generic on Jul 13 2018 01:02:51

 

Copyright (C) 1995, 2018, Oracle and/or its affiliates. All rights reserved.



                    Starting at 2018-12-07 18:31:41

***********************************************************************



Operating System Version:

Linux

Version #1 SMP Tue Aug 22 21:09:27 UTC 2017, Release 3.10.0-693.el7.x86_64

Node: bigdata13

Machine: x86_64

                         soft limit   hard limit

Address Space Size   :    unlimited    unlimited

Heap Size            :    unlimited    unlimited

File Size            :    unlimited    unlimited

CPU Time             :    unlimited    unlimited



Process id: 123346



Description: 



***********************************************************************

**            Running with the following parameters                  **

***********************************************************************



2018-12-07 18:31:41  INFO    OGG-03059  Operating system character set identified as UTF-8.



2018-12-07 18:31:41  INFO    OGG-02695  ANSI SQL parameter syntax is used for parameter parsing.



2018-12-07 18:31:41  INFO    OGG-03528  The source database character set, as determined from the table definition file, is UTF-8.

REPLICAT r2kafka

sourcedefs /etc/ogg/dirdef/ggs.t_ogg

TARGETDB LIBFILE libggjava.so SET property=dirprm/kafka.props



2018-12-07 18:31:41  INFO    OGG-15052  Using Java class path: ggjava/ggjava.jar:ggjava/resources/lib/optional/log4j-api-2.9.1.jar:ggjava/resources/lib/optional/log4j-core-2.9.1.jar:ggjava/resources/lib/optional/log4j-slf4j-impl-2.9.1.jar.

Error occurred during initialization of VM

java/lang/NoClassDefFoundError: java/lang/Object

以下是jdk环境变量和ogg的设置：



export JAVA_HOME=/usr/java/jdk1.8.0_191

export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar

export JAVA=$JAVA_HOME/bin/java

export OGG_HOME=/etc/ogg

export PATH=$PATH:$OGG_HOME

export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/server:$JAVA_HOME/jre/lib/amd64/libjsig.so:$JAVA_HOME/jre/lib/amd64/server/libjvm.so

以下是OGG的管理进程MGR：



PORT 7809

DYNAMICPORTLIST 7810-7909

AUTORESTART EXTRACT *,RETRIES 5,WAITMINUTES 3

PURGEOLDEXTRACTS ./dirdat/*,usecheckpoints, minkeepdays 3

以下是r2kafka.prm：



REPLICAT r2kafka

sourcedefs /etc/ogg/dirdef/ggs.t_ogg

TARGETDB LIBFILE libggjava.so SET property=dirprm/kafka.props

REPORTCOUNT EVERY 1 MINUTES, RATE 

GROUPTRANSOPS 10000

MAP ggs.t_ogg, TARGET ggs.t_ogg;

然后这个是kafka.props：





gg.handlerlist = kafkahandler

gg.handler.kafkahandler.type=kafka

gg.handler.kafkahandler.KafkaProducerConfigFile=custom_kafka_producer.properties

#The following resolves the topic name using the short table name

gg.handler.kafkahandler.topicMappingTemplate=${tableName}

#The following selects the message key using the concatenated primary keys

gg.handler.kafkahandler.keyMappingTemplate=${primaryKeys}

gg.handler.kafkahandler.TopicNAme=oggtest

gg.handler.kafkahandler.format=json

gg.handler.kafkahandler.SchemaTopicName=mySchemaTopic

gg.handler.kafkahandler.BlockingSend =false

gg.handler.kafkahandler.includeTokens=false

gg.handler.kafkahandler.mode=op





goldengate.userexit.writers=javawriter

javawriter.stats.display=TRUE

javawriter.stats.full=TRUE



gg.log=log4j

gg.log.level=INFO



gg.report.time=30sec



#Sample gg.classpath for Apache Kafka

gg.classpath=dirprm/:/opt/tools/kafka_2.12-2.1.0/libs/*

#Sample gg.classpath for HDP

#gg.classpath=/etc/kafka/conf:/usr/hdp/current/kafka-broker/libs/*



javawriter.bootoptions=-Xmx512m -Xms32m -Djava.class.path=ggjava/ggjava.jar

这个是custom_kafka_producer.properties：



bootstrap.servers=10.53.127.33:9092

acks=1

reconnect.backoff.ms=1000

value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer

key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer

# 100KB per partition

batch.size=16384

linger.ms=0

...全文

337 1 打赏收藏转发到动态举报

写回复

用AI写文章

1 条回复

切换为时间正序

请发表友善的回复…

发表回复

BAStriver 2018-12-12

打赏
举报

大家记得删除Linux自带的openJDK，并且kafka.props那个文件不要引入所有的kafka安装包，ogg根目录建个文件夹放四个jar：kafka-clients-0.10.0-kafka-2.1.2.jar , lz4-1.3.0.jar , slf4j-api-1.7.21.jar , snappy-java-1.1.2.6.jar就好了，不然总是包冲突或者ClassNotFound。
详细可以参考我的博客里的Oracle到Kafka的ogg心得教程。