使用spark2-submit方式提交spark作业正常运行，使用oozie启动时报错，求大佬帮忙

yef930 2020-04-02 04:34:11

直接上代码了



datas.foreachPartition(new ForeachPartitionFunction<Row>() {

			private static final long serialVersionUID = 1L;

			@SuppressWarnings("resource")

			@Override

			public void call(Iterator<Row> t) throws Exception {

				ApplicationContext context = new ClassPathXmlApplicationContext("conf/spring-config.xml");

				OracleDataService oracleDataService = context.getBean(OracleDataService.class);

				

				List<TgPoweroffTimePoint> records = new ArrayList<TgPoweroffTimePoint>();

				while(t.hasNext()) {

					Row row = t.next();

					TgPoweroffTimePoint poweroffRecord = new TgPoweroffTimePoint();

					poweroffRecord.setCollMeterId(row.getLong(0));

					poweroffRecord.setTimePoint(row.getString(1));

					poweroffRecord.setDataDate(row.getString(2));

					poweroffRecord.setCurveDensity(row.getString(3));

					records.add(poweroffRecord);

					

					if(records.size() == 1000) {

						List<TgPoweroffTimePoint> temp = new ArrayList<TgPoweroffTimePoint>();

						temp.addAll(records);

						oracleDataService.suppleAnalysis(temp);

						records.clear();

					}

				}

				if(!CollectionUtils.isEmpty(records)) {

					oracleDataService.suppleAnalysis(records);

				}

			}

		});

代码里使用spring+mybatis的方式操作数据库，使用spark2-submit的方式是可以正常运行的，但使用oozie启动任务创建sqlSessionFactory时报错了：



Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sqlSessionFactory' defined in class path resource [conf/spring-config.xml]: Invocation of init method failed; nested exception is org.springframework.core.NestedIOException: Failed to parse mapping resource: 'URL [jar:file:/hadoopdata/yarn/nm/usercache/xlsoft/appcache/application_1585627237317_0051/container_1585627237317_0051_01_000001/__app__.jar!/mappers/OracleDataMapper.xml]'; nested exception is org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. Cause: java.lang.IllegalArgumentException: Result Maps collection already contains value for com.xinlian.collStat.dao.OracleDataMapper.SysGatherStatMap

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)

	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)

	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)

	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)

	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)

	... 38 more

Caused by: org.springframework.core.NestedIOException: Failed to parse mapping resource: 'URL [jar:file:/hadoopdata/yarn/nm/usercache/xlsoft/appcache/application_1585627237317_0051/container_1585627237317_0051_01_000001/__app__.jar!/mappers/OracleDataMapper.xml]'; nested exception is org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. Cause: java.lang.IllegalArgumentException: Result Maps collection already contains value for com.xinlian.collStat.dao.OracleDataMapper.SysGatherStatMap

	at org.mybatis.spring.SqlSessionFactoryBean.buildSqlSessionFactory(SqlSessionFactoryBean.java:466)

	at org.mybatis.spring.SqlSessionFactoryBean.afterPropertiesSet(SqlSessionFactoryBean.java:340)

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)

	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)

	... 45 more

Caused by: org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. Cause: java.lang.IllegalArgumentException: Result Maps collection already contains value for com.xinlian.collStat.dao.OracleDataMapper.SysGatherStatMap

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.configurationElement(XMLMapperBuilder.java:120)

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.parse(XMLMapperBuilder.java:92)

	at org.mybatis.spring.SqlSessionFactoryBean.buildSqlSessionFactory(SqlSessionFactoryBean.java:464)

	... 48 more

Caused by: java.lang.IllegalArgumentException: Result Maps collection already contains value for com.xinlian.collStat.dao.OracleDataMapper.SysGatherStatMap

	at org.apache.ibatis.session.Configuration$StrictMap.put(Configuration.java:816)

	at org.apache.ibatis.session.Configuration$StrictMap.put(Configuration.java:788)

	at org.apache.ibatis.session.Configuration.addResultMap(Configuration.java:570)

	at org.apache.ibatis.builder.MapperBuilderAssistant.addResultMap(MapperBuilderAssistant.java:214)

	at org.apache.ibatis.builder.ResultMapResolver.resolve(ResultMapResolver.java:47)

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElement(XMLMapperBuilder.java:285)

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElement(XMLMapperBuilder.java:252)

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElements(XMLMapperBuilder.java:244)

	at org.apache.ibatis.builder.xml.XMLMapperBuilder.configurationElement(XMLMapperBuilder.java:116)

	... 50 more