ETL表对表数据抽取报错,数据库直接复制表没问题

1927
3

wwwwwx 初学数据Lv2

发表于2020-6-29 17:39

悬赏1

已解决

楼主
本帖最后由 wwwwwx 于 2020-6-29 18:14 编辑


2020-06-29 17:37:30 开始迁移到目标表:ods.ods_order_detail
2020-06-29 17:37:30 创建临时表:ods.T_YCKWXG,表结构为:uuid(C|32),oper(C|32),opeTime(C|20),delFlag(C|2),createOper(C|32),createOpeTime(C|20),orderMainUuid(C|32),skuNo(C|50),productName(C|200),productUuid(C|32),spec(C|4000),marketPrice(N|13),basePrice(N|13),promotionPrice(N|13),totalPrice(N|13),payPrice(N|13),freeMoney(N|13),totalFreePrice(N|13),refundMoney(N|13),integral(I|19),returnNum(I|19),buyNum(I|19),sunState(C|1),appraiseState(C|1),detailState(C|1),note(C|200),skuSplitRatio(N|13),version(I|19),payIntegral(I|19),isFromIntegral(C|2),area(C|10),serviceDate(C|20),serviceTime(C|20),productType(C|2),storeUuid(C|32),thirdOrderDetailId(C|50),thirdProductImage(C|200),warehouseNo(C|100),extendRate(N|13),extendSecRate(N|13),extendChannelUuid(C|32),jsWxYear(I|19),jsWxCircle(I|19),jsWxOverTime(C|25),oversold(C|2),orderDetailId(C|10),virtualCardMoney(N|13),workOrderId(C|32)
2020-06-29 17:37:31 数据查询耗时:1秒169毫秒结果集结构:
UUID(C|32),OPER(C|32),OPETIME(C|20),DELFLAG(C|2),CREATEOPER(C|32),CREATEOPETIME(C|20),ORDERMAINUUID(C|32),SKUNO(C|50),PRODUCTNAME(C|200),PRODUCTUUID(C|32),SPEC(C|4000),MARKETPRICE(N|13),BASEPRICE(N|13),PROMOTIONPRICE(N|13),TOTALPRICE(N|13),PAYPRICE(N|13),FREEMONEY(N|13),TOTALFREEPRICE(N|13),REFUNDMONEY(N|13),INTEGRAL(I|11),RETURNNUM(I|11),BUYNUM(I|11),SUNSTATE(C|1),APPRAISESTATE(C|1),DETAILSTATE(C|1),NOTE(C|200),SKUSPLITRATIO(N|13),VERSION(I|11),PAYINTEGRAL(I|11),ISFROMINTEGRAL(C|2),AREA(C|10),SERVICEDATE(C|20),SERVICETIME(C|20),PRODUCTTYPE(C|2),STOREUUID(C|32),THIRDORDERDETAILID(C|50),THIRDPRODUCTIMAGE(C|200),WAREHOUSENO(C|100),EXTENDRATE(N|13),EXTENDSECRATE(N|13),EXTENDCHANNELUUID(C|32),JSWXYEAR(I|11),JSWXCIRCLE(I|11),JSWXOVERTIME(C|25),OVERSOLD(C|2),ORDERDETAILID(C|10),VIRTUALCARDMONEY(N|13),WORKORDERID(C|32)
2020-06-29 17:37:34 com.esen.exception.RuntimeException4I18N: 执行数据迁移时出现异常
    at sun.reflect.GeneratedConstructorAccessor259.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.Util.getInstance(Util.java:408)
    at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1162)
    at com.mysql.jdbc.StatementImpl.executeBatchInternal(StatementImpl.java:1060)
    at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
    at com.esen.jdbc.pool.PooledStatement.executeBatch(PooledStatement.java:216)
    at com.esen.jdbc.pool.PooledPreparedStatement.executeBatch(PooledPreparedStatement.java:115)
    at com.esen.edf.data.migrate.jdbc.JdbcWriter4MySql.doBatchInsert(JdbcWriter4MySql.java:90)
    at com.esen.edf.data.migrate.common.buffer.DataReceiverRequestTask.call(DataReceiverRequestTask.java:69)
    at com.esen.edf.data.migrate.common.buffer.DataReceiverRequestTask.call(DataReceiverRequestTask.java:29)
    at com.esen.ethreadpool.req.RequestTaskFutureImpl.call(RequestTaskFutureImpl.java:100)
    at com.esen.scheduling.LocaleTask.call(LocaleTask.java:56)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.esen.exception.RuntimeException4I18N: 接收数据时出现异常
    ... 19 more
Caused by: com.esen.exception.RuntimeException4I18N: 批量插入数据出现异常
    ... 19 more
Caused by: com.esen.jdbc.pool.impl.mysql.MysqlPooledSQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'd8ba4c1abaf24bff87c0b696b897ee87','','22.80','15.20','15.20','60.80','60.80','0.' at line 1
    ... 19 more

2020-06-29 17:37:34 执行ETL过程[JOB_order_detail_SC]出现异常:com.esen.exception.RuntimeException4I18N: 执行数据迁移时出现异常
    at sun.reflect.GeneratedConstructorAccessor259.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.Util.getInstance(Util.java:408)
    at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1162)
    at com.mysql.jdbc.StatementImpl.executeBatchInternal(StatementImpl.java:1060)
    at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)
    at com.esen.jdbc.pool.PooledStatement.executeBatch(PooledStatement.java:216)
    at com.esen.jdbc.pool.PooledPreparedStatement.executeBatch(PooledPreparedStatement.java:115)
    at com.esen.edf.data.migrate.jdbc.JdbcWriter4MySql.doBatchInsert(JdbcWriter4MySql.java:90)
    at com.esen.edf.data.migrate.common.buffer.DataReceiverRequestTask.call(DataReceiverRequestTask.java:69)
    at com.esen.edf.data.migrate.common.buffer.DataReceiverRequestTask.call(DataReceiverRequestTask.java:29)
    at com.esen.ethreadpool.req.RequestTaskFutureImpl.call(RequestTaskFutureImpl.java:100)
    at com.esen.scheduling.LocaleTask.call(LocaleTask.java:56)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.esen.exception.RuntimeException4I18N: 接收数据时出现异常
    ... 19 more
Caused by: com.esen.exception.RuntimeException4I18N: 批量插入数据出现异常
    ... 19 more
Caused by: com.esen.jdbc.pool.impl.mysql.MysqlPooledSQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'd8ba4c1abaf24bff87c0b696b897ee87','','22.80','15.20','15.20','60.80','60.80','0.' at line 1
    ... 19 more

最近看过此主题的会员

pibangcai

esen_4PI055C3LR

最佳答案

jizk

发表于2020-6-29 17:39

只看该作者

取消 关注该作者的回复

目标库是mysql,其中有转义字符比如单引号或者反斜杠\  导致数据迁移失败
建议更换新的war包,或者由相关人员联系研发
3个回答

只看楼主

1505121235 数据老手Lv5

发表于2020-6-29 17:52

只看该作者

取消 关注该作者的回复

沙发

etl是怎么设计的呢

wwwwwx 初学数据Lv2

发表于2020-6-29 17:56

只看该作者

取消 关注该作者的回复

板凳

1505121235 发表于 2020-6-29 17:52
etl是怎么设计的呢

表输入-主题表输出

jizk 初学数据Lv2

发表于2020-6-29 17:39

只看该作者

取消 关注该作者的回复

地板

目标库是mysql,其中有转义字符比如单引号或者反斜杠\  导致数据迁移失败
建议更换新的war包,或者由相关人员联系研发

登录后可回答问题,请登录注册

快速回复 返回顶部 返回列表

小时

全天响应

分钟

快速处理问题

工程师强势助力

明星产品
解决方案
联系合作

渠道咨询电话:137-0120-6790

技术支持QQ:400-0011-866(工作日9:00-18:00)

产品建议邮箱yixin@esensoft.com

关注我们

扫TA学习更多干货

一对一专家交流

版权所有© 2006-2024 北京亿信华辰软件有限责任公司 京ICP备07017321号 京公网安备11010802016281号