登录/注册
    我的位置：首页 › 一站式数据分析平台-亿信ABI › 亿信ABI-问题求助 › ETL设计区运行失败

    [ABI 5.2] ETL设计区运行失败

    复制链接 | 收藏本帖
    12211
    2

    jen 初学数据Lv2

    发表于2021-4-20 09:28

    悬赏1

    已解决

    楼主
    本帖最后由 jen 于 2021-4-20 09:32 编辑

    执行ETL过程报错。

    报错提示：
    2021-04-20 09:22:20 开始执行ETL过程：[ETL过程1618820033847]
    2021-04-20 09:22:21 开始执行组件：SQL输入1
    2021-04-20 09:22:21 结束执行组件：SQL输入1
    2021-04-20 09:22:21 开始执行组件：hwcd
    2021-04-20 09:22:23 组件对应的输入SQL：

    SELECT T0.COM AS COM,T0.PYH_FRAMENO AS FRAMENO,T0.CALLDATE AS CALLDATE,T0.ENDDATE AS PYH_ENDDATE
    FROM (
    SELECT T0.com AS COM,T0.pyh_frameno AS PYH_FRAMENO,T0.enddate AS ENDDATE,T0.calldate AS CALLDATE
    FROM (select (case when substr(pyh_comcode,0,4) = '1312' then '1302' else substr(pyh_comcode,0,4) end) com,pyh_frameno,min(pyh_calldate) calldate,,substr(pyh_enddate,0,10) enddate
    from picc_yx_hwqd where pyh_calldate >='2021-03-01' and pyh_calldate <'2021-04-01' group by 1,2,3) T0) T0
    2021-04-20 09:22:23 开始迁移到目标表：he1300public.hwcd
    2021-04-20 09:22:25 com.esen.exception.RuntimeException4I18N: 执行数据迁移时出现异常
        at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:354)
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:245)
        at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:440)
        at com.esen.jdbc.pool.PooledStatement.getQureyResultSet(PooledStatement.java:54)
        at com.esen.jdbc.pool.impl.hive.AbstractHivePooledStatement.executeQuery(AbstractHivePooledStatement.java:41)
        at com.esen.jdbc.dialect.impl.hive.AbstractHiveDialect.getResultMetaData(AbstractHiveDialect.java:922)
        at com.esen.jdbc.dialect.impl.DialectImpl.getQueryResultMetaData(DialectImpl.java:406)
        at com.esen.ecore.util.JdbcUtils.getMetaData(JdbcUtils.java:216)
        at com.esen.edf.data.migrate.DataMigration.getAllTableFields(DataMigration.java:836)
        at com.esen.edf.data.migrate.DataMigration.cleanTableAndInsertInto(DataMigration.java:374)
        at com.esen.edf.data.migrate.DataMigration.overwriteTargetTable(DataMigration.java:348)
        at com.esen.edf.data.migrate.DataMigration.migrateIntoPhysicalTable(DataMigration.java:281)
        at com.esen.edf.data.migrate.DataMigration.execute(DataMigration.java:238)
        at com.esen.edf.domain.job.widget.AbstractEtlWidgetTarget.executeDataMigration(AbstractEtlWidgetTarget.java:628)
        at com.esen.edf.domain.job.widget.AbstractEtlWidgetTarget.process(AbstractEtlWidgetTarget.java:407)
        at com.esen.edf.domain.job.EtlJob.processEtlJob(EtlJob.java:549)
        at com.esen.edf.domain.job.EtlJob.process(EtlJob.java:445)
        at com.esen.edf.domain.job.EtlJobRequestTask.call(EtlJobRequestTask.java:69)
        at com.esen.edf.domain.job.EtlJobRequestTask.call(EtlJobRequestTask.java:26)
        at com.esen.ethreadpool.req.RequestTaskFutureImpl.call(RequestTaskFutureImpl.java:100)
        at com.esen.scheduling.LocaleTask.call(LocaleTask.java:56)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: com.esen.exception.RuntimeException4I18N: 获取表[
    SELECT T0.COM AS COM,T0.PYH_FRAMENO AS FRAMENO,T0.CALLDATE AS CALLDATE,T0.ENDDATE AS PYH_ENDDATE
    FROM (
    SELECT T0.com AS COM,T0.pyh_frameno AS PYH_FRAMENO,T0.enddate AS ENDDATE,T0.calldate AS CALLDATE
    FROM (select (case when substr(pyh_comcode,0,4) = '1312' then '1302' else substr(pyh_comcode,0,4) end) com,pyh_frameno,min(pyh_calldate) calldate,,substr(pyh_enddate,0,10) enddate
    from picc_yx_hwqd where pyh_calldate >='2021-03-01' and pyh_calldate <'2021-04-01' group by 1,2,3) T0) T0]元数据信息出现异常
        ... 25 more
    Caused by: com.esen.jdbc.pool.impl.sparksql.SparkSQLPooledSQLException: org.apache.spark.sql.catalyst.parser.ParseException:
    mismatched input 'FROM' expecting (line 3, pos 1)

    == SQL ==

    SELECT T0.COM AS COM,T0.PYH_FRAMENO AS FRAMENO,T0.CALLDATE AS CALLDATE,T0.ENDDATE AS PYH_ENDDATE
    FROM (
    -^^^
    SELECT T0.com AS COM,T0.pyh_frameno AS PYH_FRAMENO,T0.enddate AS ENDDATE,T0.calldate AS CALLDATE
    FROM (select (case when substr(pyh_comcode,0,4) = '1312' then '1302' else substr(pyh_comcode,0,4) end) com,pyh_frameno,min(pyh_calldate) calldate,,substr(pyh_enddate,0,10) enddate
    from picc_yx_hwqd where pyh_calldate >='2021-03-01' and pyh_calldate <'2021-04-01' group by 1,2,3) T0) T0

        ... 25 more

    2021-04-20 09:22:25 执行ETL过程[ETL过程1618820033847]出现异常：com.esen.exception.RuntimeException4I18N: 执行数据迁移时出现异常
        at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:354)
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:245)
        at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:440)
        at com.esen.jdbc.pool.PooledStatement.getQureyResultSet(PooledStatement.java:54)
        at com.esen.jdbc.pool.impl.hive.AbstractHivePooledStatement.executeQuery(AbstractHivePooledStatement.java:41)
        at com.esen.jdbc.dialect.impl.hive.AbstractHiveDialect.getResultMetaData(AbstractHiveDialect.java:922)
        at com.esen.jdbc.dialect.impl.DialectImpl.getQueryResultMetaData(DialectImpl.java:406)
        at com.esen.ecore.util.JdbcUtils.getMetaData(JdbcUtils.java:216)
        at com.esen.edf.data.migrate.DataMigration.getAllTableFields(DataMigration.java:836)
        at com.esen.edf.data.migrate.DataMigration.cleanTableAndInsertInto(DataMigration.java:374)
        at com.esen.edf.data.migrate.DataMigration.overwriteTargetTable(DataMigration.java:348)
        at com.esen.edf.data.migrate.DataMigration.migrateIntoPhysicalTable(DataMigration.java:281)
        at com.esen.edf.data.migrate.DataMigration.execute(DataMigration.java:238)
        at com.esen.edf.domain.job.widget.AbstractEtlWidgetTarget.executeDataMigration(AbstractEtlWidgetTarget.java:628)
        at com.esen.edf.domain.job.widget.AbstractEtlWidgetTarget.process(AbstractEtlWidgetTarget.java:407)
        at com.esen.edf.domain.job.EtlJob.processEtlJob(EtlJob.java:549)
        at com.esen.edf.domain.job.EtlJob.process(EtlJob.java:445)
        at com.esen.edf.domain.job.EtlJobRequestTask.call(EtlJobRequestTask.java:69)
        at com.esen.edf.domain.job.EtlJobRequestTask.call(EtlJobRequestTask.java:26)
        at com.esen.ethreadpool.req.RequestTaskFutureImpl.call(RequestTaskFutureImpl.java:100)
        at com.esen.scheduling.LocaleTask.call(LocaleTask.java:56)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: com.esen.exception.RuntimeException4I18N: 获取表[
    SELECT T0.COM AS COM,T0.PYH_FRAMENO AS FRAMENO,T0.CALLDATE AS CALLDATE,T0.ENDDATE AS PYH_ENDDATE
    FROM (
    SELECT T0.com AS COM,T0.pyh_frameno AS PYH_FRAMENO,T0.enddate AS ENDDATE,T0.calldate AS CALLDATE
    FROM (select (case when substr(pyh_comcode,0,4) = '1312' then '1302' else substr(pyh_comcode,0,4) end) com,pyh_frameno,min(pyh_calldate) calldate,,substr(pyh_enddate,0,10) enddate
    from picc_yx_hwqd where pyh_calldate >='2021-03-01' and pyh_calldate <'2021-04-01' group by 1,2,3) T0) T0]元数据信息出现异常
        ... 25 more
    Caused by: com.esen.jdbc.pool.impl.sparksql.SparkSQLPooledSQLException: org.apache.spark.sql.catalyst.parser.ParseException:
    mismatched input 'FROM' expecting (line 3, pos 1)

    == SQL ==

    SELECT T0.COM AS COM,T0.PYH_FRAMENO AS FRAMENO,T0.CALLDATE AS CALLDATE,T0.ENDDATE AS PYH_ENDDATE
    FROM (
    -^^^
    SELECT T0.com AS COM,T0.pyh_frameno AS PYH_FRAMENO,T0.enddate AS ENDDATE,T0.calldate AS CALLDATE
    FROM (select (case when substr(pyh_comcode,0,4) = '1312' then '1302' else substr(pyh_comcode,0,4) end) com,pyh_frameno,min(pyh_calldate) calldate,,substr(pyh_enddate,0,10) enddate
    from picc_yx_hwqd where pyh_calldate >='2021-03-01' and pyh_calldate <'2021-04-01' group by 1,2,3) T0) T0

        ... 25 more

    最近看过此主题的会员

    esen_5F7WOBYEX9

    zhangchaobin

    数据狗

    刘伟良

    Marcoco

    mkhang

    pandabuxizao

    jiangshq

    最佳答案

    豌豆君

    发表于2021-4-20 09:28

    丨

    只看该作者

    丨

    取消 关注该作者的回复

    jen 发表于 2021-4-20 09:35
    不好意思。第一次发帖，图放错了。是下面这张图。

    可以先查一下系统表结构字段类型、长度是否一直。字段类型长度等不一致可能导致迁移失败
    QQ空间 微信
    2个回答

    只看楼主

    jen 初学数据Lv2

    发表于2021-4-20 09:35

    丨

    只看该作者

    丨

    取消 关注该作者的回复

    沙发

    本帖最后由 jen 于 2021-4-20 09:38 编辑

    不好意思。第一次发帖，图放错了。是下面这张图。

    回复

    豌豆君 小试身手Lv3

    发表于2021-4-20 09:28

    丨

    只看该作者

    丨

    取消 关注该作者的回复

    板凳

    jen 发表于 2021-4-20 09:35
    不好意思。第一次发帖，图放错了。是下面这张图。

    可以先查一下系统表结构字段类型、长度是否一直。字段类型长度等不一致可能导致迁移失败

    回复

    登录后可回答问题，请登录或注册

    有问题，来提问！
    我要发帖
    热门推荐
    热门视频
    热门产品推荐
    快速回复 返回顶部 返回列表

    小时

    全天响应

    分钟

    快速处理问题

    工程师强势助力

    明星产品
    一站式数据分析平台-亿信ABI
    智能数据治理平台-睿治
    解决方案
    大数据治理方案
    数据标准化及质量管控方案
    主数据管理方案
    数据资产盘点方案
    联系合作

    400咨询：400-0011-866

    技术支持QQ：400-0011-866(工作日9:00-18:00)

    产品建议邮箱yixin@esensoft.com

    关注我们

    扫TA学习更多干货

    一对一专家交流

    版权所有© 2006-2024 北京亿信华辰软件有限责任公司 京ICP备07017321号 京公网安备11010802016281号