Skip to content

Instantly share code, notes, and snippets.

@dreamca4er
Created September 19, 2016 08:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dreamca4er/c775abf76792b870b562a5f0c0322b14 to your computer and use it in GitHub Desktop.
Save dreamca4er/c775abf76792b870b562a5f0c0322b14 to your computer and use it in GitHub Desktop.
19.09.2016 4:29:33 <INFO> Execution Progress.Initializing properties: 1%
19.09.2016 4:29:34 <INFO> Using sybase driver for url jdbc:sybase:Tds:db3.billing.novotelecom.ru:4100/v083?CHARSET=koi8
19.09.2016 4:29:34 <INFO> Execution Progress.Initialized connection id=v083, JdbcConnection{com.sybase.jdbc3.jdbc.SybConnection}, Dialect{Adaptive Server Enterprise Adaptive Server Enterprise/12.5.4/EBF 13739 ESD#1/P/Linux Intel/Enterprise Linux/ase1254/2015/32-bit/OPT/Mon Aug 7 22:57:44 2006}, properties {}: 3%
19.09.2016 4:29:35 <INFO> Using postgresql driver for url jdbc:postgresql://eva:5432/jira_arhiv
19.09.2016 4:29:35 <INFO> Execution Progress.Initialized connection id=eva, JdbcConnection{org.postgresql.jdbc3.Jdbc3Connection}, Dialect{PostgreSQL 9.3.6}, properties {flushBeforeQuery=true, statement.batchSize=100}: 5%
19.09.2016 4:29:35 <INFO> Execution Progress./etl/query[1] prepared: 6%
19.09.2016 4:29:35 <INFO> Execution Progress./etl/query[3] prepared: 7%
19.09.2016 4:29:35 <INFO> Execution Progress./etl/script[3] prepared: 10%
19.09.2016 4:29:35 <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/opt/projects/dwh_load/v083/load_v083_after_tarificator.etl.xml"
19.09.2016 4:29:35 <INFO> Execution Progress./etl/script[1] executed: 23%
19.09.2016 4:29:59 <INFO> Execution Progress./etl/query[1] executed: 36%
19.09.2016 4:41:32 <INFO> Executed statement select
SE.SE_ID as se_id,
SE.LC_Num as lc_num,
SE.SVC_ID, SE.LOGIN, SE.DELETED, SE.se1c_id,
SE.parent_SE_ID as parent_se_id,
SE.TC_ID as tc_id,
SE.created, SE.removed, SE.activated, SE.canceled,
SE.set_SE_ID as set_se_id,
LC.CONTRACT_IDENT as lc_contract_ident,
LC.SHORT_NAME as lc_short_name,
LC.COMPANY as lc_company,
address_kladr.VALUE as pr_address_kladr_tmp,
case
when SE.SVC_ID in (20,81)
then (select sp.VALUE from SE_PROPS sp
join NTK_PORT np on np.SE_ID = sp.SE_ID
join NTK_SE_RESOURCE res on res.RESOURCE_ID = np.port_id and res.RES_TYPE_ID = 2
where sp.NAME = 'ADDRESS_KLADR'
and res.SE_ID = SE.SE_ID
)
when SE.SVC_ID in (503,666,667,514,518)
then (select sp.VALUE from SE_PROPS sp
join NTK_PORT np on np.SE_ID = sp.SE_ID
join NTK_SE_RESOURCE res on res.RESOURCE_ID = np.port_id and res.RES_TYPE_ID = 2
where sp.NAME = 'ADDRESS_KLADR'
and res.SE_ID = SE.parent_SE_ID
)
when SE.SVC_ID = 3
then (select sp.VALUE from SE_PROPS sp
join NTK_PORT np on np.SE_ID = sp.SE_ID
join NTK_SE_RESOURCE res on res.RESOURCE_ID = np.port_id and res.RES_TYPE_ID = 2
join NTK_REGISTRY reg on reg.gw_id = res.SE_ID
where sp.NAME = 'ADDRESS_KLADR'
and reg.SE_ID = SE.SE_ID
)
else (select sp.VALUE from SE_PROPS sp
where sp.SE_ID = SE.SE_ID
and sp.NAME = 'ADDRESS_KLADR'
)
end as pr_address_kladr,
block_state.VALUE as pr_block_state,
block_state_updated.VALUE as pr_block_state_updated,
cost_all.VALUE as pr_cost_all,
cost_first.VALUE as pr_cost_first,
cost_period.VALUE as pr_cost_period,
oper_id.VALUE as pr_oper_id,
sost.VALUE as pr_sost,
cost_sum.VALUE as acc_cost_sum
from SE
left join SE_PROPS address_kladr on SE.SE_ID = address_kladr.SE_ID and address_kladr.NAME = 'ADDRESS_KLADR'
left join SE_PROPS block_state on SE.SE_ID = block_state.SE_ID and block_state.NAME = 'BLOCK_STATE'
left join SE_PROPS block_state_updated on SE.SE_ID = block_state_updated.SE_ID and block_state_updated.NAME = 'BLOCK_STATE_UPDATED'
left join SE_PROPS cost_all on SE.SE_ID = cost_all.SE_ID and cost_all.NAME = 'COST_ALL'
left join SE_PROPS cost_first on SE.SE_ID = cost_first.SE_ID and cost_first.NAME = 'COST_FIRST'
left join SE_PROPS cost_period on SE.SE_ID = cost_period.SE_ID and cost_period.NAME = 'COST_PERIOD'
left join SE_PROPS oper_id on SE.SE_ID = oper_id.SE_ID and oper_id.NAME = 'OPER_ID'
left join SE_PROPS sost on SE.SE_ID = sost.SE_ID and sost.NAME = 'SOST'
left join SE_ACCUM cost_sum on SE.SE_ID = cost_sum.SE_ID and cost_sum.NAME = 'COST_SUM'
left join LC on SE.LC_Num = LC.LC_Num
where SE.removed >= '2012-12-31 17:00:00' or SE.removed is null. SQL Warnings:
* java.sql.SQLWarning: Character set conversion is not available between client character set 'koi8' and server character set 'iso_1'.
* java.sql.SQLWarning: No conversions will be done.
19.09.2016 4:41:32 <INFO> Execution Progress./etl/query[2] executed: 49%
19.09.2016 4:41:32 <INFO> Execution Progress./etl/query[3] executed: 62%
19.09.2016 4:41:32 <INFO> Execution Progress./etl/script[2] executed: 75%
19.09.2016 4:41:32 <INFO> Execution Progress./etl/script[3] executed: 95%
19.09.2016 4:41:32 <INFO> Execution Progress.Complete
19.09.2016 7:12:38 <INFO> Execution statistics:
Executed 3 queries, 4 scripts, 1332064 statements
/etl/script[1]: Element successfully executed (2 statements). Working time 14 milliseconds. Avg throughput: 138,98 statements/sec.
/etl/query[1]: Element successfully executed. Working time 24604 milliseconds.
/etl/query[2]: Element successfully executed (1 statement). Working time 10564 milliseconds. Avg throughput: 0,09 statements/sec.
/etl/query[2]/script[1]: Element successfully executed 1332059 times (1332059 statements). Working time 681759 milliseconds. Avg throughput: 1953,85 statements/sec.
/etl/query[3]: Element successfully executed. Working time 18 milliseconds.
/etl/script[2]: Element successfully executed (1 statement). Working time 1 milliseconds. Avg throughput: 666,38 statements/sec.
/etl/script[3]: Element successfully executed (1 statement). Working time 0 milliseconds. Avg throughput: 6392,96 statements/sec.
Total working time: 2 hours 43 minutes 4,25 seconds
19.09.2016 7:12:38 <INFO> Successfully executed ETL file /opt/projects/dwh_load/v083/load_v083_after_tarificator.etl.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment