Skip to content

Instantly share code, notes, and snippets.

@takus
Last active August 29, 2015 14:16
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save takus/50f7f5054468b34de063 to your computer and use it in GitHub Desktop.
Save takus/50f7f5054468b34de063 to your computer and use it in GitHub Desktop.
embulk error
in:
type: mysql
user: xxxx
password: xxxx
database: xxxx
host: xxxx
select: "*"
table: "test"
out:
type: file
path_prefix: test_
file_ext: csv
formatter:
type: csv
timezone: UTC
$ java -jar embulk.jar run mysql.yml
2015-03-12 06:17:05,493 [INFO]: main:org.embulk.input.MySQLInputPlugin: Fetch size is 10000. Using server-side prepared statement.
2015-03-12 06:17:05,930 [INFO]: main:org.embulk.exec.LocalExecutor: {done: 0 / 1, running: 0}
2015-03-12 06:17:05,954 [INFO]: embulk-executor-0:org.embulk.standards.LocalFileOutputPlugin: Writing local file 'test_.000.00csv'
2015-03-12 06:17:07,498 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetch size is 10000. Using server-side prepared statement.
2015-03-12 06:17:07,507 [INFO]: embulk-executor-0:org.embulk.input.mysql.MySQLInputConnection: SQL: SELECT * FROM `test`
2015-03-12 06:28:21,261 [INFO]: embulk-executor-0:org.embulk.input.mysql.MySQLInputConnection: > 673.75 seconds
2015-03-12 06:28:21,564 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 500 rows.
2015-03-12 06:28:21,596 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 1,000 rows.
2015-03-12 06:28:21,650 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 2,000 rows.
2015-03-12 06:28:21,753 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 4,000 rows.
2015-03-12 06:28:21,950 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 8,000 rows.
2015-03-12 06:28:22,550 [ERROR]: embulk-executor-0:io.netty.util.ResourceLeakDetector: LEAK: ByteBuf.release() was not called before it's garbage-collected. Ena
ble advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dio.netty.leakDetectionLevel=advanc
ed' or call ResourceLeakDetector.setLevel()
2015-03-12 06:28:22,556 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 16,000 rows.
2015-03-12 06:28:23,426 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 32,000 rows.
2015-03-12 06:28:25,069 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 64,000 rows.
2015-03-12 06:28:28,139 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 128,000 rows.
2015-03-12 06:28:34,397 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 256,000 rows.
2015-03-12 06:28:49,829 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 512,000 rows.
2015-03-12 06:29:14,110 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 1,024,000 rows.
2015-03-12 06:30:04,607 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 2,048,000 rows.
2015-03-12 06:31:47,832 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 4,096,000 rows.
2015-03-12 06:35:05,062 [INFO]: embulk-executor-0:org.embulk.input.MySQLInputPlugin: Fetched 8,192,000 rows.
2015-03-12 06:36:30,060 [INFO]: main:org.embulk.exec.LocalExecutor: {done: 1 / 1, running: 0}
2015-03-12 06:36:30,073 [INFO]: main:org.embulk.command.Runner: Writing resume state to 'status.yml'
2015-03-12 06:36:30,116 [INFO]: main:org.embulk.command.Runner: Resume state is written. Run the transaction again with -r option to resume or use "cleanup" sub command to delete intermediate data.
org/embulk/exec/LocalExecutor.java:333:in `buildPartialExecuteException':
org.embulk.exec.PartialExecutionException: java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
from org/embulk/exec/LocalExecutor.java:477:in `doRun'
from org/embulk/exec/LocalExecutor.java:40:in `access$000'
from org/embulk/exec/LocalExecutor.java:358:in `run'
from org/embulk/exec/LocalExecutor.java:355:in `run'
from org/embulk/spi/Exec.java:21:in `doWith'
from org/embulk/exec/LocalExecutor.java:355:in `run'
from org/embulk/command/Runner.java:111:in `run'
from org/embulk/command/Runner.java:64:in `main'
from java/lang/reflect/Method.java:483:in `invoke'
from file:/mnt/embulk/embulk.jar!/embulk/command/embulk_run.rb:275:in `run'
from classpath:embulk/command/embulk.rb:39:in `(root)'
from classpath_3a_embulk/command/classpath:embulk/command/embulk.rb:39:in `(root)'
from org/embulk/cli/Main.java:13:in `main'
Caused by:
LocalExecutor.java:271:in `getRepresentativeException': java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space [21/1271]
from LocalExecutor.java:445:in `run'
from FileOutputRunner.java:106:in `run'
from CsvFormatterPlugin.java:41:in `transaction'
from FileOutputRunner.java:100:in `run'
from Encoders.java:77:in `transaction'
from Encoders.java:33:in `transaction'
from FileOutputRunner.java:97:in `run'
from LocalFileOutputPlugin.java:59:in `resume'
from LocalFileOutputPlugin.java:51:in `transaction'
from FileOutputRunner.java:63:in `transaction'
from LocalExecutor.java:435:in `run'
from Filters.java:83:in `transaction'
from Filters.java:36:in `transaction'
from LocalExecutor.java:430:in `run'
from AbstractJdbcInputPlugin.java:137:in `transaction'
from LocalExecutor.java:425:in `doRun'
from LocalExecutor.java:40:in `access$000'
from LocalExecutor.java:358:in `run'
from LocalExecutor.java:355:in `run'
from Exec.java:21:in `doWith'
from LocalExecutor.java:355:in `run'
from Runner.java:111:in `run'
from Runner.java:64:in `main'
from NativeMethodAccessorImpl.java:-2:in `invoke0'
from NativeMethodAccessorImpl.java:62:in `invoke'
from DelegatingMethodAccessorImpl.java:43:in `invoke'
from Method.java:483:in `invoke'
from JavaMethod.java:470:in `invokeDirectWithExceptionHandling'
from JavaMethod.java:328:in `invokeDirect'
from InstanceMethodInvoker.java:71:in `call'
from CachingCallSite.java:346:in `cacheAndCall'
from CachingCallSite.java:204:in `call'
from CallTwoArgNode.java:59:in `interpret'
from NewlineNode.java:105:in `interpret'
from BlockNode.java:71:in `interpret'
from CaseNode.java:138:in `interpret'
from NewlineNode.java:105:in `interpret'
from BlockNode.java:71:in `interpret'
from ASTInterpreter.java:74:in `INTERPRET_METHOD'
from InterpretedMethod.java:182:in `call'
from DefaultMethod.java:203:in `call'
from CachingCallSite.java:326:in `cacheAndCall'
from CachingCallSite.java:170:in `call'
from classpath:embulk/command/embulk.rb:39:in `__file__'
from classpath:embulk/command/embulk.rb:-1:in `load'
from Ruby.java:866:in `runScript'
from Ruby.java:859:in `runScript'
from Ruby.java:728:in `runNormally'
from Ruby.java:577:in `runFromMain'
from Main.java:395:in `doRunFromMain'
from Main.java:290:in `internalRun'
from Main.java:217:in `run'
from Main.java:197:in `main'
from Main.java:13:in `main'
Caused by:
PoolArena.java:344:in `newChunk': java.lang.OutOfMemoryError: Java heap space
from PoolArena.java:143:in `allocateNormal'
from PoolArena.java:132:in `allocate'
from PoolArena.java:94:in `allocate'
from PooledByteBufAllocator.java:223:in `newHeapBuffer'
from AbstractByteBufAllocator.java:136:in `heapBuffer'
from AbstractByteBufAllocator.java:127:in `heapBuffer'
from AbstractByteBufAllocator.java:85:in `buffer'
from PooledBufferAllocator.java:31:in `allocate'
from PageBuilder.java:49:in `newBuffer'
from PageBuilder.java:205:in `flush'
from PageBuilder.java:182:in `addRecord'
from AbstractJdbcInputPlugin.java:277:in `fetch'
from AbstractJdbcInputPlugin.java:230:in `run'
from LocalExecutor.java:623:in `call'
from LocalExecutor.java:610:in `call'
from FutureTask.java:266:in `run'
from ThreadPoolExecutor.java:1142:in `runWorker'
from ThreadPoolExecutor.java:617:in `run'
from Thread.java:745:in `run'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment