Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@kamicup
Created March 26, 2015 11:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kamicup/4df8abd672b8b25335f4 to your computer and use it in GitHub Desktop.
Save kamicup/4df8abd672b8b25335f4 to your computer and use it in GitHub Desktop.
mroonga で CREATE FULLTEXT INDEX が失敗したときのエラーログ
Thread pointer: 0x2161d70
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fa668195ea0 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x2c)[0x8f49fc]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x364)[0x66f914]
/lib64/libpthread.so.0(+0xf130)[0x7fa69ea9b130]
/usr/lib64/libgroonga.so.0(grn_p_encv+0xac)[0x7fa683486a7c]
/usr/lib64/libgroonga.so.0(grn_ii_buffer_commit+0xe55)[0x7fa683499755]
/usr/lib64/libgroonga.so.0(grn_ii_build+0x4a5)[0x7fa68349af95]
/usr/lib64/libgroonga.so.0(+0x6cfe4)[0x7fa6833d9fe4]
/usr/lib64/libgroonga.so.0(grn_obj_set_info+0x800)[0x7fa6833db4a0]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga20storage_create_indexEP5TABLEPKcP8_grn_objP16st_mroonga_shareP6st_keyPS5_SA_j+0x318)[0x7fa683856ab8]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga33storage_inplace_alter_table_indexEP5TABLEP18Alter_inplace_info+0x488)[0x7fa6838647c8]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga27storage_inplace_alter_tableEP5TABLEP18Alter_inplace_info+0xae)[0x7fa6838653de]
/usr/local/mysql/bin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x1e1c)[0x74728c]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1ab4)[0x6f4814]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x388)[0x6f8868]
/usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xf47)[0x6f9df7]
/usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1ed)[0x6c26ad]
/usr/local/mysql/bin/mysqld(handle_one_connection+0x39)[0x6c26f9]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x140)[0x92ee30]
/lib64/libpthread.so.0(+0x7df3)[0x7fa69ea93df3]
/lib64/libc.so.6(clone+0x6d)[0x7fa69db6a1ad]
@kamicup
Copy link
Author

kamicup commented Mar 26, 2015

状況)

  • InnoDBエンジンで2.5GB程のテーブルを
  • mroongaエンジンのテーブルにコピー(insert into ... select * from ...)して
  • CREATE FULLTEXT INDEX ... を実行すると、
  • 300秒を超えるタイミングでプロセスが落ちて、
  • Lost connection to MySQL server during query となり、
  • DROP できない(壊れた?)テーブルになる。

バージョン)

  • mroonga: 5.00
  • mysql: 5.6.17
  • groonga: 5.0.0
  • os: Linux version 3.14.25-23.45.amzn1.x86_64 (mockbuild@gobi-build-64002) (gcc version 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC) ) #1 SMP Wed Dec 3 21:29:43 UTC 2014

mroonga と mysql は ↓ の内容に沿ってソースからインストールしたものです。
http://memocra.blogspot.jp/2013/05/mroonga-mroongamysql.html

@kamicup
Copy link
Author

kamicup commented Mar 27, 2015

groonga.log

2015-03-27 03:57:12.932573|n|9e339700|nterms=462 chunk=1904400 total=44587KB
2015-03-27 03:57:12.943388|n|9e339700|nterms=137 chunk=893072 total=45460KB
2015-03-27 03:57:12.947806|n|9e339700|nterms=42 chunk=545484 total=45992KB
2015-03-27 03:57:13.019737|n|9e339700|nterms=553 chunk=797655 total=46771KB
2015-03-27 03:57:13.039886|n|9e339700|nterms=588 chunk=1992541 total=48717KB
2015-03-27 03:57:13.199790|n|9e339700|nterms=2400 chunk=249774 total=48961KB
2015-03-27 03:57:13.276775|A|9e339700|malloc fail (80388216)=(nil) (ii.c:7402) <480>
2015-03-27 03:57:13.323972|A|9e339700|/usr/lib64/libgroonga.so.0(grn_malloc_default+0x11b) [0x7fe9b49b96ab]
2015-03-27 03:57:13.324074|A|9e339700|/usr/lib64/libgroonga.so.0(grn_ii_buffer_commit+0x182f) [0x7fe9b4ab812f]
2015-03-27 03:57:13.324132|A|9e339700|/usr/lib64/libgroonga.so.0(grn_ii_build+0x4a5) [0x7fe9b4ab8f95]
2015-03-27 03:57:13.324177|A|9e339700|/usr/lib64/libgroonga.so.0(+0x6cfe4) [0x7fe9b49f7fe4]
2015-03-27 03:57:13.324220|A|9e339700|/usr/lib64/libgroonga.so.0(grn_obj_set_info+0x800) [0x7fe9b49f94a0]
2015-03-27 03:57:13.324264|A|9e339700|/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga20storage_create_indexEP5TABLEPKcP8_grn_objP16st_mroonga_shareP6st_keyPS5_SA_j+0x318) [0x7fe9b4e74ab8]
2015-03-27 03:57:13.324302|A|9e339700|/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga33storage_inplace_alter_table_indexEP5TABLEP18Alter_inplace_info+0x488) [0x7fe9b4e827c8]
2015-03-27 03:57:13.324336|A|9e339700|/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga27storage_inplace_alter_tableEP5TABLEP18Alter_inplace_info+0xae) [0x7fe9b4e833de]
2015-03-27 03:57:13.324365|A|9e339700|/usr/local/mysql/bin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x1e1c) [0x74728c]
2015-03-27 03:57:13.324394|A|9e339700|/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1ab4) [0x6f4814]
2015-03-27 03:57:13.324424|A|9e339700|/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x388) [0x6f8868]
2015-03-27 03:57:13.324471|A|9e339700|/usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xf47) [0x6f9df7]
2015-03-27 03:57:13.324504|A|9e339700|/usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1ed) [0x6c26ad]
2015-03-27 03:57:13.324535|A|9e339700|/usr/local/mysql/bin/mysqld(handle_one_connection+0x39) [0x6c26f9]
2015-03-27 03:57:13.324565|A|9e339700|/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x140) [0x92ee30]
2015-03-27 03:57:13.324594|A|9e339700|/lib64/libpthread.so.0(+0x7df3) [0x7fe9d0094df3]
2015-03-27 03:57:14.020834|n|ed0c1740|mroonga 5.00 started.
2015-03-27 03:57:14.020855|n|ed0c1740|log level is 'NOTICE'
^C

ip-10-0-0-xx.err

2015-03-27 03:34:56 15426 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.6.17'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
03:57:13 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

key_buffer_size=268435456
read_buffer_size=1048576
max_used_connections=2
max_threads=151
thread_count=2
connection_count=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 573457 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x2a1c5f0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fe99e338ea0 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x2c)[0x8f49fc]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x364)[0x66f914]
/lib64/libpthread.so.0(+0xf130)[0x7fe9d009c130]
/usr/lib64/libgroonga.so.0(grn_p_encv+0xac)[0x7fe9b4aa4a7c]
/usr/lib64/libgroonga.so.0(grn_ii_buffer_commit+0xe55)[0x7fe9b4ab7755]
/usr/lib64/libgroonga.so.0(grn_ii_build+0x4a5)[0x7fe9b4ab8f95]
/usr/lib64/libgroonga.so.0(+0x6cfe4)[0x7fe9b49f7fe4]
/usr/lib64/libgroonga.so.0(grn_obj_set_info+0x800)[0x7fe9b49f94a0]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga20storage_create_indexEP5TABLEPKcP8_grn_objP16st_mroonga_shareP6st_keyPS5_SA_j+0x318)[0x7fe9b4e74ab8]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga33storage_inplace_alter_table_indexEP5TABLEP18Alter_inplace_info+0x488)[0x7fe9b4e827c8]
/usr/local/mysql/lib/plugin/ha_mroonga.so(_ZN10ha_mroonga27storage_inplace_alter_tableEP5TABLEP18Alter_inplace_info+0xae)[0x7fe9b4e833de]
/usr/local/mysql/bin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x1e1c)[0x74728c]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x1ab4)[0x6f4814]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x388)[0x6f8868]
/usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xf47)[0x6f9df7]
/usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1ed)[0x6c26ad]
/usr/local/mysql/bin/mysqld(handle_one_connection+0x39)[0x6c26f9]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x140)[0x92ee30]
/lib64/libpthread.so.0(+0x7df3)[0x7fe9d0094df3]
/lib64/libc.so.6(clone+0x6d)[0x7fe9cf16b1ad]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fe980004f50): is an invalid pointer
Connection ID (thread ID): 1
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
150327 03:57:13 mysqld_safe Number of processes running now: 0
150327 03:57:13 mysqld_safe mysqld restarted
2015-03-27 03:57:13 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-03-27 03:57:13 16215 [Warning] You need to use --log-bin to make --binlog-format work.
2015-03-27 03:57:13 16215 [Note] Plugin 'FEDERATED' is disabled.
2015-03-27 03:57:13 7ffaed0c1740 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
2015-03-27 03:57:13 16215 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-03-27 03:57:13 16215 [Note] InnoDB: The InnoDB memory heap is disabled
2015-03-27 03:57:13 16215 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-03-27 03:57:13 16215 [Note] InnoDB: Compressed tables use zlib 1.2.3
2015-03-27 03:57:13 16215 [Note] InnoDB: Using CPU crc32 instructions
2015-03-27 03:57:13 16215 [Note] InnoDB: Initializing buffer pool, size = 256.0M
2015-03-27 03:57:13 16215 [Note] InnoDB: Completed initialization of buffer pool
2015-03-27 03:57:13 16215 [Note] InnoDB: Highest supported file format is Barracuda.
2015-03-27 03:57:13 16215 [Note] InnoDB: The log sequence numbers 13389522340 and 13389522340 in ibdata files do not match the log sequence number 13389522350 in the ib_logfiles!
2015-03-27 03:57:13 16215 [Note] InnoDB: Database was not shutdown normally!
2015-03-27 03:57:13 16215 [Note] InnoDB: Starting crash recovery.
2015-03-27 03:57:13 16215 [Note] InnoDB: Reading tablespace information from the .ibd files...
2015-03-27 03:57:13 16215 [Note] InnoDB: Restoring possible half-written data pages 
2015-03-27 03:57:13 16215 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 3134201, file name mysql-bin.000020
2015-03-27 03:57:13 16215 [Note] InnoDB: 128 rollback segment(s) are active.
2015-03-27 03:57:13 16215 [Note] InnoDB: Waiting for purge to start
2015-03-27 03:57:14 16215 [Note] InnoDB: 5.6.17 started; log sequence number 13389522350
2015-03-27 03:57:14 16215 [Note] Server hostname (bind-address): '*'; port: 3306
2015-03-27 03:57:14 16215 [Note] IPv6 is available.
2015-03-27 03:57:14 16215 [Note]   - '::' resolves to '::';
2015-03-27 03:57:14 16215 [Note] Server socket created on IP: '::'.
2015-03-27 03:57:14 16215 [Note] Event Scheduler: Loaded 0 events
2015-03-27 03:57:14 16215 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.6.17'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment