Skip to content

Instantly share code, notes, and snippets.

@qix

qix/gist:035556db5ffbf9415550

Created April 29, 2015 19:45
Show Gist options
  • Save qix/035556db5ffbf9415550 to your computer and use it in GitHub Desktop.
Save qix/035556db5ffbf9415550 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
```
mysql> SHOW AGGREGATORS;
+----------------+------+--------+--------------------+------------------------------+-------------------+
| Host | Port | State | Opened_Connections | Average_Roundtrip_Latency_ms | Master_Aggregator |
+----------------+------+--------+--------------------+------------------------------+-------------------+
| 127.0.0.1 | 3306 | online | 0 | NULL | 1 |
| 10.240.76.251 | 3306 | online | 2 | 1.184 | 0 |
| 10.240.194.230 | 3306 | online | 2 | 1.175 | 0 |
+----------------+------+--------+--------------------+------------------------------+-------------------+
mysql> SHOW LEAVES;
+--------------+------+--------------------+--------------+-----------+--------+--------------------+------------------------------+
| Host | Port | Availability_Group | Pair_Host | Pair_Port | State | Opened_Connections | Average_Roundtrip_Latency_ms |
+--------------+------+--------------------+--------------+-----------+--------+--------------------+------------------------------+
| memsql-leaf1 | 3306 | 1 | memsql-leaf3 | 3306 | online | 34 | 1.279 |
| memsql-leaf2 | 3306 | 1 | memsql-leaf4 | 3306 | online | 35 | 1.163 |
| memsql-leaf3 | 3306 | 2 | memsql-leaf1 | 3306 | online | 36 | 0.828 |
| memsql-leaf4 | 3306 | 2 | memsql-leaf2 | 3306 | online | 34 | 0.770 |
| memsql-leaf5 | 3306 | 2 | memsql-leaf6 | 3306 | online | 5 | 0.893 |
| memsql-leaf6 | 3306 | 1 | memsql-leaf5 | 3306 | online | 4 | 0.702 |
+--------------+------+--------------------+--------------+-----------+--------+--------------------+------------------------------+
mysql> SHOW PARTITIONS;
+---------+--------------+------+--------+--------+
| Ordinal | Host | Port | Role | Locked |
+---------+--------------+------+--------+--------+
| 0 | memsql-leaf1 | 3306 | Master | 0 |
| 0 | memsql-leaf3 | 3306 | Slave | 0 |
| 1 | memsql-leaf2 | 3306 | Master | 0 |
| 1 | memsql-leaf4 | 3306 | Slave | 0 |
| 2 | memsql-leaf3 | 3306 | Master | 0 |
| 2 | memsql-leaf1 | 3306 | Slave | 0 |
| 3 | memsql-leaf4 | 3306 | Master | 0 |
| 3 | memsql-leaf2 | 3306 | Slave | 0 |
| 3 | memsql-leaf5 | 3306 | Slave | 0 |
| 3 | memsql-leaf6 | 3306 | Slave | 0 |
| 4 | memsql-leaf1 | 3306 | Master | 0 |
| 4 | memsql-leaf3 | 3306 | Slave | 0 |
| 5 | memsql-leaf2 | 3306 | Master | 0 |
| 5 | memsql-leaf4 | 3306 | Slave | 0 |
| 6 | memsql-leaf3 | 3306 | Master | 0 |
| 6 | memsql-leaf1 | 3306 | Slave | 0 |
| 7 | memsql-leaf4 | 3306 | Master | 0 |
| 7 | memsql-leaf2 | 3306 | Slave | 0 |
| 8 | memsql-leaf1 | 3306 | Master | 0 |
| 8 | memsql-leaf3 | 3306 | Slave | 0 |
| 9 | memsql-leaf2 | 3306 | Master | 0 |
| 9 | memsql-leaf4 | 3306 | Slave | 0 |
| 10 | memsql-leaf3 | 3306 | Master | 0 |
| 10 | memsql-leaf1 | 3306 | Slave | 0 |
| 11 | memsql-leaf4 | 3306 | Master | 0 |
| 11 | memsql-leaf2 | 3306 | Slave | 0 |
| 12 | memsql-leaf1 | 3306 | Master | 0 |
| 12 | memsql-leaf3 | 3306 | Slave | 0 |
| 13 | memsql-leaf2 | 3306 | Master | 0 |
| 13 | memsql-leaf4 | 3306 | Slave | 0 |
| 14 | memsql-leaf3 | 3306 | Master | 0 |
| 14 | memsql-leaf1 | 3306 | Slave | 0 |
| 15 | memsql-leaf4 | 3306 | Master | 0 |
| 15 | memsql-leaf2 | 3306 | Slave | 0 |
| 16 | memsql-leaf1 | 3306 | Master | 0 |
| 16 | memsql-leaf3 | 3306 | Slave | 0 |
| 17 | memsql-leaf2 | 3306 | Master | 0 |
| 17 | memsql-leaf4 | 3306 | Slave | 0 |
| 18 | memsql-leaf3 | 3306 | Master | 0 |
| 18 | memsql-leaf1 | 3306 | Slave | 0 |
| 19 | memsql-leaf4 | 3306 | Master | 0 |
| 19 | memsql-leaf2 | 3306 | Slave | 0 |
| 20 | memsql-leaf1 | 3306 | Master | 0 |
| 20 | memsql-leaf3 | 3306 | Slave | 0 |
| 21 | memsql-leaf2 | 3306 | Master | 0 |
| 21 | memsql-leaf4 | 3306 | Slave | 0 |
| 22 | memsql-leaf3 | 3306 | Master | 0 |
| 22 | memsql-leaf1 | 3306 | Slave | 0 |
| 23 | memsql-leaf4 | 3306 | Master | 0 |
| 23 | memsql-leaf2 | 3306 | Slave | 0 |
| 24 | memsql-leaf1 | 3306 | Master | 0 |
| 24 | memsql-leaf3 | 3306 | Slave | 0 |
| 25 | memsql-leaf2 | 3306 | Master | 0 |
| 25 | memsql-leaf4 | 3306 | Slave | 0 |
| 26 | memsql-leaf3 | 3306 | Master | 0 |
| 26 | memsql-leaf1 | 3306 | Slave | 0 |
| 27 | memsql-leaf4 | 3306 | Master | 0 |
| 27 | memsql-leaf2 | 3306 | Slave | 0 |
| 28 | memsql-leaf1 | 3306 | Master | 0 |
| 28 | memsql-leaf3 | 3306 | Slave | 0 |
| 29 | memsql-leaf2 | 3306 | Master | 0 |
| 29 | memsql-leaf4 | 3306 | Slave | 0 |
| 30 | memsql-leaf3 | 3306 | Master | 0 |
| 30 | memsql-leaf1 | 3306 | Slave | 0 |
| 31 | memsql-leaf4 | 3306 | Master | 0 |
| 31 | memsql-leaf2 | 3306 | Slave | 0 |
+---------+--------------+------+--------+--------+
66 rows in set (0.02 sec)
mysql> EXPLAIN REBALANCE PARTITIONS;
+-------------------+---------+--------------+-------------+-------+
| Action | Ordinal | Target_Host | Target_Port | Phase |
+-------------------+---------+--------------+-------------+-------+
| COPY PARTITION | 1 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 1 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 0 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 0 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 3 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 3 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 2 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 2 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 5 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 5 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 4 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 4 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 7 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 7 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 6 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 6 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 9 | memsql-leaf5 | 3306 | 3 |
| COPY PARTITION | 9 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 8 | memsql-leaf6 | 3306 | 3 |
| COPY PARTITION | 8 | memsql-leaf5 | 3306 | 3 |
| PROMOTE PARTITION | 0 | memsql-leaf6 | 3306 | 4 |
| PROMOTE PARTITION | 1 | memsql-leaf5 | 3306 | 4 |
| PROMOTE PARTITION | 2 | memsql-leaf6 | 3306 | 4 |
| PROMOTE PARTITION | 3 | memsql-leaf5 | 3306 | 4 |
| PROMOTE PARTITION | 4 | memsql-leaf6 | 3306 | 4 |
| PROMOTE PARTITION | 5 | memsql-leaf5 | 3306 | 4 |
| PROMOTE PARTITION | 6 | memsql-leaf6 | 3306 | 4 |
| PROMOTE PARTITION | 7 | memsql-leaf5 | 3306 | 4 |
| PROMOTE PARTITION | 8 | memsql-leaf6 | 3306 | 4 |
| PROMOTE PARTITION | 9 | memsql-leaf5 | 3306 | 4 |
| DROP PARTITION | 0 | memsql-leaf3 | 3306 | 5 |
| DROP PARTITION | 1 | memsql-leaf4 | 3306 | 5 |
| DROP PARTITION | 2 | memsql-leaf1 | 3306 | 5 |
| DROP PARTITION | 3 | memsql-leaf2 | 3306 | 5 |
| DROP PARTITION | 4 | memsql-leaf3 | 3306 | 5 |
| DROP PARTITION | 5 | memsql-leaf4 | 3306 | 5 |
| DROP PARTITION | 6 | memsql-leaf1 | 3306 | 5 |
| DROP PARTITION | 7 | memsql-leaf2 | 3306 | 5 |
| DROP PARTITION | 8 | memsql-leaf3 | 3306 | 5 |
| DROP PARTITION | 9 | memsql-leaf4 | 3306 | 5 |
+-------------------+---------+--------------+-------------+-------+
40 rows in set (0.04 sec)
mysql> REBALANCE PARTITIONS;
mysql> SHOW REBALANCE STATUS;
+-------------------+---------+--------------+-------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+
| Action | Ordinal | Target_Host | Target_Port | Phase | Status | Running_Time |
+-------------------+---------+--------------+-------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+
| COPY PARTITION | 1 | memsql-leaf5 | 3306 | 3 | success | 229462 |
| COPY PARTITION | 1 | memsql-leaf6 | 3306 | 3 | success | 251414 |
| COPY PARTITION | 0 | memsql-leaf6 | 3306 | 3 | success | 256079 |
| COPY PARTITION | 0 | memsql-leaf5 | 3306 | 3 | success | 256079 |
| COPY PARTITION | 3 | memsql-leaf5 | 3306 | 3 | success | 256079 |
| COPY PARTITION | 3 | memsql-leaf6 | 3306 | 3 | success | 256077 |
| COPY PARTITION | 2 | memsql-leaf6 | 3306 | 3 | success | 256077 |
| COPY PARTITION | 2 | memsql-leaf5 | 3306 | 3 | success | 256077 |
| COPY PARTITION | 5 | memsql-leaf5 | 3306 | 3 | success | 256077 |
| COPY PARTITION | 5 | memsql-leaf6 | 3306 | 3 | success | 256076 |
| COPY PARTITION | 4 | memsql-leaf6 | 3306 | 3 | success | 256076 |
| COPY PARTITION | 4 | memsql-leaf5 | 3306 | 3 | success | 256076 |
| COPY PARTITION | 7 | memsql-leaf5 | 3306 | 3 | success | 256076 |
| COPY PARTITION | 7 | memsql-leaf6 | 3306 | 3 | success | 256076 |
| COPY PARTITION | 6 | memsql-leaf6 | 3306 | 3 | success | 260555 |
| COPY PARTITION | 6 | memsql-leaf5 | 3306 | 3 | success | 260555 |
| COPY PARTITION | 9 | memsql-leaf5 | 3306 | 3 | success | 260555 |
| COPY PARTITION | 9 | memsql-leaf6 | 3306 | 3 | success | 260554 |
| COPY PARTITION | 8 | memsql-leaf6 | 3306 | 3 | success | 260554 |
| COPY PARTITION | 8 | memsql-leaf5 | 3306 | 3 | success | 260554 |
| PROMOTE PARTITION | 0 | memsql-leaf6 | 3306 | 4 | success | 3297 |
| PROMOTE PARTITION | 1 | memsql-leaf5 | 3306 | 4 | success | 3854 |
| PROMOTE PARTITION | 2 | memsql-leaf6 | 3306 | 4 | success | 3854 |
| PROMOTE PARTITION | 3 | memsql-leaf5 | 3306 | 4 | success | 3854 |
| PROMOTE PARTITION | 4 | memsql-leaf6 | 3306 | 4 | failed (Query 'PAUSE REPLICATING `authbox_4`' failed on leaf memsql-leaf3:3306 while pausing replication (preparing to failover) with error 1733: Unable to replay files for database 'authbox_4'. Run SHOW DATABASES EXTENDED and visit http://developers.memsql.com/docs/4.0/database_states.html for more information) | 3854 |
| PROMOTE PARTITION | 5 | memsql-leaf5 | 3306 | 4 | success | 4038 |
| PROMOTE PARTITION | 6 | memsql-leaf6 | 3306 | 4 | success | 4038 |
| PROMOTE PARTITION | 7 | memsql-leaf5 | 3306 | 4 | success | 4038 |
| PROMOTE PARTITION | 8 | memsql-leaf6 | 3306 | 4 | success | 4038 |
| PROMOTE PARTITION | 9 | memsql-leaf5 | 3306 | 4 | success | 4196 |
| DROP PARTITION | 0 | memsql-leaf3 | 3306 | 5 | success | 1900 |
| DROP PARTITION | 1 | memsql-leaf4 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 2 | memsql-leaf1 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 3 | memsql-leaf2 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 4 | memsql-leaf3 | 3306 | 5 | scheduled | NULL |
| DROP PARTITION | 5 | memsql-leaf4 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 6 | memsql-leaf1 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 7 | memsql-leaf2 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 8 | memsql-leaf3 | 3306 | 5 | success | 3196 |
| DROP PARTITION | 9 | memsql-leaf4 | 3306 | 5 | success | 3196 |
+-------------------+---------+--------------+-------------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------+
40 rows in set (0.00 sec)
@qix
Copy link
Author

qix commented Apr 29, 2015

I noted that the partitions on memsql-leaf3 where unrecoverable, and started a new rebalance. Note the plan is a bit strange (copy partition 4 to memsql-leaf3, and later drop it.)

memsql-leaf3> SHOW DATABASES EXTENDED;
+--------------------+---------+-------------+---------------+-------------+---------------------------------------------------------+-------------+------------+--------------+
| Database           | Commits | Role        | State         | Position    | Details                                                 | AsyncSlaves | SyncSlaves | Memory (MBs) |
+--------------------+---------+-------------+---------------+-------------+---------------------------------------------------------+-------------+------------+--------------+
| authbox            |      34 | async slave | replicating   | 0:41945     |                                                         | 0           | 0          |         0.00 |
| authbox_10         |  521024 | master      | online        | 0:137604357 |                                                         | 1           | 0          |       170.96 |
| authbox_12         |  479491 | async slave | replicating   | 0:124652369 |                                                         | 0           | 0          |       157.37 |
| authbox_14         |  499482 | master      | online        | 0:132037635 |                                                         | 1           | 0          |       168.71 |
| authbox_16         |  479775 | async slave | replicating   | 0:125783246 |                                                         | 0           | 0          |       160.03 |
| authbox_18         |  495463 | master      | online        | 0:129546071 |                                                         | 1           | 0          |       166.31 |
| authbox_20         |  496389 | master      | unrecoverable | NULL        | See /var/lib/memsql/tracelogs/memsql.log for more info. | 0           | 0          |       167.46 |
| authbox_22         |  507457 | master      | online        | 0:133362205 |                                                         | 1           | 0          |       167.99 |
| authbox_24         |  774056 | master      | unrecoverable | NULL        | See /var/lib/memsql/tracelogs/memsql.log for more info. | 0           | 0          |       252.38 |
| authbox_26         |  530212 | master      | online        | 0:139780242 |                                                         | 1           | 0          |       171.32 |
| authbox_28         |  491317 | async slave | replicating   | 0:128655408 |                                                         | 0           | 0          |       162.59 |
| authbox_30         |  540925 | master      | online        | 0:137626853 |                                                         | 0           | 0          |       172.54 |
| authbox_4          |  471001 | master      | unrecoverable | NULL        | See /var/lib/memsql/tracelogs/memsql.log for more info. | 0           | 0          |       160.42 |
| information_schema |      36 | master      | online        | 0:91580     |                                                         | 0           | 0          |         0.00 |
| memsql             |      33 | master      | online        | 0:2129      |                                                         | 0           | 0          |         0.00 |
| sharding           |     681 | sync slave  | replicating   | 0:31463     |                                                         | 0           | 0          |         0.00 |
+--------------------+---------+-------------+---------------+-------------+---------------------------------------------------------+-------------+------------+--------------+
16 rows in set (0.00 sec)

mysql> DROP PARTITION authbox:4 ON `memsql-leaf3`;
Query OK, 1 row affected (0.24 sec)

mysql> DROP PARTITION authbox:24 ON `memsql-leaf3`;
Query OK, 1 row affected (0.32 sec)

mysql> DROP PARTITION authbox:20 ON `memsql-leaf3`;
Query OK, 1 row affected (0.32 sec)

mysql> EXPLAIN REBALANCE PARTITIONS;
+-------------------+---------+--------------+-------------+-------+
| Action            | Ordinal | Target_Host  | Target_Port | Phase |
+-------------------+---------+--------------+-------------+-------+
| COPY PARTITION    |       4 | memsql-leaf3 |        3306 |     2 |
| COPY PARTITION    |      20 | memsql-leaf3 |        3306 |     2 |
| COPY PARTITION    |      24 | memsql-leaf3 |        3306 |     2 |
| COPY PARTITION    |       4 | memsql-leaf6 |        3306 |     3 |
| COPY PARTITION    |       4 | memsql-leaf5 |        3306 |     3 |
| PROMOTE PARTITION |       4 | memsql-leaf6 |        3306 |     4 |
| DROP PARTITION    |       4 | memsql-leaf3 |        3306 |     5 |
+-------------------+---------+--------------+-------------+-------+

@qix
Copy link
Author

qix commented Apr 29, 2015

We now good though

mysql> EXPLAIN REBALANCE PARTITIONS;
Empty set (0.00 sec)

@qix
Copy link
Author

qix commented Apr 29, 2015

Killed memsql-leaf3 with kill -9

Everything seems to be working fine.

mysql> SHOW PARTITIONS;
+---------+--------------+------+--------+--------+
| Ordinal | Host         | Port | Role   | Locked |
+---------+--------------+------+--------+--------+
|       0 | memsql-leaf6 | 3306 | Master |      0 |
|       0 | memsql-leaf5 | 3306 | Slave  |      0 |
|       1 | memsql-leaf6 | 3306 | Slave  |      0 |
|       1 | memsql-leaf5 | 3306 | Master |      0 |
|       2 | memsql-leaf6 | 3306 | Master |      0 |
|       2 | memsql-leaf5 | 3306 | Slave  |      0 |
|       3 | memsql-leaf6 | 3306 | Slave  |      0 |
|       3 | memsql-leaf5 | 3306 | Master |      0 |
|       4 | memsql-leaf6 | 3306 | Master |      0 |
|       4 | memsql-leaf5 | 3306 | Slave  |      0 |
|       5 | memsql-leaf6 | 3306 | Slave  |      0 |
|       5 | memsql-leaf5 | 3306 | Master |      0 |
|       6 | memsql-leaf6 | 3306 | Master |      0 |
|       6 | memsql-leaf5 | 3306 | Slave  |      0 |
|       7 | memsql-leaf6 | 3306 | Slave  |      0 |
|       7 | memsql-leaf5 | 3306 | Master |      0 |
|       8 | memsql-leaf6 | 3306 | Master |      0 |
|       8 | memsql-leaf5 | 3306 | Slave  |      0 |
|       9 | memsql-leaf6 | 3306 | Slave  |      0 |
|       9 | memsql-leaf5 | 3306 | Master |      0 |
|      10 | memsql-leaf1 | 3306 | Master |      0 |
|      11 | memsql-leaf4 | 3306 | Master |      0 |
|      11 | memsql-leaf2 | 3306 | Slave  |      0 |
|      12 | memsql-leaf1 | 3306 | Master |      0 |
|      13 | memsql-leaf2 | 3306 | Master |      0 |
|      13 | memsql-leaf4 | 3306 | Slave  |      0 |
|      14 | memsql-leaf1 | 3306 | Master |      0 |
|      15 | memsql-leaf4 | 3306 | Master |      0 |
|      15 | memsql-leaf2 | 3306 | Slave  |      0 |
|      16 | memsql-leaf1 | 3306 | Master |      0 |
|      17 | memsql-leaf2 | 3306 | Master |      0 |
|      17 | memsql-leaf4 | 3306 | Slave  |      0 |
|      18 | memsql-leaf1 | 3306 | Master |      0 |
|      19 | memsql-leaf4 | 3306 | Master |      0 |
|      19 | memsql-leaf2 | 3306 | Slave  |      0 |
|      20 | memsql-leaf1 | 3306 | Master |      0 |
|      21 | memsql-leaf2 | 3306 | Master |      0 |
|      21 | memsql-leaf4 | 3306 | Slave  |      0 |
|      22 | memsql-leaf1 | 3306 | Master |      0 |
|      23 | memsql-leaf4 | 3306 | Master |      0 |
|      23 | memsql-leaf2 | 3306 | Slave  |      0 |
|      24 | memsql-leaf1 | 3306 | Master |      0 |
|      25 | memsql-leaf2 | 3306 | Master |      0 |
|      25 | memsql-leaf4 | 3306 | Slave  |      0 |
|      26 | memsql-leaf1 | 3306 | Master |      0 |
|      27 | memsql-leaf4 | 3306 | Master |      0 |
|      27 | memsql-leaf2 | 3306 | Slave  |      0 |
|      28 | memsql-leaf1 | 3306 | Master |      0 |
|      29 | memsql-leaf2 | 3306 | Master |      0 |
|      29 | memsql-leaf4 | 3306 | Slave  |      0 |
|      30 | memsql-leaf1 | 3306 | Master |      0 |
|      31 | memsql-leaf4 | 3306 | Master |      0 |
|      31 | memsql-leaf2 | 3306 | Slave  |      0 |
+---------+--------------+------+--------+--------+
53 rows in set (0.00 sec)

@qix
Copy link
Author

qix commented Apr 29, 2015

Brought it back, but now seeing some strange issues. Why is it failing?

mysql> select IpAddress, COUNT(DISTINCT Machine) as numUniqueMachines from feed_action where timestamp < date_sub(now(), interval -1 day) group by 1 order by numUniqueMachines desc limit 100;
ERROR 1146 (42S02): Leaf Error (memsql-leaf1:3306): Table 'authbox_30.feed_action' doesn't exist

memsql-leaf1> SHOW DATABASES EXTENDED;
+--------------------+---------+-------------+-------------+-------------+---------+-------------+------------+--------------+
| Database           | Commits | Role        | State       | Position    | Details | AsyncSlaves | SyncSlaves | Memory (MBs) |
+--------------------+---------+-------------+-------------+-------------+---------+-------------+------------+--------------+
| authbox            |      34 | async slave | replicating | 0:41945     |         | 0           | 0          |         0.00 |
| authbox_10         |  521094 | master      | online      | 0:137620507 |         | 0           | 0          |       170.72 |
| authbox_12         |  479638 | master      | online      | 0:124684308 |         | 1           | 0          |       157.63 |
| authbox_14         |  499573 | master      | online      | 0:132057215 |         | 0           | 0          |       168.72 |
| authbox_16         |  479894 | master      | online      | 0:125808031 |         | 1           | 0          |       160.92 |
| authbox_18         |  495597 | master      | online      | 0:129577320 |         | 0           | 0          |       165.33 |
| authbox_20         |  530029 | master      | online      | 0:138658452 |         | 1           | 0          |       176.91 |
| authbox_22         |  507567 | master      | online      | 0:133391110 |         | 0           | 0          |       167.00 |
| authbox_24         |  808450 | master      | online      | 0:230061275 |         | 1           | 0          |       257.15 |
| authbox_26         |  530372 | master      | online      | 0:139818638 |         | 0           | 0          |       172.33 |
| authbox_28         |  491427 | master      | online      | 0:128676391 |         | 0           | 0          |       161.73 |
| authbox_30         |  485120 | master      | online      | 0:126753310 |         | 0           | 0          |       162.56 |
| information_schema |      36 | master      | online      | 0:91580     |         | 0           | 0          |         0.00 |
| memsql             |      39 | master      | online      | 0:2203      |         | 0           | 0          |         0.00 |
| sharding           |     717 | sync slave  | replicating | 0:32982     |         | 0           | 0          |         0.00 |
+--------------------+---------+-------------+-------------+-------------+---------+-------------+------------+--------------+
15 rows in set (0.00 sec)

@qix
Copy link
Author

qix commented Apr 29, 2015

The new table I created seems to be gone

memsql-leaf1> SHOW TABLES IN authbox_28;
+-----------------------+
| Tables_in_authbox_28  |
+-----------------------+
| feed_action           |
| feed_action_label     |
| pivot_action_ids      |
| pivot_entity_20150409 |
| pivot_entity_20150410 |
+-----------------------+
5 rows in set (0.00 sec)

memsql-leaf1> SHOW TABLES IN authbox_30;
+-----------------------+
| Tables_in_authbox_30  |
+-----------------------+
| pivot_action_ids      |
| pivot_entity_20150409 |
| pivot_entity_20150410 |
+-----------------------+
3 rows in set (0.00 sec)

@qix
Copy link
Author

qix commented Apr 29, 2015

The table seemed to be fine from SHOW PARTITIONS / SHOW EXTENDED STATUS, but was missing the new tables.

Issue was easily fixed by manually creating the tables on 30, but with loss of data I assume.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment