Skip to content

Instantly share code, notes, and snippets.

View mattlord's full-sized avatar
😶‍🌫️

Matt Lord mattlord

😶‍🌫️
View GitHub Profile
@mattlord
mattlord / vstream_vevent_enum_test.md
Last active December 6, 2023 23:05
VStream VEvent Test of ENUMs

Setup the test env:

git checkout main && make build

pushd examples/local

./101_initial_cluster.sh

mysql < ../common/insert_commerce_data.sql
@mattlord
mattlord / demo.sh
Last active November 4, 2023 15:31
KubeCon NA 2023 Vitess Data Migration Demo
#!/bin/bash
set -o pipefail
export SKIP_VTADMIN=1
function display () {
echo -e "\033[33m${1}\033[0m"
}
@mattlord
mattlord / proto_examples.txt
Last active January 25, 2024 17:34
vtctlclient and etcdctl examples
❯ vtctldclient GetTablets
zone1-0000000100 commerce 0 replica localhost:15100 localhost:17100 [] <null>
zone1-0000000101 commerce 0 primary localhost:15101 localhost:17101 [] 2023-09-28T16:37:49Z
zone1-0000000102 commerce 0 rdonly localhost:15102 localhost:17102 [] <null>
zone1-0000000200 customer 0 primary localhost:15200 localhost:17200 [] 2023-09-28T16:39:19Z
zone1-0000000201 customer 0 replica localhost:15201 localhost:17201 [] <null>
zone1-0000000202 customer 0 rdonly localhost:15202 localhost:17202 [] <null>
❯ etcdctl get --prefix /vitess/zone1/tablets/zone1-0000000100/Tablet -w protobuf | protoc --proto_path=${HOME}/git/vitess/proto --decode=topodata.Keyspace topodata.proto
1 {
@mattlord
mattlord / grpc_examples.md
Last active August 30, 2023 03:58
grpcurl examples with Vitess
grpcurl -plaintext -format json -import-path proto -proto proto/vtgateservice.proto \
  -H 'username: guser1'  -H 'password: gpass1' \
  -d '{"tablet_type":"RDONLY", "vgtid":{ "shard_gtids": [{"keyspace":"", "shard":"", "gtid":"current"}]} }' \
  localhost:15306 vtgateservice.Vitess/VStream
❯ grpcurl -plaintext -format json -import-path ${VTROOT}/proto -proto ${VTROOT}/proto/vtctlservice.proto \
 -d '{"keyspace":"commerce", "sql":"create table t1 (id int)", "ddl_strategy":"direct"}' \
@mattlord
mattlord / buffer_test.md
Last active January 25, 2024 17:34
buffering test

Test Case:

export SKIP_VTADMIN=1
pushd ${VTROOT}/examples/local

sed -i "" '42i\
  --enable_buffer \\
' ../common/scripts/vtgate-up.sh

sed -i "" -e "s/semi_sync/none/g" 101_initial_cluster.sh
#!/bin/bash
#
# From within ./examples/local/
#
pushd "${VTROOT}/examples/local"
source ../common/env.sh
CELL=zone1 ../common/scripts/etcd-up.sh

Command

You pass vtcombo a combination of flags that you would normally pass to the individual components (vtgate, vtctld, vttablet). For example:

vtcombo -logtostderr=true -proto_topo "$(cat ./topology)" -schema_dir ./vschema -mysql_server_port 15306 -mysql_server_bind_address 0.0.0.0 -mysql_auth_server_impl none -db_socket /tmp/mysql.sock -db_host localhost -mycnf-file /etc/mysql/my.cnf -db_app_user root -db_allprivs_user root -db_appdebug_user root -db_dba_user root -db_repl_user root -dbddl_plugin vttest -port 15000 -grpc_port 15001 -service_map 'grpc-vtgateservice,grpc-vtctl,grpc-vtctld' -db_charset utf8mb4 -vschema_ddl_authorized_users='%'

Example Topology File (-proto_topo)

keyspaces {
$ cat /tmp/sites.sql
CREATE TABLE sites(
  site_id bigint,
  name varchar(128),
  primary key(site_id)
);

$ cat /tmp/sites.vschema
{

For VReplication streams, we must choose a tablet to serve the role of vstreamer (source) and vapplier (target) in the replication stream.

Key context for the tablet picker: cells, keyspace, shard, and tablet types (primary/master, replica).

ℹ️ VReplication will only look for tablet pairings within the same cell. If you want to have cross-cell streams then you will need to create a CellAlias that contains the list of potential cells and specify that using the -cell flag in your VReplication workflow commands.

The tablet picker logic for is here. It gets a set of valid candidates for the source and target of the stream:

  • Source (vstreamer): it uses the random ta
mysql> SELECT @@version;
+-----------+
| @@version |
+-----------+
| 8.0.25 |
+-----------+
1 row in set (0.00 sec)
mysql> SELECT UUID() INTO @myuuid;