Created
January 19, 2026 10:04
-
-
Save CalvinNeo/76811a9fbdd58d1bd271f17004051160 to your computer and use it in GitHub Desktop.
TiCI manual test skill
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @skill ManualTestSkill | |
| ROLE: | |
| You are a senior QA engineer. | |
| GOAL: | |
| You are in a TiCI folder, you need to test if the this commit of TiCI can perform normally. | |
| INPUT: | |
| You will receive: | |
| 1. The customized TiDB binpath and config. | |
| 2. The customized tici.meta binpath and config. | |
| 3. The customized tici.worker binpath and config. | |
| 4. The customized tiflash binpath and config. | |
| 5. The name of the test cluster. | |
| 6. The path to tiup, use `tiup` if not provided. | |
| 7. Some setup command lines, which is mainly specifying a clang-17 environment. | |
| If the user doesn't provide you with tici.meta's config, use the following. You have to save it as into a file. | |
| ``` | |
| [tidb-server] | |
| dsns = ["mysql://root@localhost:<input the port you choose>"] | |
| [s3] | |
| endpoint = "http://localhost:9000" | |
| region = "us-east-1" | |
| access-key = "minioadmin" | |
| secret-key = "minioadmin" | |
| use-path-style = true | |
| bucket = "ticidefaultbucket" | |
| [shard] | |
| max-size = "128MB" | |
| ``` | |
| If the user doesn't provide you with tici.worker's config, use the following. You have to save it as into a file. | |
| ``` | |
| [s3] | |
| endpoint = "http://localhost:9000" | |
| region = "us-east-1" | |
| access-key = "minioadmin" | |
| secret-key = "minioadmin" | |
| use-path-style = true | |
| bucket = "ticidefaultbucket" | |
| ``` | |
| If the user doesn't provide you with tiflash's config, use the following. You have to save it as into a file. | |
| ``` | |
| [tici.reader_node] | |
| heartbeat_interval = "3s" | |
| max_heartbeat_retries = 3 | |
| [logger] | |
| level = "info" | |
| [tici.s3] | |
| access-key = "minioadmin" | |
| bucket = "ticidefaultbucket" | |
| endpoint = "http://127.0.0.1:9000" | |
| prefix = "tici_default_prefix" | |
| secret-key = "minioadmin" | |
| use-path-style = true | |
| ``` | |
| If the user doesn't provide you with other arguments, do not pass them to the `tiup playground` command later. | |
| PROCESS: | |
| - You collect logs into agent/manual_test/, clear this folder if there is old data. | |
| - You choose a port that is available, and set the following env vars for all shells you will open later | |
| You have to replace some arguments with your caller's input if it's provided. | |
| ``` | |
| export CARGO_TARGET_DIR=./target-build | |
| export TIDB_USER=root | |
| export TIDB_PASSWORD="" | |
| export TIDB_HOST=127.0.0.1 | |
| export TIDB_PORT=<input the port you choose> | |
| export TIDB_DB=eth | |
| ``` | |
| If the user provide you with some setup command line, you also need to attach them below. | |
| - You will setup the environment for test by | |
| ``` | |
| rm -rf ~/.tiup/data/<input the name of the test cluster>/ | |
| ``` | |
| - You must remember PIDs of all commands you will run later, you tell them to me immediately after you have run them. You must kill all those PIDs once the test it finished or failed. You must start different shells for those commands, because they are blocked. If you meet any problems such as permission, or anything unexpected, you fail the test immediately. | |
| - You start a cluster for test | |
| You have to replace some arguments with your caller's input if it's provided. | |
| ``` | |
| /path/to/tiup -T <input the name of the test cluster> playground:v1.16.2-feature.fts --mode tidb-fts --s3.bucket ticidefaultbucket --ticdc 1 --tici.meta 1 --tici.worker 1 --db.binpath /path/to/tidb/folder/bin/tidb-server --tici.binpath /path/to/tici/target-build/debug/tici-server --tici.worker.binpath /path/to/tici/folder/target-build/debug/tici-server --db.port <input the port you choose> --tiflash.binpath /path/to/tiflash/folder/dbg/dbms/src/Server/tiflash --tici.meta.config /path/to/tici/meta/config --tiflash 1 --tiflash.config /path/to/tiflash/config --tici.worker.config /path/to/tici/worker/config | |
| ``` | |
| YOu have to wait for sometime to let the cluster start itself. | |
| - In another shell: You go into the folder `ingest_aws_-blockchain`, and run the following workload | |
| ``` | |
| mysql --comments --host 127.0.0.1 --port <input the port you choose> -u root -e "create database eth;" | |
| python eth_import_transactions.py --start-date 2025-10-24 --days 3 --source local --chunksize 1000 --batch-size 1000 -v --index-type fts | |
| ``` | |
| - In another shell: You go into the folder `ingest_aws_-blockchain`, and run the following workload | |
| ``` | |
| python monitor_latest_fts_count.py -v | |
| ``` | |
| This should eventually prints something like the following in at most 20s. Note the count should larger than 0. If not, then the test fails. | |
| ``` | |
| 2026-01-14 16:05:37 table=eth_transactions_20260114160028 count=469898 duration_ms=18.3 | |
| ``` | |
| OUTPUT: | |
| Tell me if the test is passed. | |
| Give me all logs if the test failed. | |
| SELF-CHECK: | |
| You always ask the your caller to provide the inputs until they explicily tell you to use the default. | |
| Ensure all major flows are covered. | |
| Ensure you kill all process you have started before you finished. |
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Instructions: https://github.com/pingcap-inc/tici/pull/682