Wakame-vdc / OpenVnet Advent Calendar 2014 12月22日担当です。
今回はWakame-vdcのCIで使用されているテストについてご紹介します。
Wakame-vdcのCIの概要はこちら
Wakame-vdcではmusselというbashで書かれたWebAPIClientを使用してテストを行っています。
musselで実装されているテストの種類は以下になります。
- unit test : musselの単体テスト
- component test : Wakame-vdcへ渡すパラメータのテスト
- integration test: Wakame-vdcのWebAPI単体テスト
- acceptance test : Wakame-vdcのWebAPIを複数組み合わせたシナリオテスト
- experiment test : Wakame-vdcのWebAPIとサードパーティプログラムとの連携テスト
このうちintegration testとacceptance testがWakame-vdcのCIで使用されています。
今回はサンプルとしてacceptanceテストのt.instance.secg.icmp.shの中身をご紹介します。
$ cat -n t.instance.secg.icmp.sh
1 #!/bin/bash
2 #
3 # requires:
4 # bash
5 # cat, ssh-keygen, ping, rm
6 #
7
8 ## include files
9
10 . ${BASH_SOURCE[0]%/*}/helper_shunit2.sh
11 . ${BASH_SOURCE[0]%/*}/helper_instance.sh
12
13 ## variables
14
15 ## functions
16
17 function render_secg_rule() {
18 cat <<-EOS
19 icmp:-1,-1,ip4:0.0.0.0/0
20 EOS
21 }
22
23 function after_create_instance() {
24 instance_ipaddr=$(run_cmd instance show ${instance_uuid} | hash_value address)
25 wait_for_network_to_be_ready ${instance_ipaddr}
26 }
27
28 ### step
29
30 function test_drop_icmp() {
31 cat <<-EOS > ${rule_path}
32 #
33 EOS
34 run_cmd security_group update ${security_group_uuid}
35 sleep 3
36
37 wait_for_network_not_to_be_ready ${instance_ipaddr}
38 assertEquals 0 $?
39 }
40
41 function test_accept_icmp() {
42 render_secg_rule > ${rule_path}
43 run_cmd security_group update ${security_group_uuid}
44 sleep 3
45
46 wait_for_network_to_be_ready ${instance_ipaddr}
47 assertEquals 0 $?
48 }
49
50 ## shunit2
51
52 . ${shunit2_file}
このテストの内容以下の通りです。
- security groupでicmpの疎通が許可されたインスタンスを起動する。
- security groupのルールを変更しpingが通らないことを確認する。
- security groupのルールを変更しpingが通ることを確認する。
- インスタンスを削除する。
またインスタンスの起動や削除などテスト毎に使用するようなものはhelpersにて共通化されています。
$ ls -la
total 38
drwxrwxr-x+ 1 itune なし 0 Oct 7 16:59 .
drwxrwxr-x+ 1 itune なし 0 Jul 24 14:49 ..
-rw-rw-r-- 1 itune なし 459 May 19 2014 alarm.sh
-rw-rw-r-- 1 itune なし 2388 May 19 2014 document.sh
-rw-rw-r-- 1 itune なし 606 Oct 7 16:59 dolphin.sh
-rw-rw-r-- 1 itune なし 847 May 19 2014 external_ip.sh
-rw-rw-r-- 1 itune なし 4366 May 19 2014 instance.sh
-rw-rw-r-- 1 itune なし 209 May 19 2014 interactive.sh
-rw-rw-r-- 1 itune なし 1640 May 19 2014 load_balancer.sh
-rw-rw-r-- 1 itune なし 1965 Jul 8 14:57 retry.sh
-rw-rw-r-- 1 itune なし 558 May 19 2014 ssh.sh
-rw-rw-r-- 1 itune なし 726 May 19 2014 ssl.sh
-rw-rw-r-- 1 itune なし 316 May 19 2014 vars.sh
共通化したことにより、新機能実装時にもテストコードを書く時間がある程度短縮される様になりました。
Wakame-vdcではgithubにコードをコミットした時、毎日深夜の定期実行にてテストが継続的に実施されています。
これにより新機能の動作確認、既存機能への影響などがいち早くわかるようになりコードの品質管理にも役立っています。
またテストは自動実行されるためエンジニアの時間の使い方にも変化がありました。
日々テストが追加され今ではこんなに多くなりました。
$ git ls-files
Makefile
README.md
dolphin.d/base.sh
dolphin.d/event.sh
dolphin.d/notification.sh
functions
functions_dolphin
misc/fake-dcmgr-daemon.sh
mussel.sh
mussel_dolphin.sh
test/Makefile
test/acceptance-vnet/v12.03/helper_shunit2.sh
test/acceptance-vnet/v12.03/nat/Makefile
test/acceptance-vnet/v12.03/nat/helper_instance.sh
test/acceptance-vnet/v12.03/nat/helper_shunit2.sh
test/acceptance-vnet/v12.03/nat/t.external-ip.sh
test/acceptance/Makefile
test/acceptance/v12.03/1shot/Makefile
test/acceptance/v12.03/1shot/helper_instance.sh
test/acceptance/v12.03/1shot/helper_load_balancer.sh
test/acceptance/v12.03/1shot/helper_shunit2.sh
test/acceptance/v12.03/1shot/t.instance.backup.sh
test/acceptance/v12.03/1shot/t.instance.base.sh
test/acceptance/v12.03/1shot/t.instance.lbnode.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.force-poweroff-halting-instance.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.guest-shutdown.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.reboot-icmp.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.strict.sh
test/acceptance/v12.03/1shot/t.instance.secg.icmp.sh
test/acceptance/v12.03/1shot/t.instance.secg.tcp.sh
test/acceptance/v12.03/1shot/t.load_balancer.sh
test/acceptance/v12.03/1shot/t.volume.backup.sh
test/acceptance/v12.03/Makefile
test/acceptance/v12.03/alarm/Makefile
test/acceptance/v12.03/alarm/helper_shunit2.sh
test/acceptance/v12.03/alarm/t.log.alarm.lifecycle.sh
test/acceptance/v12.03/alarm/t.log.alarm.validation.sh
test/acceptance/v12.03/helper_shunit2.sh
test/acceptance/v12.03/instance-clone/Makefile
test/acceptance/v12.03/instance-clone/helper_instance.sh
test/acceptance/v12.03/instance-clone/helper_shunit2.sh
test/acceptance/v12.03/instance-clone/t.image.lifecycle.sh
test/acceptance/v12.03/lb-cluster/Makefile
test/acceptance/v12.03/lb-cluster/helper_instance.sh
test/acceptance/v12.03/lb-cluster/helper_load_balancer.sh
test/acceptance/v12.03/lb-cluster/helper_load_balancer_with_registered_instances.sh
test/acceptance/v12.03/lb-cluster/helper_shunit2.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.single.sh
test/acceptance/v12.03/local-volume-instance/Makefile
test/acceptance/v12.03/local-volume-instance/helper_instance.sh
test/acceptance/v12.03/local-volume-instance/helper_shunit2.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.base.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.big-vol.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.disk-full.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.mount.sh
test/acceptance/v12.03/natbox/Makefile
test/acceptance/v12.03/natbox/helper_instance.sh
test/acceptance/v12.03/natbox/helper_shunit2.sh
test/acceptance/v12.03/natbox/t.instance.network_vif.external-ip.lifecycle.existing-inst.sh
test/acceptance/v12.03/natbox/t.instance.network_vif.external-ip.lifecycle.sh
test/acceptance/v12.03/sg-cluster/Makefile
test/acceptance/v12.03/sg-cluster/helper_instance.sh
test/acceptance/v12.03/sg-cluster/helper_load_balancer.sh
test/acceptance/v12.03/sg-cluster/helper_shunit2.sh
test/acceptance/v12.03/sg-cluster/t.instance.secg.known-friends.sh
test/acceptance/v12.03/sg-cluster/t.instance.secg.unknown-friends.sh
test/acceptance/v12.03/sg-cluster/t.lb.secg.unknown-friends.sh
test/acceptance/v12.03/shared-volume-instance-migration/Makefile
test/acceptance/v12.03/shared-volume-instance-migration/helper_instance.sh
test/acceptance/v12.03/shared-volume-instance-migration/helper_shunit2.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.base.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.different.host.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.same.host.sh
test/acceptance/v12.03/shared-volume-instance/Makefile
test/acceptance/v12.03/shared-volume-instance/helper_instance.sh
test/acceptance/v12.03/shared-volume-instance/helper_shunit2.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.backup.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.base.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.lifecycle.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.mount.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.mount.with.instance.sh
test/component/Makefile
test/component/functions/Makefile
test/component/functions/helper_shunit2.sh
test/component/functions/t.call_api.sh
test/component/functions/t.cmd_destroy.sh
test/component/functions/t.cmd_index.sh
test/component/functions/t.cmd_show.sh
test/component/functions/t.cmd_xcreate.sh
test/component/functions/t.cmd_xget.sh
test/component/v12.03/Makefile
test/component/v12.03/helper_shunit2.sh
test/component/v12.03/t.alarm.sh
test/component/v12.03/t.backup_object.sh
test/component/v12.03/t.backup_storage.sh
test/component/v12.03/t.base.sh
test/component/v12.03/t.dc_network.sh
test/component/v12.03/t.host_node.sh
test/component/v12.03/t.image.sh
test/component/v12.03/t.instance.sh
test/component/v12.03/t.instance_monitoring.sh
test/component/v12.03/t.ip_handle.sh
test/component/v12.03/t.ip_pool.sh
test/component/v12.03/t.load_balancer.sh
test/component/v12.03/t.network.sh
test/component/v12.03/t.network_vif.sh
test/component/v12.03/t.network_vif_monitor.sh
test/component/v12.03/t.security_group.sh
test/component/v12.03/t.ssh_key_pair.sh
test/component/v12.03/t.storage_node.sh
test/component/v12.03/t.volume.sh
test/experiment/Makefile
test/experiment/v12.03/Makefile
test/experiment/v12.03/alarm/helper_alarm.sh
test/experiment/v12.03/alarm/helper_shunit2.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.1.instance.max.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.1.instance.min.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.10.instance.max.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.10.instance.min.sh
test/experiment/v12.03/alarm/tag.list
test/experiment/v12.03/dolphin-ha/helper_dolphin.sh
test/experiment/v12.03/dolphin-ha/helper_instance.sh
test/experiment/v12.03/dolphin-ha/helper_log_alarm.sh
test/experiment/v12.03/dolphin-ha/helper_shunit2.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.event.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.instance.create.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.instance.destroy.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.log.alarm.setup.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.log.alarm.verify.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.notification.sh
test/experiment/v12.03/ha/helper_shunit2.sh
test/experiment/v12.03/ha/t.instance.move.sh
test/experiment/v12.03/ha/vmware-ws.sh
test/experiment/v12.03/helper_shunit2.sh
test/experiment/v12.03/sg-cluster/Makefile
test/experiment/v12.03/sg-cluster/helper_instance.sh
test/experiment/v12.03/sg-cluster/helper_shunit2.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.complex.1.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete2.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete3.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete4.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.multiple-groups.sh
test/experiment/v12.03/zabbix/helper_instance.sh
test/experiment/v12.03/zabbix/helper_shunit2.sh
test/experiment/v12.03/zabbix/helper_zabbix.sh
test/experiment/v12.03/zabbix/t.instance.monitoring.1shot.sh
test/helpers/alarm.sh
test/helpers/document.sh
test/helpers/dolphin.sh
test/helpers/external_ip.sh
test/helpers/instance.sh
test/helpers/interactive.sh
test/helpers/load_balancer.sh
test/helpers/retry.sh
test/helpers/ssh.sh
test/helpers/ssl.sh
test/helpers/vars.sh
test/integration/Makefile
test/integration/v12.03/Makefile
test/integration/v12.03/helper_shunit2.sh
test/integration/v12.03/ro/Makefile
test/integration/v12.03/ro/helper_shunit2.sh
test/integration/v12.03/ro/t.alarm.sh
test/integration/v12.03/ro/t.backup_object.sh
test/integration/v12.03/ro/t.backup_storage.sh
test/integration/v12.03/ro/t.dc_network.sh
test/integration/v12.03/ro/t.host_node.sh
test/integration/v12.03/ro/t.image.sh
test/integration/v12.03/ro/t.instance.sh
test/integration/v12.03/ro/t.ip_handle.sh
test/integration/v12.03/ro/t.ip_pool.sh
test/integration/v12.03/ro/t.load_balancer.sh
test/integration/v12.03/ro/t.network.sh
test/integration/v12.03/ro/t.network_vif.sh
test/integration/v12.03/ro/t.security_group.sh
test/integration/v12.03/ro/t.ssh_key_pair.sh
test/integration/v12.03/ro/t.storage_node.sh
test/integration/v12.03/ro/t.volume.sh
test/integration/v12.03/rw/Makefile
test/integration/v12.03/rw/helper_instance.sh
test/integration/v12.03/rw/helper_load_balancer.sh
test/integration/v12.03/rw/helper_shunit2.sh
test/integration/v12.03/rw/t.instance.lifecycle.base.sh
test/integration/v12.03/rw/t.instance.lifecycle.poweroff.sh
test/integration/v12.03/rw/t.instance.vifs.networking.running-out-of-ip.sh
test/integration/v12.03/rw/t.instance.vifs.null.sh
test/integration/v12.03/rw/t.instance.vifs.security_group.non-existent.sh
test/integration/v12.03/rw/t.ip_handle.sh
test/integration/v12.03/rw/t.ip_pool.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.multi-cidr.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.multi-ip.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.single-cidr.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.single-ip.sh
test/integration/v12.03/rw/t.load_balancer.balance-algorithm.sh
test/integration/v12.03/rw/t.load_balancer.httpchk.sh
test/integration/v12.03/rw/t.load_balancer.lifecycle.base.sh
test/integration/v12.03/rw/t.load_balancer.multiple-listen.sh
test/integration/v12.03/rw/t.load_balancer.ssl-cert.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.multi-cidr.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.multi-ip.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.single-cidr.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.single-ip.sh
test/integration/v12.03/rw/t.load_balancer.update.balance-algorithm.sh
test/integration/v12.03/rw/t.load_balancer.update.httpchk.sh
test/integration/v12.03/rw/t.load_balancer.update.multiple-listen.sh
test/integration/v12.03/rw/t.load_balancer.vifs.sh
test/integration/v12.03/rw/t.log.alarm.sh
test/integration/v12.03/rw/t.security_group.nesting.sh
test/integration/v12.03/rw/t.security_group.self-referred.sh
test/integration/v12.03/rw/t.security_group.sh
test/integration/v12.03/rw/t.ssh_key_pair.sh
test/integration/v12.03/rwx/Makefile
test/integration/v12.03/rwx/helper_instance.sh
test/integration/v12.03/rwx/helper_instance_vifs_multi.sh
test/integration/v12.03/rwx/helper_instance_vifs_single.sh
test/integration/v12.03/rwx/helper_load_balancer.sh
test/integration/v12.03/rwx/helper_shunit2.sh
test/integration/v12.03/rwx/t.instance.lifecycle.poweroff-poweron.sh
test/integration/v12.03/rwx/t.instance.lifecycle.reboot.sh
test/integration/v12.03/rwx/t.instance.lifecycle.stop-start.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.networking.non-secg.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.networking.secg.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.null-net.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.external-ip.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.non-secg.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.secg.sh
test/integration/v12.03/rwx/t.instance.vifs.single.null-net.sh
test/integration/v12.03/rwx/t.instance_monitoring.process.sh
test/integration/v12.03/rwx/t.load_balancer.external_ip.sh
test/integration/v12.03/rwx/t.load_balancer.lifecycle.poweroff-poweron.sh
test/integration/v12.03/rwx/t.log.alarm.sh
test/integration/v12.03/rwx/t.network_vif_monitor.port.sh
test/shunit2
test/unit/Makefile
test/unit/functions/Makefile
test/unit/functions/helper_shunit2.sh
test/unit/functions/t.add_args_param.sh
test/unit/functions/t.add_param.sh
test/unit/functions/t.base_uri.sh
test/unit/functions/t.call_api.sh
test/unit/functions/t.cmd_default.sh
test/unit/functions/t.cmd_destroy.sh
test/unit/functions/t.cmd_help.sh
test/unit/functions/t.cmd_index.sh
test/unit/functions/t.cmd_put.sh
test/unit/functions/t.cmd_show.sh
test/unit/functions/t.cmd_xcreate.sh
test/unit/functions/t.cmd_xget.sh
test/unit/functions/t.curl_opts.sh
test/unit/functions/t.extract_args.sh
test/unit/functions/t.invoke_task.sh
test/unit/functions/t.load_musselrc.sh
test/unit/functions/t.request_header.sh
test/unit/functions/t.request_param.sh
test/unit/functions/t.run_cmd.sh
test/unit/functions/t.shlog.sh
test/unit/functions/t.strfile_type.sh
test/unit/functions/t.suffix.sh
test/unit/functions/t.urlencode_data.sh
test/unit/v12.03/Makefile
test/unit/v12.03/helper_shunit2.sh
test/unit/v12.03/t.alarm.sh
test/unit/v12.03/t.backup_object.sh
test/unit/v12.03/t.backup_storage.sh
test/unit/v12.03/t.base.sh
test/unit/v12.03/t.dc_network.sh
test/unit/v12.03/t.host_node.sh
test/unit/v12.03/t.image.sh
test/unit/v12.03/t.instance.sh
test/unit/v12.03/t.instance_monitoring.sh
test/unit/v12.03/t.ip_handle.sh
test/unit/v12.03/t.ip_pool.sh
test/unit/v12.03/t.load_balancer.sh
test/unit/v12.03/t.network.sh
test/unit/v12.03/t.network_vif.sh
test/unit/v12.03/t.network_vif_monitor.sh
test/unit/v12.03/t.security_group.sh
test/unit/v12.03/t.ssh_key_pair.sh
test/unit/v12.03/t.storage_node.sh
test/unit/v12.03/t.undefined.sh
test/unit/v12.03/t.volume.sh
v12.03.d/alarm.sh
v12.03.d/backup_object.sh
v12.03.d/backup_storage.sh
v12.03.d/base.sh
v12.03.d/dc_network.sh
v12.03.d/host_node.sh
v12.03.d/image.sh
v12.03.d/instance.sh
v12.03.d/instance_monitoring.sh
v12.03.d/ip_handle.sh
v12.03.d/ip_pool.sh
v12.03.d/load_balancer.sh
v12.03.d/network.sh
v12.03.d/network_vif.sh
v12.03.d/network_vif_monitor.sh
v12.03.d/security_group.sh
v12.03.d/ssh_key_pair.sh
v12.03.d/storage_node.sh
v12.03.d/volume.sh