Create a gist now

Instantly share code, notes, and snippets.

Wakame-vdcで使用されているテストについて

Wakame-vdcで使用されているテストについて

Wakame-vdc / OpenVnet Advent Calendar 2014 12月22日担当です。

今回はWakame-vdcのCIで使用されているテストについてご紹介します。

Wakame-vdcのCIの概要はこちら

Wakame-vdcではmusselというbashで書かれたWebAPIClientを使用してテストを行っています。

musselで実装されているテストの種類

musselで実装されているテストの種類は以下になります。

  • unit test : musselの単体テスト
  • component test : Wakame-vdcへ渡すパラメータのテスト
  • integration test: Wakame-vdcのWebAPI単体テスト
  • acceptance test : Wakame-vdcのWebAPIを複数組み合わせたシナリオテスト
  • experiment test : Wakame-vdcのWebAPIとサードパーティプログラムとの連携テスト

このうちintegration testとacceptance testがWakame-vdcのCIで使用されています。

Wakame-vdcで使用されているテストの一部の紹介

今回はサンプルとしてacceptanceテストのt.instance.secg.icmp.shの中身をご紹介します。

$ cat -n t.instance.secg.icmp.sh
     1  #!/bin/bash
     2  #
     3  # requires:
     4  #  bash
     5  #  cat, ssh-keygen, ping, rm
     6  #
     7
     8  ## include files
     9
    10  . ${BASH_SOURCE[0]%/*}/helper_shunit2.sh
    11  . ${BASH_SOURCE[0]%/*}/helper_instance.sh
    12
    13  ## variables
    14
    15  ## functions
    16
    17  function render_secg_rule() {
    18    cat <<-EOS
    19          icmp:-1,-1,ip4:0.0.0.0/0
    20          EOS
    21  }
    22
    23  function after_create_instance() {
    24    instance_ipaddr=$(run_cmd instance show ${instance_uuid} | hash_value address)
    25    wait_for_network_to_be_ready ${instance_ipaddr}
    26  }
    27
    28  ### step
    29
    30  function test_drop_icmp() {
    31    cat <<-EOS > ${rule_path}
    32          #
    33          EOS
    34    run_cmd security_group update ${security_group_uuid}
    35    sleep 3
    36
    37    wait_for_network_not_to_be_ready ${instance_ipaddr}
    38    assertEquals 0 $?
    39  }
    40
    41  function test_accept_icmp() {
    42    render_secg_rule > ${rule_path}
    43    run_cmd security_group update ${security_group_uuid}
    44    sleep 3
    45
    46    wait_for_network_to_be_ready ${instance_ipaddr}
    47    assertEquals 0 $?
    48  }
    49
    50  ## shunit2
    51
    52  . ${shunit2_file}

このテストの内容以下の通りです。

  1. security groupでicmpの疎通が許可されたインスタンスを起動する。
  2. security groupのルールを変更しpingが通らないことを確認する。
  3. security groupのルールを変更しpingが通ることを確認する。
  4. インスタンスを削除する。

またインスタンスの起動や削除などテスト毎に使用するようなものはhelpersにて共通化されています。

$ ls -la
total 38
drwxrwxr-x+ 1 itune なし    0 Oct  7 16:59 .
drwxrwxr-x+ 1 itune なし    0 Jul 24 14:49 ..
-rw-rw-r--  1 itune なし  459 May 19  2014 alarm.sh
-rw-rw-r--  1 itune なし 2388 May 19  2014 document.sh
-rw-rw-r--  1 itune なし  606 Oct  7 16:59 dolphin.sh
-rw-rw-r--  1 itune なし  847 May 19  2014 external_ip.sh
-rw-rw-r--  1 itune なし 4366 May 19  2014 instance.sh
-rw-rw-r--  1 itune なし  209 May 19  2014 interactive.sh
-rw-rw-r--  1 itune なし 1640 May 19  2014 load_balancer.sh
-rw-rw-r--  1 itune なし 1965 Jul  8 14:57 retry.sh
-rw-rw-r--  1 itune なし  558 May 19  2014 ssh.sh
-rw-rw-r--  1 itune なし  726 May 19  2014 ssl.sh
-rw-rw-r--  1 itune なし  316 May 19  2014 vars.sh

共通化したことにより、新機能実装時にもテストコードを書く時間がある程度短縮される様になりました。

テストを書いてよかったこと

Wakame-vdcではgithubにコードをコミットした時、毎日深夜の定期実行にてテストが継続的に実施されています。

これにより新機能の動作確認、既存機能への影響などがいち早くわかるようになりコードの品質管理にも役立っています。

またテストは自動実行されるためエンジニアの時間の使い方にも変化がありました。

現在のテストケース一覧

日々テストが追加され今ではこんなに多くなりました。

$ git ls-files
Makefile
README.md
dolphin.d/base.sh
dolphin.d/event.sh
dolphin.d/notification.sh
functions
functions_dolphin
misc/fake-dcmgr-daemon.sh
mussel.sh
mussel_dolphin.sh
test/Makefile
test/acceptance-vnet/v12.03/helper_shunit2.sh
test/acceptance-vnet/v12.03/nat/Makefile
test/acceptance-vnet/v12.03/nat/helper_instance.sh
test/acceptance-vnet/v12.03/nat/helper_shunit2.sh
test/acceptance-vnet/v12.03/nat/t.external-ip.sh
test/acceptance/Makefile
test/acceptance/v12.03/1shot/Makefile
test/acceptance/v12.03/1shot/helper_instance.sh
test/acceptance/v12.03/1shot/helper_load_balancer.sh
test/acceptance/v12.03/1shot/helper_shunit2.sh
test/acceptance/v12.03/1shot/t.instance.backup.sh
test/acceptance/v12.03/1shot/t.instance.base.sh
test/acceptance/v12.03/1shot/t.instance.lbnode.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.force-poweroff-halting-instance.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.guest-shutdown.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.reboot-icmp.sh
test/acceptance/v12.03/1shot/t.instance.lifecycle.strict.sh
test/acceptance/v12.03/1shot/t.instance.secg.icmp.sh
test/acceptance/v12.03/1shot/t.instance.secg.tcp.sh
test/acceptance/v12.03/1shot/t.load_balancer.sh
test/acceptance/v12.03/1shot/t.volume.backup.sh
test/acceptance/v12.03/Makefile
test/acceptance/v12.03/alarm/Makefile
test/acceptance/v12.03/alarm/helper_shunit2.sh
test/acceptance/v12.03/alarm/t.log.alarm.lifecycle.sh
test/acceptance/v12.03/alarm/t.log.alarm.validation.sh
test/acceptance/v12.03/helper_shunit2.sh
test/acceptance/v12.03/instance-clone/Makefile
test/acceptance/v12.03/instance-clone/helper_instance.sh
test/acceptance/v12.03/instance-clone/helper_shunit2.sh
test/acceptance/v12.03/instance-clone/t.image.lifecycle.sh
test/acceptance/v12.03/lb-cluster/Makefile
test/acceptance/v12.03/lb-cluster/helper_instance.sh
test/acceptance/v12.03/lb-cluster/helper_load_balancer.sh
test/acceptance/v12.03/lb-cluster/helper_load_balancer_with_registered_instances.sh
test/acceptance/v12.03/lb-cluster/helper_shunit2.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.http.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.https.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.ssl.node.single.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.multi.balancing.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.multi.sh
test/acceptance/v12.03/lb-cluster/t.lb.tcp.node.single.sh
test/acceptance/v12.03/local-volume-instance/Makefile
test/acceptance/v12.03/local-volume-instance/helper_instance.sh
test/acceptance/v12.03/local-volume-instance/helper_shunit2.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.base.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.big-vol.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.disk-full.sh
test/acceptance/v12.03/local-volume-instance/t.volume.local.mount.sh
test/acceptance/v12.03/natbox/Makefile
test/acceptance/v12.03/natbox/helper_instance.sh
test/acceptance/v12.03/natbox/helper_shunit2.sh
test/acceptance/v12.03/natbox/t.instance.network_vif.external-ip.lifecycle.existing-inst.sh
test/acceptance/v12.03/natbox/t.instance.network_vif.external-ip.lifecycle.sh
test/acceptance/v12.03/sg-cluster/Makefile
test/acceptance/v12.03/sg-cluster/helper_instance.sh
test/acceptance/v12.03/sg-cluster/helper_load_balancer.sh
test/acceptance/v12.03/sg-cluster/helper_shunit2.sh
test/acceptance/v12.03/sg-cluster/t.instance.secg.known-friends.sh
test/acceptance/v12.03/sg-cluster/t.instance.secg.unknown-friends.sh
test/acceptance/v12.03/sg-cluster/t.lb.secg.unknown-friends.sh
test/acceptance/v12.03/shared-volume-instance-migration/Makefile
test/acceptance/v12.03/shared-volume-instance-migration/helper_instance.sh
test/acceptance/v12.03/shared-volume-instance-migration/helper_shunit2.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.base.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.different.host.sh
test/acceptance/v12.03/shared-volume-instance-migration/t.shared.volume.instance.migration.same.host.sh
test/acceptance/v12.03/shared-volume-instance/Makefile
test/acceptance/v12.03/shared-volume-instance/helper_instance.sh
test/acceptance/v12.03/shared-volume-instance/helper_shunit2.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.backup.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.base.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.lifecycle.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.mount.sh
test/acceptance/v12.03/shared-volume-instance/t.volume.shared.mount.with.instance.sh
test/component/Makefile
test/component/functions/Makefile
test/component/functions/helper_shunit2.sh
test/component/functions/t.call_api.sh
test/component/functions/t.cmd_destroy.sh
test/component/functions/t.cmd_index.sh
test/component/functions/t.cmd_show.sh
test/component/functions/t.cmd_xcreate.sh
test/component/functions/t.cmd_xget.sh
test/component/v12.03/Makefile
test/component/v12.03/helper_shunit2.sh
test/component/v12.03/t.alarm.sh
test/component/v12.03/t.backup_object.sh
test/component/v12.03/t.backup_storage.sh
test/component/v12.03/t.base.sh
test/component/v12.03/t.dc_network.sh
test/component/v12.03/t.host_node.sh
test/component/v12.03/t.image.sh
test/component/v12.03/t.instance.sh
test/component/v12.03/t.instance_monitoring.sh
test/component/v12.03/t.ip_handle.sh
test/component/v12.03/t.ip_pool.sh
test/component/v12.03/t.load_balancer.sh
test/component/v12.03/t.network.sh
test/component/v12.03/t.network_vif.sh
test/component/v12.03/t.network_vif_monitor.sh
test/component/v12.03/t.security_group.sh
test/component/v12.03/t.ssh_key_pair.sh
test/component/v12.03/t.storage_node.sh
test/component/v12.03/t.volume.sh
test/experiment/Makefile
test/experiment/v12.03/Makefile
test/experiment/v12.03/alarm/helper_alarm.sh
test/experiment/v12.03/alarm/helper_shunit2.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.1.instance.max.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.1.instance.min.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.10.instance.max.sh
test/experiment/v12.03/alarm/t.log.alarm.stress.10.instance.min.sh
test/experiment/v12.03/alarm/tag.list
test/experiment/v12.03/dolphin-ha/helper_dolphin.sh
test/experiment/v12.03/dolphin-ha/helper_instance.sh
test/experiment/v12.03/dolphin-ha/helper_log_alarm.sh
test/experiment/v12.03/dolphin-ha/helper_shunit2.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.event.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.instance.create.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.instance.destroy.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.log.alarm.setup.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.log.alarm.verify.sh
test/experiment/v12.03/dolphin-ha/t.dolphin.notification.sh
test/experiment/v12.03/ha/helper_shunit2.sh
test/experiment/v12.03/ha/t.instance.move.sh
test/experiment/v12.03/ha/vmware-ws.sh
test/experiment/v12.03/helper_shunit2.sh
test/experiment/v12.03/sg-cluster/Makefile
test/experiment/v12.03/sg-cluster/helper_instance.sh
test/experiment/v12.03/sg-cluster/helper_shunit2.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.complex.1.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete2.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete3.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.instances-seg-add-delete4.sh
test/experiment/v12.03/sg-cluster/t.instance.secg.multiple-groups.sh
test/experiment/v12.03/zabbix/helper_instance.sh
test/experiment/v12.03/zabbix/helper_shunit2.sh
test/experiment/v12.03/zabbix/helper_zabbix.sh
test/experiment/v12.03/zabbix/t.instance.monitoring.1shot.sh
test/helpers/alarm.sh
test/helpers/document.sh
test/helpers/dolphin.sh
test/helpers/external_ip.sh
test/helpers/instance.sh
test/helpers/interactive.sh
test/helpers/load_balancer.sh
test/helpers/retry.sh
test/helpers/ssh.sh
test/helpers/ssl.sh
test/helpers/vars.sh
test/integration/Makefile
test/integration/v12.03/Makefile
test/integration/v12.03/helper_shunit2.sh
test/integration/v12.03/ro/Makefile
test/integration/v12.03/ro/helper_shunit2.sh
test/integration/v12.03/ro/t.alarm.sh
test/integration/v12.03/ro/t.backup_object.sh
test/integration/v12.03/ro/t.backup_storage.sh
test/integration/v12.03/ro/t.dc_network.sh
test/integration/v12.03/ro/t.host_node.sh
test/integration/v12.03/ro/t.image.sh
test/integration/v12.03/ro/t.instance.sh
test/integration/v12.03/ro/t.ip_handle.sh
test/integration/v12.03/ro/t.ip_pool.sh
test/integration/v12.03/ro/t.load_balancer.sh
test/integration/v12.03/ro/t.network.sh
test/integration/v12.03/ro/t.network_vif.sh
test/integration/v12.03/ro/t.security_group.sh
test/integration/v12.03/ro/t.ssh_key_pair.sh
test/integration/v12.03/ro/t.storage_node.sh
test/integration/v12.03/ro/t.volume.sh
test/integration/v12.03/rw/Makefile
test/integration/v12.03/rw/helper_instance.sh
test/integration/v12.03/rw/helper_load_balancer.sh
test/integration/v12.03/rw/helper_shunit2.sh
test/integration/v12.03/rw/t.instance.lifecycle.base.sh
test/integration/v12.03/rw/t.instance.lifecycle.poweroff.sh
test/integration/v12.03/rw/t.instance.vifs.networking.running-out-of-ip.sh
test/integration/v12.03/rw/t.instance.vifs.null.sh
test/integration/v12.03/rw/t.instance.vifs.security_group.non-existent.sh
test/integration/v12.03/rw/t.ip_handle.sh
test/integration/v12.03/rw/t.ip_pool.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.multi-cidr.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.multi-ip.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.single-cidr.sh
test/integration/v12.03/rw/t.load_balancer.allow_list.single-ip.sh
test/integration/v12.03/rw/t.load_balancer.balance-algorithm.sh
test/integration/v12.03/rw/t.load_balancer.httpchk.sh
test/integration/v12.03/rw/t.load_balancer.lifecycle.base.sh
test/integration/v12.03/rw/t.load_balancer.multiple-listen.sh
test/integration/v12.03/rw/t.load_balancer.ssl-cert.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.multi-cidr.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.multi-ip.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.single-cidr.sh
test/integration/v12.03/rw/t.load_balancer.update.allow_list.single-ip.sh
test/integration/v12.03/rw/t.load_balancer.update.balance-algorithm.sh
test/integration/v12.03/rw/t.load_balancer.update.httpchk.sh
test/integration/v12.03/rw/t.load_balancer.update.multiple-listen.sh
test/integration/v12.03/rw/t.load_balancer.vifs.sh
test/integration/v12.03/rw/t.log.alarm.sh
test/integration/v12.03/rw/t.security_group.nesting.sh
test/integration/v12.03/rw/t.security_group.self-referred.sh
test/integration/v12.03/rw/t.security_group.sh
test/integration/v12.03/rw/t.ssh_key_pair.sh
test/integration/v12.03/rwx/Makefile
test/integration/v12.03/rwx/helper_instance.sh
test/integration/v12.03/rwx/helper_instance_vifs_multi.sh
test/integration/v12.03/rwx/helper_instance_vifs_single.sh
test/integration/v12.03/rwx/helper_load_balancer.sh
test/integration/v12.03/rwx/helper_shunit2.sh
test/integration/v12.03/rwx/t.instance.lifecycle.poweroff-poweron.sh
test/integration/v12.03/rwx/t.instance.lifecycle.reboot.sh
test/integration/v12.03/rwx/t.instance.lifecycle.stop-start.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.networking.non-secg.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.networking.secg.sh
test/integration/v12.03/rwx/t.instance.vifs.multi.null-net.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.external-ip.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.non-secg.sh
test/integration/v12.03/rwx/t.instance.vifs.single.networking.secg.sh
test/integration/v12.03/rwx/t.instance.vifs.single.null-net.sh
test/integration/v12.03/rwx/t.instance_monitoring.process.sh
test/integration/v12.03/rwx/t.load_balancer.external_ip.sh
test/integration/v12.03/rwx/t.load_balancer.lifecycle.poweroff-poweron.sh
test/integration/v12.03/rwx/t.log.alarm.sh
test/integration/v12.03/rwx/t.network_vif_monitor.port.sh
test/shunit2
test/unit/Makefile
test/unit/functions/Makefile
test/unit/functions/helper_shunit2.sh
test/unit/functions/t.add_args_param.sh
test/unit/functions/t.add_param.sh
test/unit/functions/t.base_uri.sh
test/unit/functions/t.call_api.sh
test/unit/functions/t.cmd_default.sh
test/unit/functions/t.cmd_destroy.sh
test/unit/functions/t.cmd_help.sh
test/unit/functions/t.cmd_index.sh
test/unit/functions/t.cmd_put.sh
test/unit/functions/t.cmd_show.sh
test/unit/functions/t.cmd_xcreate.sh
test/unit/functions/t.cmd_xget.sh
test/unit/functions/t.curl_opts.sh
test/unit/functions/t.extract_args.sh
test/unit/functions/t.invoke_task.sh
test/unit/functions/t.load_musselrc.sh
test/unit/functions/t.request_header.sh
test/unit/functions/t.request_param.sh
test/unit/functions/t.run_cmd.sh
test/unit/functions/t.shlog.sh
test/unit/functions/t.strfile_type.sh
test/unit/functions/t.suffix.sh
test/unit/functions/t.urlencode_data.sh
test/unit/v12.03/Makefile
test/unit/v12.03/helper_shunit2.sh
test/unit/v12.03/t.alarm.sh
test/unit/v12.03/t.backup_object.sh
test/unit/v12.03/t.backup_storage.sh
test/unit/v12.03/t.base.sh
test/unit/v12.03/t.dc_network.sh
test/unit/v12.03/t.host_node.sh
test/unit/v12.03/t.image.sh
test/unit/v12.03/t.instance.sh
test/unit/v12.03/t.instance_monitoring.sh
test/unit/v12.03/t.ip_handle.sh
test/unit/v12.03/t.ip_pool.sh
test/unit/v12.03/t.load_balancer.sh
test/unit/v12.03/t.network.sh
test/unit/v12.03/t.network_vif.sh
test/unit/v12.03/t.network_vif_monitor.sh
test/unit/v12.03/t.security_group.sh
test/unit/v12.03/t.ssh_key_pair.sh
test/unit/v12.03/t.storage_node.sh
test/unit/v12.03/t.undefined.sh
test/unit/v12.03/t.volume.sh
v12.03.d/alarm.sh
v12.03.d/backup_object.sh
v12.03.d/backup_storage.sh
v12.03.d/base.sh
v12.03.d/dc_network.sh
v12.03.d/host_node.sh
v12.03.d/image.sh
v12.03.d/instance.sh
v12.03.d/instance_monitoring.sh
v12.03.d/ip_handle.sh
v12.03.d/ip_pool.sh
v12.03.d/load_balancer.sh
v12.03.d/network.sh
v12.03.d/network_vif.sh
v12.03.d/network_vif_monitor.sh
v12.03.d/security_group.sh
v12.03.d/ssh_key_pair.sh
v12.03.d/storage_node.sh
v12.03.d/volume.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment