Skip to content

Instantly share code, notes, and snippets.

@SohamRatnaparkhi
Last active August 21, 2023 06:42
Show Gist options
  • Save SohamRatnaparkhi/f7ddb3cb746678ac26a69d683644bfe1 to your computer and use it in GitHub Desktop.
Save SohamRatnaparkhi/f7ddb3cb746678ac26a69d683644bfe1 to your computer and use it in GitHub Desktop.
Chaos-center testing

Chaos-Center Testing

Understanding the Significance of Testing

In the realm of software development, the creation of robust, dependable, and easily maintainable code stands as a paramount objective. Achieving these goals hinges upon a fundamental practice: the crafting of effective tests. Tests serve a dual purpose. Firstly, they ascertain that your code functions as intended. Secondly, they provide a safety net that empowers you to introduce modifications and enhancements without the looming dread of inadvertently disrupting existing functionalities.

Testing structure in Go

When it comes to writing unit tests in Go, the language follows a well-established structure.

Test File Naming Convention:

Within the Go ecosystem, adhering to a specific naming convention for test files is essential. For instance, if you have a primary file named module.go, the accompanying tests will be housed in a file titled module_test.go. In the context of the Chaoscenter package, the core packages that undergo testing include experiments, experiment_runs, and chaos_hub. These packages comprise two pivotal files: service.go and handler.go. Consequently, the test suite for service.go can be found in service_test.go, while the tests for handler.go are present within handler_tests.go.

Importing Testing Packages:

There are 2 testing packages, used for testing:

  • testing: This is the standard Go testing package, which offers functions for executing and asserting tests. (link)
  • testify\mock: The mock package, part of the testify suite, facilitates the creation of object mocks and the validation of expected method calls. This is very useful in situations where you have to test function making calls to certain enviroments which are absent while testing. For example, a database, or a GitOps system. (link)

In further sections, we shall explore more on them.

Test Functions conventions

According to the Golang testing package, we follow these naming conventions.

func helloWorld() {} // target function
func TestHelloWorld() {} // test function

type FooStruct struct {}
func (f *FooStruct) Bar() {} // target method
func TestFooStruct_Bar(){} // test function

n the Chaos-Center testing methodology, the testing process is divided into three distinct stages: "given," "when," and "then."

  • given: This initial stage used to setting up the necessary context for the test. It involves defining the function arguments, constant data required for all tests, configuring mock data if needed, and specifying a boolean parameter indicating whether you are expecting an error to occur.
  • when: In this phase, you outline the specific test case you want to evaluate. This could involve invoking a function, method, or action that you intend to test.
  • then: The final stage involves executing the actual test. This encompasses calling the function or performing the action established in the "When" stage and subsequently verifying that the outcome matches your expectations.

Here's an illustrative example of a test function adhering to this structure:

func TestStruct_MyFunc(t *testing.T) {
    type args struct {
        param1 string
        param2 string
    }

    //given
    someGlobalTestCont := "some_value"

    tests := []struct {
        name string
        args args
        wantErr string
        given func()
    }{
        {
            // given
            name: "test1",
            args: args{
                param1: "param1",
                param2: "param2",
            },
            given: func() {
                // mock data
                // for example mocking the database
            },
            wantErr: false,
        },
    }
    for _, tc := range tests {
        t.Run(tc.name, func(t *testing.T) {
            // when
            tc.given()
            err := MyFunc(tc.args.param1, tc.args.param2)

            // then
            if (err != nil) != tc.wantErr {
                t.Errorf("MyFunc() error = %v, wantErr %v", err, tc.wantErr)
            }
        })
    }
}

Mocking data

While testing, particularly when dealing with functions that rely on external resources like databases, it's common to employ a technique called mocking. Mocking allows you to simulate the behavior of these external systems, ensuring that your tests remain independent and predictable.

For instance, consider a scenario where a function interacts with a MongoDB database. Instead of making actual database calls during testing, you can create mock implementations to control the responses. For this, the testify/mock library is used.

Here's how you can do it:

  1. Mocking the MongoDB Operator: You create a mock version of the MongoDB operator's Get method using a library like github.com/stretchr/testify/mock. This mock method expects specific arguments and is programmed to return a predefined SingleResult object and an error. When the Get method is called with specific arguments, it responds with the predetermined singleResult and a nil error.
  2. Predefined Response: You construct a SingleResult object, named singleResult, using predefined data stored in the findResult variable. This singleResult represents the response you anticipate the MongoDB operator's Get method to return.
  3. Ensuring Compatibility: It's crucial to ensure that the structure of your mock data (findResult) precisely matches the structure of the actual database data. This deep matching (i.e, same key-value pairs) is essential to prevent any potential issues like segmentation faults or errors resulting from disparities between the mock data and real data. Here's a demo implementation of the same
findResult := bson.D{
					{Key: "some_key", Value: someValue},
	       	}
singleResult := mongo.NewSingleResultFromDocument(findResult, nil, nil)
mongodbMockOperator.On("Get", mock.Anything, mock.Anything, mock.Anything).Return(singleResult, nil).Once()

mongodbMockOperator.On("Get", mock.Anything, mongodb.ChaosExperimentRunsCollection, mock.Anything).Return(singleResult, nil).Once()

The get function in the mock mongo operator is written as follows:

func (m MongoOperator) Get(ctx context.Context, param1 int, query bson.D) (*mongo.SingleResult, error) {
	args := m.Called(ctx, param1, query)
	return args.Get(0).(*mongo.SingleResult), args.Error(1)
}

Here, the single result will actually hold the values we want to return which is intern the data in findResult. The mock operator will return the single result, and the error (if any) as well, and hence no database calls are made.

In chaoscenter mocks are available for

  • experiments service
  • experiment_runs service
  • chaos_infrastructure service
  • gitops service
  • mongo operator

The Chaos Experiments package within ChaosCenter comprises two essential files: service.go and handler.go. These files play a pivotal role in the package's functionality. Specifically, all functions within service.go are associated with the chaosExperimentService struct. The NewChaosExperimentService function is responsible for initializing this service and requires parameters such as chaosExperimentOperator, infraOperator, and chaosExperimentRunOperator.

During the testing phase, a distinctive approach is employed to ensure smooth testing without actual database interactions. The following steps elucidate this process:

  1. Mocking Operators:

In the testing environment, the functions within the chaosExperiment, chaosExperimentRun, and infraOperator utilize a mock MongoDB operator. This substitution prevents any real database calls from being triggered during testing.

  1. Indirect Operator Returns:

By employing the mock MongoDB operator, the functions within the chaosExperiment, chaosExperimentRun, and infraOperator internally return their respective operators, all equipped with the mock MongoDB operator. The layered structure of the chaos center allows mock operators to be passed in testing environment and real operators in production.

opr5

Tests for service.go (link)

For the Process Experiment function following test have been added for it:

Test name Result Expects
error?
success: Process Experiment (type-workflow) Process an experiment with mock manifest
type as workflow
No
success: Process Experiment (type-cron_workflow) Process an experiment with mock manifest
type as cron-workflow
No
success: Process Experiment (type-chaos_engine) Process an experiment with mock manifest
type as chaos-engine
No
success: Process Experiment (type-chaos_schedule) Process an experiment with mock manifest
of type chaos-schedule
No
failure: Process Experiment (type-random(incorrect)) Process an experiment with mock manifest
with random type
Yes
failure: incorrect experiment name Throw an error as the experiment name does'nt
match the manifest type provide
Yes
failure: unable to unmarshal experiment manifest Throw an error dur to incorrect formatting of
yaml manifest
Yes
failure: inactive infra Failure to process experiment due to
inactive infra
Yes
failure: incorrect project ID ProjectID provided in the request doesn't match
with the one provided in function arguments
Yes
failure: mongo returns empty result Mongo returns a nil experiment Yes

Tests for ProcessExperimentCreation

Test name Result Expects
 error?
success: Process Experiment Creation Creates an experiment when weights are not
provided.
No
success: Process Experiment Creation with weights Creates an experiment when weights are
provided.
No

Tests for ProcessExperimentUpdate

Test name Result Expects
 error?
success: Process Experiment Update Function ran as expectd on the provide
mock data and arguments.
No
failure: incorrect experiment manifest Function fails due to incorrect manifest format Yes
failure: failed to update experiment Mongo update operation failure Yes

Tests for handler.go (link)

Tests for SaveChaosExperiment

Test name Result Expects
 error?
Save Chaos Experiment Successfully save an experiment No
Failure: mongo error MongoDB fails to get an experiment Yes
Failed to process experiment Failure to incorrect experiment format Yes
Failed to create experiment Chaos experiment service fails to create an experiment Yes

Tests for DeleteChaosExperiment

Test name Result Expects
 error?
success: Delete Chaos Experiment Successfully delete an experiment No
failure: mongo error while retrieving the
experiment details
MongoDB fails to get experiments Yes
failure: mongo error while retrieving the
experiment run details
MongoDB fails to get experiment runs Yes
failure: unable to delete experiment-runs Choas experiment run service fails to delete experiment
run
Yes

Tests for UpdateChaosExperiment

Test name Result Expects
 error?
Update Chaos Experiment Successfully update an experiment No
Failed to process experiment Chaos experiment service fails to process an experiment Yes
Update failed Chaos experiment service fails to update an experiment Yes

Tests for GetExperiment

Test name Result Expects
 error?
success: Get Experiment Successfully Get an experiment No
failure: Kubernetes infra details absent Kubernetes Infra details absent in the experiment
details returned
Yes
failure: empty mongo cursor returned MongoDB returns empty cursor for an aggregate request Yes
failure: Absent experiment run details Experiment run details absent in the experiment returned from DB Yes

Tests for ListExperiment

Test name Result Expects
 error?
List Experiment Successfully list experiments associated with a project ID No
success: sorting in descending order of name List experiments successfully in descending order of name No
success: sorting in descending order of time List experiments successfully in descending order of time No
failure: listing experiment MongoDB fails to aggregate experiments Yes

Tests for DisableCronExperiment

Test name Result Expects
 error?
Success: Disable Cron Experiment Successfully disable a cron experiment No
Failure: mongo error while updating the
experiment details
Chaos experiment service fails to update experiment Yes

Tests for GetExperimentStats

Test name Result Expects
 error?
success: get experiment stats Successfully get experiment stats No
failure: empty cursor returned MongoDB returns empty cursor Yes
failure: getting experiment stats MongoDB fails to aggregate experiments Yes

Tests for handler.go (link)

Tests for GetExperiment

Test name Result Expects
 error?
success: Get Experiment Run Successfully Get an experiment run No
failure: Kubernetes infra details absent Kubernetes Infra details absent in the experiment run
details returned
Yes
failure: empty mongo cursor returned MongoDB returns empty cursor for an aggregate request Yes
failure: Absent experiment run details Experiment run details absent in the experiment returned from DB Yes

Tests for ListExperimentRuns

Test name Result Expects
 error?
success: ListExperimentRun Successfully list experiments runs associated with a project ID No
success: sorting in descending order of name List experiment runs successfully in descending order of name No
success: sorting in descending order of time List experiment runs successfully in descending order of time No

Tests for RunChaosWorkFlow

Test name Result Expects
 error?
success: RunChaosWorkFlow Successfully run a chaos experiement No

Tests for GetExperimentRunStats

Test name Result Expects
 error?
success: GetExperimentRunStats Successfully get experiment run stats No
failure: GetExperimentRunStats MongoDB fails to aggregate experiment runs Yes

Tests for service.go (link)

Tests for ProcessExperimentRunDelete

Test name Result Expects
 error?
success: deleting experiment run Successfully delete experiment run No
failure: deleting experiment run MongoDB fails to update experiment runs Yes

Tests for ProcessCompletedExperimentRun

Test name Result Expects
 error?
success: processing completed experiment run Successfully process experiment run No
failure: can't find unique experiment run Unique experiment runs not found Yes
failure: nil mongo single result MongoDB fails to return an experiment Yes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment