Imagine we have a test suite, in that test suite we want to run scenarios against a deployed application that has accounts configured.
That deployed application has a number of accounts, and they are configured for possible real-world possibilities.
That we want to run that test suite either locally from our own development setup, and from a Continous Integration step.
Our company's security team might ask us NOT TO commit passwords in the code base.
The test suite might run from different users locally, or from any other deployment (e.g. UAT, production, etc).
Where we might have a Continous Integration utility where we store secrets, and where they become available as Continous Integration shell Environment variables.
In the test suite, we want to be able to tell to run a scenario for a given user
(e.g. userWithProfilePicture)
So we need a way to tell from the test case which user to use, and "trust"
that the proper username and password be used.
Over time, in a big team, we might add more than two users, and we might want to run tests not only on UAT, but maybe on production.
So we need systems:
- To make it easy in tests to say which user to use
- To fill up each possible test accounts's
usernameandpassword - To support loading from either a file, or a shell environent
- Have the warranty that ONLY and ALL users has values, and are non empty
So that:
- Each test-case has code completion and helpers
- Test-case fails ONLY WHEN the actual feature fails, NOT when there's missing data
Say we have a known number of keys (e.g. 'userWithProfilePicture',
'userWithProfilePicture') that must exist, where each entry has a Data
Transfer Object of a specific shape (e.g.
{username: string, password: string}), and that they can't be with empty
values (e.g. '' is unacceptable).
Instead of using TypeScript enum where in the end would make an Object
(see constant enum members),
let's leverage ECMAScript Map.
When we use non constant enum, we get pretty much an object where each key and values reverselookup
export enum FooTestingUserNames {
userWithProfilePicture = 'userWithProfilePicture',
userTwoKey = 'userTwoKey',
}Becomes
export var FooTestingUserNames
;(function (FooTestingUserNames) {
FooTestingUserNames['userWithProfilePicture'] = 'userWithProfilePicture'
FooTestingUserNames['userTwoKey'] = 'userTwoKey'
})(FooTestingUserNames || (FooTestingUserNames = {}))Where if we would use const enum FooTestingUserNames {}, would have no output
at all.
Most of code base omit or forgets that we can have const enums (even the
author of this note, by the way!).
Symptom is still that the object FooTestingUserNames from which we want to get
the value from it (e.g. FooTestingUserNames.userWithProfilePicture) as a way
to select from another hash map by from an object that can be mutated at run
time is a risk for an unexpected state.
The main idea of having FooTestingUser.userWithProfilePicture is that we want
to know that FooTestingUser can either pick userWithProfilePicture OR
userTwoKey.
That’s useful for type checking and switching, but it is contrived and not always easy to properly check.
From those keys (userWithProfilePicture OR userTwoKey), we want an object
shape that might look like this Record<'username' | 'password', string>
export type IAccountCredential = Record<'username' | 'password', string>
export enum FooTestingUserNames {
userWithProfilePicture = 'userWithProfilePicture',
userTwoKey = 'userTwoKey',
}
export const FooTestingUser = {
userWithProfilePicture: {
username: 'user1@localhost',
password: 'password1',
},
userTwoKey: { username: 'user2@localhost', password: 'password2' },
}But we might often see colleagues forget to type which FooTestingUser keys can
and MUST only contain defined entries
So we'd manually add typing Record<FooTestingUserNames, IAccountCredential> in
a review comment.
export const FooTestingUser: Record<FooTestingUserNames, IAccountCredential> = {
userWithProfilePicture: {
username: 'user1@localhost',
password: 'password1',
},
userTwoKey: { username: 'user2@localhost', password: 'password2' },
}Which is fine, because if we try to add userThreeKey we get an error message
which contains;
export const FooTestingUser: Record<FooTestingUserNames, IAccountCredential> = {
userWithProfilePicture: {
username: 'user1@localhost',
password: 'password1',
},
userTwoKey: { username: 'user2@localhost', password: 'password2' },
userThreeKey: { username: 'user3@localhost', password: 'password3' },
//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// Type '{ ... userThreeKey: { ... }; }' is not assignable to type 'Record<FooTestingUserNames, IAccountCredential>'.
// Object literal may only specify known properties, and 'userThreeKey' does not exist in type 'Record<FooTestingUserNames, IAccountCredential>'
}Type '{ userThreeKey: { username: string; password: string; }; }' is not assignable to type 'Record<FooTestingUserNames, IAccountCredential>'
In other words userThreeKey is not in the known list of FooTestingUserNames
and that is a good thing. But, at that is type checking, not enforced during run
time.
Also, those values can't be changed dynamically from a JSON file, or from shell environment.
Typically in a company, the Continuous Integration system is shared, and for good and for bac, we all have to deal with its particularities.
We could manually implement that scenario by hand specifically for one company's project, and let other teams re-implement that problem.
The ideal solution should be re-usable, and if there are missing or unexpected situation, to fail before starting the test run.
The concerns are at 3 levels:
-
We can’t know if one project uses exactly the same "user" types, it has to be configurable
- Ability to describe the possible users that must exist
- Ability to tell which Shell environment variable names to use for each possible users and fields
- Ability to tell which Input file should be read from, should it exist
-
Detect and Load the data
- Load from a data source, and only continue if all criterias are met
- If the first one failed, carry on with the other one
- If non passed, throw an exception
- Only one of the following is acceptable;
- If there are Shell environment variables, it we should read from it fist
- If the Input file is present, load, parse its contents, and ensure it fills all
- Load from a data source, and only continue if all criterias are met
-
Expose available possible users
- Ability to just say
FooTestingUser.userWithProfilePicturein the code, and the test suite code will get the appropriate data.
- Ability to just say
To work on that scenario, we can write up functionaly based on example above, where some recurring situations occurs, they're expanded in the following notes:
- Renoir's references to books and chapters Gist Using Enum and user-defined discriminator
- How can one can use Enum and utility functions expanded from Enums, Gist "TypeScript type Discriminator factory"
- Enforcing and validating runtime configuration
/**
* The Data Transfer Object containing the current values for both keys
* username and password.
*/
export type IAccountCredential = Record<'username' | 'password', string>
/**
* When we want to pass an unknown object, best normalize so we can use it for picking values.
*/
export interface INormalizedHashMap {
readonly discriminant: 'NormalizedHashMap'
[key: string]: string
}
/**
* What `create` returns is a contextualized manager.
* That manager should make sure it loads data
* and ensure it throws errors if something unexpected
*/
export interface ITestingAccountsCredentialMarshaller<
N extends Readonly<string>,
K extends N[number]
> {
/**
* true when all credentials are set
*/
readonly loaded: boolean
/**
* List of user name keys
*/
readonly TEST_ACCOUNTS: ReadonlySet<K>
/**
* When we have to pick key and values for each account from
* process.env, we got to know for which user and keys they are
* going to be mapped for.
*/
createProcessEnvPickerMap(): Map<K, IAccountCredential | null>
getCredential(name: K): IAccountCredential
/**
* Define a credential map for an user name (keys)
*/
setCredential(name: K, accountCredential: IAccountCredential): boolean
isAccountName(name: string): name is K
loadFromNormalized(
normalized: INormalizedHashMap,
pickerMap: Map<K, IAccountCredential | null>,
): boolean
toHashMap(): Readonly<Record<K, IAccountCredential>>
}It should look like this
// File: where-we-define-things.ts
import {
createTestingAccountsCredentialMarshaller,
normalizeHashMap,
} from 'Type-Guarded-limited-Map-of-named-Data-Transfer-Object'
/**
* To enforce which users MUST exist, tell their names in this array
*
* So that we can organize how to load values for them.
*/
export const THIS_APPLICATION_TEST_USERS = createTestingAccountsCredentialMarshaller(
['userWithProfilePicture'],
)
/**
* This is to say;
*
* For this app, we know in CI mode, we'll have each user with their credentials coming
* from process.env.<key>, here is the mapping.
*
* If:
* - The system detects any missing account names ("userWithProfilePicture"), it will throw an exception.
* - During CI mode, we don't get value for any of those names, it will throw an exception
* - Any of those fails, we'll trust the data will be loaded in another way
*/
const processEnvPickerMap = THIS_APPLICATION_TEST_USERS.createProcessEnvPickerMap()
processEnvPickerMap.set('userWithProfilePicture', {
// Say we have a CI with process.env.username_for_userWithProfilePicture for userWithProfilePicture’s username value
username: 'username_for_userWithProfilePicture',
password: 'password_for_userWithProfilePicture',
})
/**
* This is the object from which we’ll know from the tests which account names are availables.
*/
export type IFooTestingUser = ReturnType<
typeof THIS_APPLICATION_TEST_USERS.toHashMap
>
export const tryLoadPromProcessEnv = (): IFooTestingUser => {
const normalized = normalizeHashMap(process.env)
THIS_APPLICATION_TEST_USERS.loadFromNormalized(
normalized,
processEnvPickerMap,
)
// If any keys missing (e.g. we have another entry in the initial array from create above) it would fail here
const FooTestingUser = THIS_APPLICATION_TEST_USERS.toHashMap()
// For now, but maybe to be reworked, this is how we make sure all is there and fine
return FooTestingUser
}
export const tryLoadFromOtherMethod = (): IFooTestingUser => {
// If it worked from tryLoadPromProcessEnv, this should fail ASAP and not mutate.
/**
* Really, this should be done in some way from a JSON file and fill the values.
* But, for this example, let's do by hand.
*/
THIS_APPLICATION_TEST_USERS.setCredential('userWithProfilePicture', {
username: 'user1@localhost',
password: 'password1',
})
// If any keys missing (e.g. we have another entry in the initial array from create above) it would fail here
const FooTestingUser = THIS_APPLICATION_TEST_USERS.toHashMap()
return FooTestingUser
}Then from another file, say a test suite helper
// File: some-helper.ts
import { Helper } from 'codeceptjs'
import {
tryLoadPromProcessEnv,
tryLoadFromOtherMethod,
THIS_APPLICATION_TEST_USERS,
} from './where-we-define-things'
export class SomeHelper extends Helper {
_init(): void {
try {
// Whichever, works first, but only one will.
// In any case,
// The job of those actions is to fill `THIS_APPLICATION_TEST_USERS`'s _internal
// Map with their respective IAccountCredential values set.
// It is also to make sure only and all `TEST_ACCOUNTS` "account names" are present,
// with non empty values.
tryLoadPromProcessEnv()
// Try load from elsewhere too here
tryLoadFromOtherMethod()
// Maybe those should throw two exeptions
// - Everything is already set (no need to continue)
// - Something we need to know and fail for.
} catch (_) {
/* */
}
}
getUser: ReturnType<typeof THIS_APPLICATION_TEST_USERS.getCredential> = (
testUser,
) => {
// Once loaded, from whichever data source, first that succeeded,
// we can delegate getting values using getCredential
const credentialMap = THIS_APPLICATION_TEST_USERS.getCredential(testUser)
return credentialMap
}
}
