In my project, Jervis, it supports Matrix building via YAML using Jenkins scripted pipeline. However, it is quite easy to calculate matrix building using the cartesian product on a list of lists containing maps (the matrix axes).
For user pipelines the following script approvals must be allowed:
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods combinations java.util.Collection
The following Jenkins scripted pipeline will build combinations across two matrix axes. However, adding more axes to the matrix is just as easy as adding another entry to the Map matrix_axes.
// you can add more axes and this will still work
Map matrix_axes = [
PLATFORM: ['linux', 'windows', 'mac'],
BROWSER: ['firefox', 'chrome', 'safari', 'edge']
]
@NonCPS
List getMatrixAxes(Map matrix_axes) {
List axes = []
matrix_axes.each { axis, values ->
List axisList = []
values.each { value ->
axisList << [(axis): value]
}
axes << axisList
}
// calculate cartesian product
axes.combinations()*.sum()
}
// filter the matrix axes since
// Safari is not available on Linux and
// Edge is only available on Windows
List axes = getMatrixAxes(matrix_axes).findAll { axis ->
!(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
!(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}
// parallel task map
Map tasks = [failFast: false]
for(int i = 0; i < axes.size(); i++) {
// convert the Axis into valid values for withEnv step
Map axis = axes[i]
List axisEnv = axis.collect { k, v ->
"${k}=${v}"
}
// let's say you have diverse agents among Windows, Mac and Linux all of which have proper labels for their platform and what browsers are available on those agents.
String nodeLabel = "os:${axis['PLATFORM']} && browser:${axis['BROWSER']}"
tasks[axisEnv.join(', ')] = { ->
node(nodeLabel) {
withEnv(axisEnv) {
stage("Build") {
echo nodeLabel
sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
}
stage("Test") {
echo nodeLabel
sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
}
}
}
}
}
stage("Matrix builds") {
parallel(tasks)
}Matrix axes contain the following combinations:
[PLATFORM=linux, BROWSER=firefox]
[PLATFORM=windows, BROWSER=firefox]
[PLATFORM=mac, BROWSER=firefox]
[PLATFORM=linux, BROWSER=chrome]
[PLATFORM=windows, BROWSER=chrome]
[PLATFORM=mac, BROWSER=chrome]
[PLATFORM=windows, BROWSER=safari]
[PLATFORM=mac, BROWSER=safari]
[PLATFORM=windows, BROWSER=edge]
Adding choices requires only a few changes to the above script:
Map response = [:]
stage("Choose combinations") {
response = input(
id: 'Platform',
message: 'Customize your matrix build.',
parameters: [
choice(
choices: ['all', 'linux', 'mac', 'windows'],
description: 'Choose a single platform or all platforms to run tests.',
name: 'PLATFORM'),
choice(
choices: ['all', 'chrome', 'edge', 'firefox', 'safari'],
description: 'Choose a single browser or all browsers to run tests.',
name: 'BROWSER')
])
}
// filter the matrix axes since
// Safari is not available on Linux and
// Edge is only available on Windows
List axes = getMatrixAxes(matrix_axes).findAll { axis ->
(response['PLATFORM'] == 'all' || response['PLATFORM'] == axis['PLATFORM']) &&
(response['BROWSER'] == 'all' || response['BROWSER'] == axis['BROWSER']) &&
!(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
!(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}Which renders the following choice dialog
And can customize the matix axes filtered when selected.
For user experience, you'll want your choices to automatically reflex your the matrix axis options you have available. For example, let's say you want to add a new dimension for Java to the matrix.
// you can add more axes and this will still work
Map matrix_axes = [
PLATFORM: ['linux', 'windows', 'mac'],
JAVA: ['openjdk8', 'openjdk10', 'openjdk11'],
BROWSER: ['firefox', 'chrome', 'safari', 'edge']
]To support dynamic choices your choice and matrix axis filter needs to be updated to the following.
Map response = [:]
stage("Choose combinations") {
response = input(
id: 'Platform',
message: 'Customize your matrix build.',
parameters: matrix_axes.collect { key, options ->
choice(
choices: ['all'] + options.sort(),
description: "Choose a single ${key.toLowerCase()} or all to run tests.",
name: key)
})
}
// filter the matrix axes since
// Safari is not available on Linux and
// Edge is only available on Windows
List axes = getMatrixAxes(matrix_axes).findAll { axis ->
response.every { key, choice ->
choice == 'all' || choice == axis[key]
} &&
!(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
!(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}It will dynamically generate choices based on available matrix axes and will automatically filter if users customize it. Here's an example dialog and rendered choice when the pipeline executes.
See the full script with dynamic choices (click to expand)
// you can add more axes and this will still work
Map matrix_axes = [
PLATFORM: ['linux', 'windows', 'mac'],
JAVA: ['openjdk8', 'openjdk10', 'openjdk11'],
BROWSER: ['firefox', 'chrome', 'safari', 'edge']
]
@NonCPS
List getMatrixAxes(Map matrix_axes) {
List axes = []
matrix_axes.each { axis, values ->
List axisList = []
values.each { value ->
axisList << [(axis): value]
}
axes << axisList
}
// calculate cartesian product
axes.combinations()*.sum()
}
Map response = [:]
stage("Choose combinations") {
response = input(
id: 'Platform',
message: 'Customize your matrix build.',
parameters: matrix_axes.collect { key, options ->
choice(
choices: ['all'] + options.sort(),
description: "Choose a single ${key.toLowerCase()} or all to run tests.",
name: key)
})
}
// filter the matrix axes since
// Safari is not available on Linux and
// Edge is only available on Windows
List axes = getMatrixAxes(matrix_axes).findAll { axis ->
response.every { key, choice ->
choice == 'all' || choice == axis[key]
} &&
!(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
!(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}
// parallel task map
Map tasks = [failFast: false]
for(int i = 0; i < axes.size(); i++) {
// convert the Axis into valid values for withEnv step
Map axis = axes[i]
List axisEnv = axis.collect { k, v ->
"${k}=${v}"
}
// let's say you have diverse agents among Windows, Mac and Linux
// all of which have proper labels for their platform and what
// browsers are available on those agents.
String nodeLabel = "os:${axis['PLATFORM']} && browser:${axis['BROWSER']}"
tasks[axisEnv.join(', ')] = { ->
node(nodeLabel) {
withEnv(axisEnv) {
stage("Build") {
echo nodeLabel
sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
}
stage("Test") {
echo nodeLabel
sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
}
}
}
}
}
stage("Matrix builds") {
parallel(tasks)
}