Unit Testing Best Practices
Table Of Contents
- Unit Testing Declarative Pipelines
- Encapsulate Pipeline logic within Groovy functions
- Mocking Jenkins Dependencies
- Add plugin dependency to Gradle
- Mocking Environment Variables
- Testing environment variables
- Mock external shared library methods
- Integration Testing
- Mock errors
- Mock external shared library methods
- Call Graph Example
Unit Testing Declarative Pipelines
The edgex-global-pipelines shared library leverages the Jenkins Spock framework for unit testing Jenkins pipeline scripts and functions. The Jenkins Spock unit test framework does not currently support unit testing of Jenkins Declarative Pipeline code.
Encapsulate Pipeline logic within Groovy functions
In order to facilitate unit testing of the edgex-global-pipelines shared library, the DevOps team has made a deliberate effort to to minimize the amount of scripting logic contained within Jenkins declarative pipelines. This is accomplished by encapsulating pipeline logic within a Groovy function and calling the function in the declarative pipeline step as needed. Localizing pipeline logic within Groovy functions enables the Jenkins Spock framework to provide greater test coverage of Pipeline logic.
An example this approach can be seen within the
Build -> amd64 -> Prep stage of the edgeXBuildCApp Delcarative Pipeline. Note the logic for prepping the base build image is encapsulated into a method named
prepBaseBuildImage and it is called within the declarative Pipeline. Also the
prepBaseBuildImage function logic is thoroughly unit tested in the edgeXBuildCApp Spec
Mocking Jenkins Dependencies
Always leverage the builtin capabilities of the Jenkins-Spock framework for mocking Jenkins plugins. For example, if you come across the following error when unit testing your code:
java.lang.IllegalStateException: During a test, the pipeline step [stepName] was called but there was no mock for it.
stepNamebut there is no mock for it. You are able to explicitly mock the pipeline step using
explictlyMockPipelineStepmethod available in the Jenkins-Spock framework. However it is recommended that the plugin that contains the corresponding step be added as a dependency in the
build.gradlefile. For instructions on how to do this, refer to the Add plugin dependency to Gradle section.
Add plugin dependency to Gradle
- Note the name of the Pipeline Step to add.
- Go to Pipeline Steps Reference page.
- Use your browser and search for the Pipeline Step within the page.
- If the Pipeline Step is found, click on the Pipeline that it belongs to, the page for the respective Pipeline should open.
- Under the heading click on the
View this plugin on the Plugins sitelink, the plugins.jenkins.io page should open.
- In the plugins.jenkins.io page note the ID for the Pipeline. You will use this ID in the next step.
- Go to Maven Repository page.
- Enter the ID in the search, and locate the result from the results displayed, click on the respective link.
- In the page, click on the
- If you know the version then click it, otherwise click on the latest version that is listed.
- In the Gradle tab, note the group, name and version.
- Edit the
build.gradlefile, add the dependency found above to the dependencies section.
Mocking Environment Variables
Always ensure the source code under test uses one of the following idioms for getting or setting Environment Variables, doing this will simplify the ability to mock environment variables in the unit test:
- Getting the value of an environment variable
- Setting the value of an environment variable
env.VARIABLE = VALUE
env[VARIABLE] = VALUE
Testing environment variables
Within your unit tests, environment variables are set using the
.getBinding().setVariable('name', 'value') idiom. Where the name is
env and the value is a map you define within your unit test. The map should define all environment variables the code under test expects, likewise the map can be used to assert any environment variables that the code under test sets.
A good example of this practice is the EdgeXSetupEnvironmentSpec
Mock external shared library methods
edgex-global-pipelines Jenkins shared library consists of multiple scripts exposing methods for various functional areas, where each script is named after the particular functional area it serves. The shared library includes a
EdgeX script that serves as utility script containing methods that are shared amongst other scripts. It is common practice for a method in one script call a method in another script, to mock the interaction you use the
explictlyMockPipelineVariable to mock the script, then
getPipelineMock method to verify the interaction or stub it if necessary.
Mock the external script named
Get the script mock and stub the call to
method to return
'value' for any argument passed in:
getPipelineMock('script.method').call(_) >> 'value'
Integration Testing is defined as a type of testing where software modules are integrated logically and tested as a group. The Jenkins-Spock framework provides the ability to load any number of scripts to test within a given Spec Test. There are instances where performing integration tests is more practical, if you wish to do so then we recommend naming the Spec Test with
Int as to differentiate between unit and integration tests.
A good example of this practice is the EdgeXReleaseDockerImageIntSpec
error when wanting to conditionally terminate part of your script. Error is a Pipeline Step whose plugin has been added as a dependency to our project thus is already mocked by the framework. An example showing how you can assert that an error is thrown with a specific message:
1 * getPipelineMock('error').call('error message')
Mock external shared library methods
The difficulties of mocking functions within the same script under test have been described in the following issue: Issue 78. Due to the nature of how the scripts that comprise the
edgex-global-pipelines shared library are written; where a deliberate intent is made to develop small, functionally cohesive methods that contribute to a single well-defined task. This development intent results in having scripts with multi-layered call graphs, where methods may call multiple methods from within the same script. We find that the workaround provided in the issue is complicated and doesn't scale well in our environment. For these reasons the method outlined below is being suggested.
- For the script under test, document its call graph. A call graph is a control flow graph, which represents calling relationships between methods in a script or program. Each node represents a method and each edge (f, g) indicates that method f calls method g. An example EdgeXReleaseGitTag call graph is depicted below.
- Create a second script with the same name as the original script with the word Util added to the end, for example
- Analyze the call graph, methods that reside in odd numbered layers should continue to reside in the first script, methods at even numbered layers should be moved from the first script into the second script.
- Create a Spec Test for both scripts.
Mocking of methods between both scripts follow the same pattern described for Mock external shared library methods. The only difference with this approach is that the scripts are (for the lack of a better word) name spaced for the respective functional area.
Call Graph Example
NOTE The approach outlined above is not recommended as the standard development approach, but as an alternative to re-writing the script under test if mocking of the internal method calls becomes unwieldy.