Using SpecSync with Postman
Last updated
Was this helpful?
Last updated
Was this helpful?
is a popular REST API testing tool with a cloud-based storage for Postman Collections.
SpecSync can be used to synchronize the tests defined in Postman with Azure DevOps Test Cases and publish test results of Postman tests executed with .
In order to use SpecSync with Postman tests, you need to enable the plugin by adding the following section to the SpecSync configuration file (specsync.json).
You might need to set other settings a well, as described below. You can find a complete configuration sample in the below.
Postman collections contain folders, sub-folders and requests. Requests can have additional pm.test checks as well. Postman does not define an exact "test" concept. Some teams think about the pm.test checks as tests, some use a request as a test and for some a test is a folder that contain multiple requests in a sequence.
SpecSync synchronizes the tests to Azure DevOps Test Cases, so when planning the Postman integration the first thing you need to decide is what is a test for you in your Postman Collection.
By default SpecSync will treat the Requests as tests but you can also instruct SpecSync to use a folder as tests in the following ways:
The IDs of the linked artifacts (Test Case, work items) can be defined as a link, so if someone visits the test in Postman they can easily navigate to the related artifact. SpecSync automatically creates such links when a new test is linked to a new Test Case.
The meta-data section is automatically added to the documentation when SpecSync links the test to a newly created Test Case, but can also be added manually.
Once the tests have executed and the result XML file has been created, you can use the SpecSync publish-test-results command with the NewmanJUnitXml format to publish the results to the remote server.
SpecSync analyzes the test results and matches them to the steps of the test case. As a result you can have a detailed, step-level test report in Azure DevOps.
You can specify a Regular Expression pattern that should match to the folder names you would like to treat as tests. This is useful if you maintain a naming convention for your test-folders. To use this, you need to set the . For example, if all of these folders start with the word TEST- you need to set the testNameRegex parameter to ^TEST-.
You can specify a Regular Expression pattern that should match to a specific text in the folder documentation. To use this, you need to set the . For example, if the documentation of the test-folders contain a "Test Case" H2 heading, you need to set the testDocumentationRegex parameter to (?m)^## Test Case (the (?m) regex setting will enable multi-line mode so that ^ will match begin of each line).
You might want to exclude some folders and requests from synchronization or only enable synchronization for particular folders. For that, you can use the local/tags or the local/sourceFiles settings. (See related .) For example to exclude the Postman folder Legacy from the synchronization, you need to specify:
SpecSync needs to preserve the ID of the linked Test Case in the Postman item (folder or request). As Postman does not support creating custom fields or add tags for these items, SpecSync stores this data by default in the documentation of the items in a separate section named SpecSync by default. (You can use a different name by specifying the .)
Besides the ID of the linked Test Case, you can store other information (meta-data) in this section that is useful for the synchronization: you can specify tags that are going to be synchronized as Test Case tags in Azure DevOps and you can also specify links to other work items that will be synchronized as links using the feature.
The following example shows the documentation of a Postman item, that is linked to the Test Case 1234, annotated with the fast and important tags and linked to the User Story 123 (assuming that the story prefix is configured in ).
You can synchronize the already linked Postman tests or link the new tests to newly created Test Cases with the SpecSync . If you just want to synchronize the linked tests, but not link the new ones, you can use the --disableLocalChanges option.
During synchronization SpecSync will analyze the Postman test folders or requests and convert them to Test Case steps in a way that all requests will become a step and all pm.test check will be shown as an expected result. For the best display it is recommended to set the to true, otherwise the pm.test checks will be displayed as separate step.
The plugin can also [publish test execution results]((../features/test-result-publishing-features/publishing-test-result-files.md) to the synchronized Test Cases. For that first you need to run the Postman tests with using the junit reporter. E.g.
If you have multiple environments defined in Postman, you can define the environment names in Azure DevOps as and for the publish-test-results command using the , e.g. -c Staging.
