Pushing scenario changes to Test Cases
Last updated
Was this helpful?
Last updated
Was this helpful?
SpecSync can synchronize scenarios and scenario execution results with Azure DevOps Test Cases. The most commonly used command to achieve this is to upload (push) the scenario changes to Test Cases that is provided using the SpecSync .
The SpecSync push command processed all scenarios in the (e.g. the features
folder of your project), detects if there were any change since the last synchronization and updates the Test Case if necessary.
You can find all available command line options for the push command in the . The configuration options related to push are in the section of the configuration file.
The connection between the scenario and the Test Case in Azure DevOps is established with a test case link tag. By default the link tag uses the format @tc:1234
, but both the tc
prefix and the separator :
can be configured (with synchronization/tagPrefixSeparators
and synchronization/linkLabelSeparator
), so you can also have test case links that use different format, e.g. @TestCase=1234
. In the documentation we use the default test case link tag format.
Depending on the changes of the scenario, the push command performs different actions in Azure DevOps as the following table shows.
The scenario is new (not yet linked to a Test Case)
A new Test Case work item is created and the scenario is tagged with a link tag (e.g. @tc:1234
) to document the ID of the created Test Case.
The scenario is linked and up-to-date with the Test Case
No action is taken
The scenario has been changed since the last synchronization
The Test Case is updated with the changes. (No change in the feature file.)
The scenario has been synchronized, but since the last synchronization the Test Case has been updated.
The push command by default overwrites the changes of the Test Case fields that are maintained by SpecSync (e.g. title, steps, tags). The changes in other fields (e.g. description) are not changed.
The scenario has been removed
The push command updates certain fields of the linked Azure DevOps Test Case to represent the scenario changes as the following table shows.
Title
Steps
Tags
Updated based on the scenario tags. (The tags placed on the feature block are also replicated in the Test Case.)
Parameter values
Links
State
Area and Iteration
Automation Status
Associated Automation
Attachments
SpecSync can also help you to organize the remote Test Cases into hierarchical structures. In Azure DevOps that means that it can create and maintain Test Suite hierarchies and include the Test Cases to these Test Suites according to the configured rules. The test results can also be published to such a synchronized hierarchy.
The synchronized Test Cases can also be added to various Test Suites manually or by defining query-based or requirement-based suites in Azure DevOps.
A different conflict handling method can also be configured. See for details.
In case you would like to preserve the changes of the fields maintained by SpecSync, you have to use the first.
SpecSync does not delete the related Test Case. If SpecSync is configured with a remote scope (recommended), the Test Case will be tagged with specsync:removed
. You can review the Test Cases of this tag and delete them manually. (The tag is removed from the Test Case if the scenario is restored.) For details on how to specify the remote scope and how does SpecSync detects hand processes removed local test cases, please check the documentation.
Updated based on the scenario name. By default it is set to Scenario: <scenario-name>
or Scenario Outline: <scenario-outline-name>
. The Scenario and Scenario Outline prefixes can be skipped, see . For special situations the synchronization of the Test Case title can be omitted, see .
Updated based on the scenario steps. By default each scenario step is represented by a Test Case step, but the synchronization can be configured to use the Expected Result field of the Test Case step for the Then steps. About this and other step-related formatting option can be configured, see for details.
By default all Test Case steps are overwritten by the scenario steps, but you can configure SpecSync to keep certain Test Case steps based on a prefix. See details in .
The tag synchronization can be customized with and .
The scenario outlines are synchronized to parametrized Test Cases. The values provided in the scenario outline Examples block are saved as parameter values. Multiple examples blocks are merged into one Test Case parameter value table. See for details.
The manually edited Test Case links are never removed by SpecSync, but you can configure SpecSync to detect scenario tags that refer to an Azure DevOps work item (e.g. @bug:1234
) and create Test Case links from them. See for details.
By default the push command does not change the value of the State field, but you can configure it to reset the state to a specific value (e.g. Design
) whenever the Test Case is changed. See for details.
By default the push command does not change the value of the Area and the Iterations fields, but you can configure it initialize these fields with a configured value (e.g. MyProject\Team1
) when the Test Case is initially created. See for details.
By default the push command creates the Test Cases as Not Automated
. To be able to synchronize automated Test Cases (for all or for selected scenarios), you can configure SpecSync for this. See for details.
For MsTest-based SpecFlow or Reqnroll scenarios, SpecSync can also configure the Associated Automation field with the test method generated by SpecFlow or Reqnroll for the scenario. This can be used for . In order to publish scenario execution results, you can also consider the SpecSync that provide a more flexible solution.
The manually uploaded Test Case attachments are never removed by SpecSync, but you can configure SpecSync to detect scenario tags that refer to attached file and upload Test Case attachments from them. See for details.
For details on how to include Test Cases to hierarchies, please check the feature documentation.