SpecSync Documentation
Getting StartedConfigurationGuidesDownloads
Azure DevOps
Azure DevOps
  • Introduction to SpecSync for Azure DevOps
  • Getting started
    • Getting started using SpecFlow
    • Getting started using Cucumber or other Gherkin-based BDD tool
  • Installation & Setup
    • Install as .NET tool
    • Install as .NET Console App
    • Install as native binaries for Linux or macOS
    • Install as Docker image
    • Setup and Configure
  • Features
    • Push features
      • Pushing scenario changes to Test Cases
      • Configuring the format of the synchronized test cases
      • Synchronizing Scenario Outlines
      • Add new Test Cases to an Area or an Iteration
      • Mark Test Cases as Automated
      • Setting Test Case state on change
      • Update Test Case fields
      • Attach files to Test Cases using tags
      • Customization: Setting Test Case fields with default values
      • Customization: Update custom Test Case fields on push
      • Customization: Ignoring marked Test Case steps
      • Customization: Ignoring Test Case Tags
      • Customization: Ignore non-supported local tags
      • Customization: Mapping tags
      • Customization: Synchronizing scenarios from feature branches
      • Customization: Reset Test Case state after change
      • Customization: Automatically link changed Test Cases
      • Customization: Synchronize linked artifact titles
      • Customization: Add Test Cases to Suites
      • Customization: Do not synchronize title
    • Pull features
      • Pulling Test Case changes to local scenarios
    • Common synchronization features
      • Configuration key
      • Remote scope
      • Linking Work Items using tags
      • Synchronizing Test Case hierarchies using Test Suites
      • Include synchronized Test Cases to a Test Suite (deprecated)
      • Excluding scenarios from synchronization
      • Synchronization conflict resolution
      • Re-link scenarios to new Test Cases
    • Test result publishing features
      • Publishing test result files
      • Support for Azure DevOps Test Plan / Test Suite based test execution
      • Customization: Publishing test results to multiple Test Suites
    • General features
      • Azure DevOps authentication options
      • Configuration file
      • Hierarchical configuration files
      • Local test case conditions
      • Configuration wizards
      • SpecSync plugins
    • Customizations
    • Plugin list
  • Licensing
  • Guides
    • What is my Azure DevOps project URL?
    • How to define the local feature-set to be synchronized
    • Filters and scopes
    • How to synchronize automated test cases
    • How to use SpecSync from build or release pipeline
    • How to publish test results from pipelines using the VSTest task
    • How to use the SpecSync Azure DevOps pipeline tasks
    • How to link GitHub pull requests
    • How to upgrade to a newer version of SpecSync
    • How to attach files to test results
    • Using SpecSync with SpecFlow+
    • Using SpecSync with Cucumber
    • Using SpecSync with Cypress
    • Using SpecSync with Postman
    • Using SpecSync with TestNG
    • Using SpecSync on macOS or Linux
    • Using SpecSync inside a Docker container
    • How to handle Test Cases of multiple parallel application releases
    • Migrating from SpecSync v3 to v5
    • Migrating from SpecSync v2 to v3
    • Migrating from SpecSync v1 to v2
  • Changelog
  • Release Model and Roadmap
  • Downloads
  • Reference
    • Command line reference
      • init
      • upgrade
      • push
      • pull
      • publish-test-results
      • re-link
      • version
    • Configuration reference
      • toolSettings
      • local
      • remote
      • knownRemotes
      • synchronization
        • push
        • pull
        • automation
        • state
        • areaPath
        • iterationPath
        • links
        • attachments
        • format
        • fieldUpdates
      • hierarchies
      • publishTestResults
      • specFlow
      • reqnroll
      • customizations
    • Compatibility
    • Older versions
  • Contact
    • SpecSync Support
    • Troubleshooting
    • FAQ
  • Project Website
Powered by GitBook
On this page
  • Step 1: Define feature file sets
  • Step 2: Initialize configuration file using the SpecSync init command
  • Step 3: Review SpecSync features and extend configuration settings
  • Step 4: Verify setup by synchronizing scenarios
  • Setup SpecFlow plugin to support "Test Suite based execution with Scenario Outline wrappers" automation strategy for .NET SpecFlow or Reqnroll projects
  • Package to be used for SpecFlow
  • Package to be used for Reqnroll

Was this helpful?

  1. Installation & Setup

Setup and Configure

PreviousInstall as Docker imageNextFeatures

Last updated 3 months ago

Was this helpful?

Once you have installed SpecSync for Azure DevOps tool, you need to configure it. This means you have to create a SpecSync configuration file (specsync.json) that contains the Azure DevOps project connection details and other synchronization settings.

Step 1: Define feature file sets

SpecSync has to be configured for every feature file set. Feature file set is a folder that contains feature files that you want to synchronize as one batch.

Feature file set is a term introduced by SpecSync to denote the set of feature files that belong together in a platform independent way. Earlier we used the term project folder, but it was misleading, because people might call different things as "project" on .NET, Java and node.js platforms and also it could be confused with Azure DevOps projects.

If you use SpecSync with SpecFlow, the Visual Studio projects configured with SpecFlow are the feature file sets.

For other BDD tools, like Cucumber, usually the features folder(s) are set as feature file sets.

Each feature file set will have its own SpecSync configuration file (specsync.json). For projects with multiple feature file sets, you can use as well.

It is up to you how you define your feature file sets. The following table contains a few usual setup variations.

Platform & tool
Feature file set (location of specsync.json)

.NET solution with a single SpecFlow project

The SpecFlow project folder (e.g. MyProject.Specs)

.NET solution with multiple SpecFlow projects

Each SpecFlow project folder (e.g. MyProject.BackOffice.Specs and MyProject.ClientApp.Specs)

node.js app with Cucumber.js

The parent folder of the features folder (e.g. /src/test/)

node.js app with multiple feature file sets

The parent folders of the features folders (e.g. /src/test/backoffice/ and /src/test/clientapp/)

Java app with Cucumber Java

The parent folder of the features folder (e.g. /src/test/resources/)

You can also consider setting the features folder itself as a root of the feature file set (e.g. /src/test/resources/features/ instead of /src/test/resources/).

The feature file sets of your project and be changed later by moving the specysnc.json configuration file to a different location.

Step 2: Initialize configuration file using the SpecSync init command

The easiest way to initialize the configuration file and configure the most important settings is to use the SpecSync init command. To use the command open a command window or bash shell and change the current directory to the root folder of the feature file set and invoke the following command.

dotnet specsync init
SpecSync4AzureDevOps.cmd init
<EXTRACTED-SPECSYNC-FOLDER>/SpecSync4AzureDevOps init

It is recommended to let the init command test the authentication to the Azure DevOps project to make sure the connection is configured properly.

Step 3: Review SpecSync features and extend configuration settings

Step 4: Verify setup by synchronizing scenarios

When the SpecSync configuration file is complete, you can perform the first synchronization task and verify the results using the push command.

For larger feature file sets it is recommended to make the first synchronization only for a few selected scenarios tagged with a particular tag. You can even temporarily tag a few scenarios in order to test the synchronization. In our example we assume that the selected scenarios are tagged with @specsync_test.

dotnet specsync push --tagFilter @specsync_test
SpecSync4AzureDevOps.cmd push --tagFilter @specsync_test
<EXTRACTED-SPECSYNC-FOLDER>/SpecSync4AzureDevOps push --tagFilter @specsync_test

The synchronization should run successfully and SpecSync should create Test Cases in Azure DevOps for the selected scenarios and link to them using a tag.

Setup SpecFlow plugin to support "Test Suite based execution with Scenario Outline wrappers" automation strategy for .NET SpecFlow or Reqnroll projects

SpecSync can synchronize scenarios to Azure DevOps Test Cases and publish test execution results without any additional NuGet package.

If your project is not configured to use the "Test Suite based execution with Scenario Outline wrappers" automation strategy, the plugin will not have any impact (can be safely removed).

Package to be used for SpecFlow

SpecFlow version
SpecFlow plugin NuGet package

v2.3

v2.4

v3.0

v3.1

v3.3

Package to be used for Reqnroll

Plugin version
Reqnroll Compatibility
SpecSync Compatibility

v3.4.*

2.*

3.4.*

The init command will ask a few questions in order the setup the connection to the Azure DevOps project, like the project URL and the authentication details. If you are unsure about what is exactly your Azure DevOps project URL, please check the page. You can fine more information about the authentication options in the page.

The init command configures the settings that are enough for the first synchronization. SpecSync comes with useful default values for all settings, but you can still review the and extend the if you need to. It is recommended to think about whether you would like to or consider .

To use the command open a command window or bash shell and change the current directory to the root folder of the feature file set and invoke the following command. For all command line option of the push command, please check the .

In case of errors, you can retry the synchronization with an additional --verbose option that displays additional diagnostic information. If this does not help, please check the page or contact .

There is a special way to publish test results that require you to install a SpecFlow plugin as a NuGet package. This special way, the "Test Suite based execution with Scenario Outline wrappers" automation strategy is described in detail in the that describes the conditions when this strategy has to be used.

The exact NuGet package to be installed depends on the SpecFlow version of your project. For example for SpecFlow v2.4.*, install . The following table contains the supported SpecFlow versions and the related plugin packages.

For Reqnroll we provide a single package: that can be used with Reqnroll 2.1 or later. You might need to use a different version of the plugin depending on the SpecSync and Reqnroll version you use.

hierarchical configuration files
What is my Azure DevOps project URL
Azure DevOps authentication options
SpecSync features
configuration
exclude any scenarios from the synchronization
including all Azure DevOps Test Cases synchronized by SpecSync into a Test Suite
Features
Configuration reference
command line reference
Troubleshooting
SpecSync Support
SpecSync.AzureDevOps.SpecFlow.2-4
SpecSync.AzureDevOps.TestSuiteBasedExecution.Reqnroll
SpecSync.AzureDevOps.SpecFlow.2-3
SpecSync.AzureDevOps.SpecFlow.2-4
SpecSync.AzureDevOps.SpecFlow.3-0
SpecSync.AzureDevOps.SpecFlow.3-1
SpecSync.AzureDevOps.SpecFlow.3-3
documentation