Getting started using Cucumber or other Gherkin-based BDD tool

This chapter goes through the setup and the synchronization steps for non-SpecFlow projects. For SpecFlow projects, please check page Getting started using SpecFlow.

SpecSync can synchronize any scenarios that are written in Gherkin format. Gherkin format is used by many tools in many platforms, like Cucumber, Cucumber JVM, Cucumber.js, Behat, Behave and also SpecFlow.

SpecSync can also be used to publish test results of various Cucumber projects including Cucumber Java, Behave and SpecFlow so that Jira can track the historical test executions and provide you with various reports. To be able to publish the test results, the publish-test-result command has to be used.

The SpecSync synchronization tool can be executed as a command line tool from Windows, macOS and Linux-based systems. See Installation & Setup and Using SpecSync on macOS or Linux pages for details.

In this guide we will use Cucumber.js as an example, but the steps can also be applied for other tools as well.

Preparation

For setting up SpecSync for Jira, you need a Cucumber project and a Jira project. For the supported Jira versions, please check the Compatibility list.

In our guide, we will use a calculator example (my_calculator) that uses Cucumber.js v5.1. The sample project can be downloaded from GitHub.

For a synchronization target we use a Jira project: https://specsyncdemo.atlassian.net/MyCalculator.

Installation

Install SpecSync using one of the available installation options on your operating system. In this guide we assume you have downloaded one of the native binaries and it to a folder on your system.

Set an environment variable SPECSYNC_DIR to the folder where the SpecSync executable (SpecSync4Jira) is extracted to (depending on the package it might be the tools folder of the extracted zip file).

Initialize configuration

Initialize the SpecSync configuration in your local repository root by invoking the SpecSync init command. This command will create a SpecSync configuration file specsync.json.

$SPECSYNC_DIR/SpecSync4Jira init

The init command will ask you for your Jira project URL and the authentication credentials. Alternatively you can manually create the configuration based on an empty configuration file downloadable from http://schemas.specsolutions.eu/specsync-jira-empty.json.

Basic configuration

The init command configured the connection details to your Jira project, but there are a few other things that has to be configured before the first synchronization.

In our sample application the feature files are located at the test/features folder. We need to specify this in the local section of the configuration file.

Before the first synchronization we have to review and change a few settings in this file. For this example we will synchronize all feature files within the test/features folder. For synchronizing only a specific set of feature files, please check the local Configuration documentation.

1. Open the specsync.json file in your IDE (e.g. Visual Studio Code) from your project folder.

2. Set the value of the local/featureFileSource/type setting to folder and the local/featureFileSource/folder setting to test/features. This will instruct SpecSync to process the feature files from that specific folder.

If feature file source is not specified, SpecSync will synchronize the feature files from the folder of the configuration file (including subfolders).

The specsync.json file after basic configuration has been set

specsync.json

{
  "$schema": "http://schemas.specsolutions.eu/specsync4jira-config-latest.json",

  // See configuration options and samples at http://speclink.me/specsyncconfig.

  "remote": {
    "serverUrl": "https://specsyncdemo.atlassian.net/",
    "project": "MyCalculator",
    "user": "52yny........................................ycsetda"
  },
  "local": {
    "featureFileSource": {
      "type": "folder",
      "folder": "test/features"
    }
  }
}

First synchronization

1. Make sure your project compiles and runs.

2. We recommend starting from a state where

   • all tests pass,

   • the modified files are checked in to source control.

3. Open a command line prompt and navigate to the project root folder

4. Invoke SpecSync push command:

   $SPECSYNC_DIR/SpecSync4Jira push

5. If you haven't specified any credentials in the configuration file, an authentication dialog will popup, where you have to specify your credentials for accessing the Jira project.

As a result, the scenarios from the project will be linked to newly created Jira test cases, and you will see a result like this.

Result of the first synchronization

Scenarios are synchronized to normal, Scenario Outlines to parametrized Test Cases.

Check Test Case in Jira

1. Find one of the created test case in Jira.

There are a couple of things you can note here.

• The name of the scenario has been synchronized as the title of the test case. (The "Scenario:" prefix can be omitted by changing the synchronization format configurations.)
• The tags of the scenario have been synchronized as test case tags (labels).

• The steps of the scenario have been synchronized to the test case depending on the used Jira TCM solution. In most of the integrations it is synchronized as Gherkin text.

Verify feature file and commit changes

1. Open one of the feature files from in the IDE. SpecSync modified the file and added a few tags.

2. Each scenario and scenario outline has been tagged with a @tc:... tag making the link between the scenario and the created test case.

@tc:P01-T1234
@important
Scenario: Add two positive numbers

The feature files are changed only when synchronizing new scenarios (linking). To avoid file changes (e.g. when running the synchronization from a build server) the --disableLocalChanges command line switch can be used. See Synchronizing test cases from build for details.

Verify if the project still compiles and the tests pass (they should, since we have only added tags), and commit (check-in) your changes.

Synchronize an update

Now let's make a change in one of the scenarios and synchronize the changes to the related test case.

1. Update the title and the steps of the scenario, for example change the scenario Add two positive numbers to Multiply two positive numbers, change add to multiply in the When step and update the expected result to 377:

   @tc:P01-T1234
   Scenario: Multiply two positive numbers
     Given I have entered the following numbers
        | number |
        | 29     |
        | 13     |
     When I choose multiply
     Then the result should be 377

2. Make sure it still compiles and the test passes.

3. Run the synchronization again:

   $SPECSYNC_DIR/SpecSync4Jira push

The result shows that the test case for the scenario has been updated, but the other test cases have remained unchanged (up-to-date).

Result of synchronizing an update

1. Refresh the test case in your browser to see the changed title and steps.

   

   Updated test case in Jira

Publishing test results (optional)

So far we have shown how to keep the Jira Test Cases in sync with the scenarios in the feature files. This provides you easily accessible documentation and traceability. The synchronized Test Cases can also be executed as manual tests from Jira.

To turn the Test Cases into a living documentation, you can also publish test results of the scenarios connected to the Test Cases.

For that you need to execute the tests and use the publish-test-results command of SpecSync to publish these results to Jira. SpecSync supports several test result file formats and BDD tools. The list of currently supported tools can be found on the Compatibility page.

The list of supported formats are constantly growing and you can also create a SpecSync plugin to support your own test result format.

The following example shows how to publish a Cucumber Java test result file.

dotnet specsync publish-test-results --testResultFile mytestresult.xml 
  --testResultFileFormat cucumberJavaJUnitXml

The command analyses the test result file and connects the results with the scenarios to be able to associate them to Test Cases.