Changelog

The How to upgrade to a newer version of SpecSync guide contains information about how to upgrade to a newer version.

For planned features in future releases please check the Release Model and Roadmap page.

v1.3.2 - 2024/03/06

Improvements

  • Support for Reqnroll, a fork & reboot of the SpecFlow project (#1272)

  • Allow specifying authentication method explicitly in configuration (#1265)

  • Display a note in the verbose log if an environment variable was resolved to empty because it did not exist (#1270)

  • Allow using {env:ENV_VAR} placeholders in authentication related configuration settings (#1269)

  • Various stability and maintainability improvements (#1252, #1254, #1256, #1255)

v1.3.1 - 2023/12/05

Improvements

  • Support for .NET 8: The .NET tool installation now works on systems that only have .NET 8.0 SDK. (#1239)

  • Warning when SpecSync is used with .NET Core 3.1 (#1243)

  • NUnit Plugin v1.0.2: Allow loading NUnit XML (v2, v3) test result files (#1232)

  • Support for step-level results with Python Behave XML result files (#1226)

  • Various stability and maintainability improvements (#1224, #1230, #1240, #1242)

Bug fixes

  • Fix: xUnit test result loading error for wider scenario outline examples tables when using xunit.runner.visualstudio v2.5 (#1236)

  • Fix: Configuration load error: "Unable to find a constructor to use for type SpecSync.Configuration.ConfigurationWarning" when --configOverride is used (#1238)

  • Fix: NUnit Plugin: NullReferenceException when loading NUnit XML v3 result files (#1231)

  • Fix: Test result files are processed multiple times with --configOverride, --testRunSetting or --testResultSetting command line options used (#1247)

  • Fix: Jira.ZephyrScale: publishing multiple test executions for the same test might fail with attachment error E0018 (#1248)

  • Fix: Jira.ZephyrScale: error when result attachment is published for a non-existing iteration instead of warning (#1249)

v1.3.0 - 2023/03/01

New features

  • Zephyr Scale: Support for setting version & iteration field on test results (#1223)

  • Interactive conflict resolution when both the local and the remote test case has changed. The change provides an improved resolution for pull and the interactive resolution can also be used (optionally) for push. You can configure the conflict resolution method separately for push and pull and you can choose from methods: forceOverride, interactive, error, skip, ignore. Choosing for the error method can be also used to get notified about conflicting changes on the server. With the --force command line option, the changes can be overridden independently of the configured method. See Synchronization conflict resolution for details. (#613)

  • Additional template placeholders for accessing remote server related settings ({remote-project-url}, {remote-server-url}, {remote-project-name}). See full list of available placeholders in the field updates reference. (#1024)

  • Local test case conditions can be filtered for test name using $name = 'My scenario' syntax. The name filter can be combined with tag or source file filter as well, e.g. $sourceFile ~ **/Addition.feature and $name = 'Basic addition' This applies to all condition configuration settings and can also be used with the -tagFilter command line option. See all possibilities for local test case conditions in the feature description. (#1025)

  • Introducing the generic --filter command line option that can be used to filter for both tags, names and source files. E.g. --filter '@basic and $sourceFile ~ **/Addition.feature and $name = "Basic addition"'.

  • Allow not synchronizing Test Case title. See Customization: Do not synchronize title for details. (#1081)

  • Errors are reported with an error ID that can be shared with support and used to search solutions in the documentation (#1082)

  • SpecSync Feed: SpecSync can load and display important service outages, hints and tips (#1037)

New Plugins

See all plugins available on nuget.org in the plugin list

Improvements

  • Zephyr Scale: Support for step-level attachments in test results (#1207)

  • Display unique ID for warnings. Now all warnings can be better identified with a code, like W5133. The next major release (v3.5) will provide further features for fine-grain handling of warnings. (#1210)

  • Reminder of using preview release that is released more than 90 days ago. Do not allow using preview releases released more than 180 days ago. (#1202)

  • Support "Ignore non-supported local tags customization" for pull. The not supported tags are not removed. (#1107)

  • List test case synchronization errors at the end, like warnings (#1086)

  • Report output warnings and errors in a way that is recognized by Azure DevOps pipeline as error/warning (#1039)

  • Improvements for plugin development:

    • Allow plugin Test Result Loaders to set DataRow on the test result (#1209)

    • Base classes for REST API connections (#1017, #1023)

    • More flexibility for JUnitXml parsers (#1018, #1031)

    • More flexibility for JsonFileWriter (#1019)

    • Allow registering value loaders (#1020)

    • Allow deserializing parameters (#1021)

  • Better error messages for errors reported as "unhandled" (#1149, #1155, #1152, #1157, #1151, #1154, #1153, #1156)

  • Various stability and maintainability improvements (#995, #1000, #1013, #1033, #1035, #1089, #1078, #1067, #1040, #1159, #1148, #1147, #1140, #1204, #1211, #1170, #1172, #1206, #1215, #1206, #1215)

Bug fixes

  • Fix: Zephyr Scale: JsonSerializationException with Jira Zephyr Scale version 9.15.3

  • Fix: Jira TCM Customization: Test case links might be removed (#1218)

  • Fix: Jira TCM Customization: Jira Test Case might not be updated when scenario changed (#1217)

  • Fix: NUnit v2 XML test result "Failure" is not recognized (#1001)

  • Fix: Using tags with different casing might cause unnecessary updates (#936)

  • Fix: Config source types might be calculated incorrectly (#1015)

  • Fix: PyTest: Incorrect outcome published for dynamically skipped tests (#1027)

  • Fix: The error message does not contain the file name if test results are specified with a folder name, the format is not specified and there is a non-TRX file in the folder (#1070)

  • Fix: Improve error handling of various unhandled errors (#1085)

  • Fix: Not supported local tags are removed during pull if they are used in fieldUpdates (#1119)

  • Fix: Pull removes tags that have been handled by field updates (#1111)

  • Fix: IndexOutOfRangeException when Scenario Outline is pulled without data iterations (#1120)

  • Fix: JUnitXml: Test Cases (represented as <testsuite>) with failing steps (represented as <testcase>) should be treated as failed, even if the last one passed (#1083)

  • Fix: Unhandled error when Gherkin file contains comments in description (#1157)

v1.2.0 - 2023/03/01

New features

  • In all expressions where you can specify a local test case condition (synchronization/fieldUpdates[]/condition, synchronization/fieldUpdates[]/conditionalValue/*), you can now also specify source file path conditions as well using the $sourceFile ~ path/**/myfeature.feature format. E.g. the following expression selects scenarios that are tagged with @important and are in the pricing folder or any sub-folders below: @important and $sourceFile ~ pricing/. See Local test case conditions for details. (#878)

  • SpecSync plugins can now be loaded from NuGet packages that are distributed through nuget.org, custom NuGet feeds or from local folders. See SpecSync plugin documentation for examples and further details. There is also a list of plugins available on nuget.org. (#814)

  • Allow skip publishing test results for local test cases using the --tagFilter and --sourceFileFilter options. (#853)

  • Support Cucumber Java Surefire XML report (#795)

  • Support for SpecFlow v4.0 (#901)

  • Support for .NET 7: The .NET tool installation now works on systems that only have .NET 7.0 SDK. (#868)

New Plugins

See all plugins available on nuget.org in the plugin list

  • SpecSync.Plugin.ScenarioOutlinePerExamplesTestCase: Plugin that synchronizes scenario outline examples blocks as individual test cases. See plugin details and usage in the SpecSync plugins GitHub repository (#718)

  • SpecSync.Plugin.ExcelTestSource: This plugin can be used to synchronize a local test cases from Excel file using the format that Azure DevOps uses when you export Test Cases to CSV. See plugin details and usage in the SpecSync plugins GitHub repository

  • SpecSync.Plugin.MsTestTestSource: Allows synchronizing "C# MsTest Tests" and publish results from TRX result files. See plugin details and usage in the SpecSync plugins GitHub repository

  • SpecSync.Plugin.NUnitTestSource: Allows synchronizing "C# NUNit Tests" and publish results from TRX result files. See plugin details and usage in the SpecSync plugins GitHub repository (#962)

  • SpecSync.Plugin.GenericTestResultMatcher: A SpecSync plugin that can be used to override test result matching rules. See plugin details and usage in the SpecSync plugins GitHub repository

Deprecation notices

  • The configuration setting publishTestResults/ignoreNotExecutedTests has been deprecated as ignoring test results with not executed outcome is the default now. (In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used.) (#894)

  • The configuration settings local/featureFileSource/type, local/featureFileSource/filePath and local/featureFileSource/folder are moved to the local group and renamed to local/projectType, local/projectFilePath, local/folder. The old settings work, but show a warning. (#873)

  • The .NET 5 framework is out of support and will not receive security updates in the future (see https://aka.ms/dotnet-core-support). SpecSync versions released after 31/3/2023 will not run with .NET 5. Please use SpecSync with .NET 6 or any of the other supported platforms. (#919)

  • The configuration setting synchronization/link[]/targetWorkItemType has been renamed to synchronization/link[]/targetType and SpecSync now verifies if a valid issue type has been specified when linking. The old setting works, but shows a warning. The additional verification is performed even if the old setting was used. (#822)

  • The configuration setting customizations/synchronizeLinkedArtifactTitles/linkLabelSeparator has been moved to synchronization/linkLabelSeparator. The old settings work, but show a warning. (#850)

Breaking changes

  • The test results with not executed outcome are not published by default. In order to force including them (the earlier behavior), the publishTestResults/includeNotExecutedTests setting can be used. (#894)

  • The default link-label separator has been changed from : to ;. To continue using : as link label separator, please set synchronization/linkLabelSeparator. (#861)

  • The tag prefixes specified in synchronization/links[]/tagPrefix are ensured to contain only word characters (letters, numbers, underscore, dash). (#849, #930)

  • Tag predicates in local test case conditions must start with '@'. Earlier tag predicates were also accepted, but from this version the @ has to be used, even if the local test case does not use this prefix for tags. (#877)

  • The value specified as synchronization/link[]/targetWorkItemType (now renamed to synchronization/link[]/targetType) is used now to verify if the type of the linked issue. The verification is performed when establishing new links. To disable the verification and allow linking any issue types please remove this setting. (#822)

  • The "pull" command now adds and removes local test case tags based on the links of the Test Case. (#863)

Improvements

  • Allow newline ({br}) and environment variable ({env:MY_ENV}) placeholders in run/result parameters (#957)

  • Handle TAB characters for C# test sources (#950)

  • No harmful change should be performed when the license file is not specified, but the number of scenarios exceed the free license limit (#944)

  • Use provided file name for Cucumber JSON attachments (#960)

  • Include suite name in the "unable to create suite" error message. (#933)

  • The user-specific configuration file got a separate JSON schema: https://schemas.specsolutions.eu/specsync-user-config-latest.json (#859)

  • The type of the linked issues can be restricted using the synchronization/link[]/targetType or the customizations/linkOnChange/links[]/targetType configuration settings. (#822, #867, #866)

  • Filter, scope and other conditions are satisfied if there is at least one scenario outline examples that satisfies the condition. (#717)

  • Improve performance of filtered / scoped command executions (#856)

  • Detecting non-interactive user session for operations require user input and automatically cancel to avoid blocking process (#891)

  • Improved publish-test-results reporting (#915)

  • The test results with not executed outcome (NotExecuted, NotApplicable, NotRunnable, NotImpacted) are not published by default as they might hide earlier results of the Test Case. In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used. (#894, #893)

  • The product name (SpecSync for Azure DevOps or SpecSync for Jira) is visible in the license file when opened in a text editor. (#876)

  • SpecSync NuGet packages are signed with Code Signing Certificate of Spec Solutions Kft., thumbprint: dd5e69f05ef38c016508380ca0d2294dbbe1ba69 (#882)

  • JiraTCM: Perform temporary change for the summary field to be able to track local hash when only relations changed (#676)

  • Improve error message for unknown file source (#977)

  • Detect Base64 text embeddings in CucumberJson results (#991)

  • Various stability and maintainability improvements (#691, #973, #955, #942, #911, #837, #808, #898, #880, #874, #887, #981, #982, #995, #981, #982)

Bug fixes

  • Fix: Environment variables are not allowed anymore for local/featureFileSource/filePath and local/featureFileSource/folder (#947)

  • Fix: Text file attachments ("text/plain" mime type) are included in STDOUT instead of publishing as attachment with Cucumber JSON format (Cypress) (#959)

  • Fix: Attachments from before/after hooks are not published with Cypress (#958)

  • Fix: Incorrect Gherkin might be produced when Test Case step action or expected result field contains multiple lines (#923)

  • Fix: Custom field with value null is overridden unnecessarily to '' (#934)

  • Fix: Test Case might be updated always although it is up-to-date (#935)

  • Fix: Wait on stats calls block execution (#914)

  • Fix: Plugins cannot be loaded with console app (#910)

  • Fix: File paths in tags are not normalized on Linux and macOS (#912)

  • Fix: Different hash is calculated when running on Windows or macOS/Linux causing unnecessary updates (#881)

  • Fix: Plugins might be reported twice (#864)

  • Fix: Scenario test results of pytest-playwright are not found (#917)

  • Fix: Cucumber JUnit XML report does not find additional scenario outline examples with recent Cucumber version (#796)

  • Fix: Cucumber JSON scenario outline results are incorrectly detected as re-runs (#904)

  • Fix: Cucumber test results might not be detected when scenario outline placeholder is used in the title (#472)

  • Fix: Configuration files are reported twice when configOverride is used (#890)

  • Fix: SpecSync might ask for password when version command is used (#848)

  • Fix: SpecSync Plugins: NoKeywordParser produces invalid TestSourceData (#791)

  • Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)

  • Fix: Gherkin comments are not allowed in description (#978)

  • Fix: Unhandled error when Scenario Outline examples list contains column multiple times (#996)

  • Fix: Using tags with different casing might cause unnecessary updates (#936)

  • Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)

v1.1.1 - 2022/11/14

Improvements

  • The SpecSync Docker images are now based on the ubuntu:22.04 image and the Linux binaries are compatible with Ubuntu 22.04 and do not depend on the ICU packages. (#696)

Bug Fixes

  • Fix: Cannot authenticate to Jira server with Personal Access Token (PAT) (#858)

  • Fix: SpecSync version command shows missing config error when there is no configuration file (#860)

v1.1.0 - 2022/10/31

New features

  • Support for Jira Server/Data Center - SpecSync can now also be used for Server or Data Center installation of Jira with the Jira TCM Customization solution.

  • Support for Zephyr Scale on Jira Server/Data Center - The support requires minimal configuration and supports both Test Case synchronization (push, pull) and test result publishing. (#767, #774)

  • Support updating Test Case fields that are not updated automatically by SpecSync using the fieldUpdates configuration. See feature description for details. (#737)

Breaking changes

  • In the remote configuration section the server access has to be specified with a separate serverUrl and project setting instead of a single projectUrl. The projectUrl setting is deprecated but still accepted by SpecSync.

v1.0.1 - 2022/04/05

Initial version supporting Jira Test Case management.

PreviousUsing SpecSync inside a Docker containerNextRelease Model and Roadmap

Last updated