Changelog

The How to upgrade to a newer version of SpecSync guide contains information about how to upgrade to a newer version.

For planned features in future releases please check the Release Model and Roadmap page.

v5.0.0 - 2025/12/12

All fixes up to SpecSync for Jira v1.3.4 are included in this release.

For detailed instructions on how to upgrade please check the Migrating from SpecSync v1 to v5 guide.

New features

  • SpecSync switches to semantic versioning. (#1233)

    • From this version, the major version number (currently 5) represents compatibility range and breaking changes.

    • The minor version number (currently 0) always denotes non-breaking changes and feature additions.

    • The patch number will only be used for release hot-fixes.

    • The SpecSync plugins will be versioned according to the main version: a plugin version 5.4.1 is compatible with any SpecSync v5.* versions.

    • Both SpecSync for Azure DevOps and SpecSync for Jira will use the same versioning scheme and the plugins are cross-compatible between these products.

    • We recommend to always upgrade to the latest minor and patch version within the same major version.

  • Configuration upgrade wizard: The specsync upgrade command analyzes your configuration file, applies the necessary changes for deprecated settings and offers configuring new features. The upgrade command should be used primarily for major version upgrades, but can be re-invoked any time to configure optional features. See Configuration wizards for details. (#1279)

  • Improved integration to Azure DevOps pipelines:

    • When a SpecSync command fails, it automatically fails the pipeline task without using non-zero exit codes. This produces cleaner error messages. The non-zero exit codes can be forced with --forceExitCode command line option. (#1110)

    • SpecSync v5 does not fail the Azure DevOps pipelines automatically for warnings, but registers the warning to the pipeline result instead. If you want to fail the pipelines for warnings, use the --treatWarningsAsErrors option. (#1110)

    • The log file produced by the --log command line option is automatically included to the pipeline build logs. (#1216)

    • The pipeline output log uses colors

  • There are new options to maintain "remote scope". Remote scope is a piece of information in Jira that is used to detect if a local test case has been removed. Configuring a remote scope is optional, but if it is not specified SpecSync will not be able to detect removed local test cases and could not mark the related Test Cases and remove them from the suite hierarchies. The remote scope is now configured using the remote/scope configuration setting. See Remote scope for details. There are more options now to use remote scope: (#1282, #1619)

    • Configure a tag remote scope that uses a tag specsync:<configuration-key> to know if a Test Case was managed by SpecSync. (#588)

  • The SpecSync configurations can be identified with a "configuration key". The configuration key can be specified in the configurationKey setting of the configuration file. The configuration key is included in the Test Case change comments, so SpecSync is able to detect and provide a warning by default when a Test Case is synchronized from multiple SpecSync configurations. This behavior can be changed by the synchronization/push/configurationConflictResolution setting. The configuration key is also used for marking Test Cases of deleted local test cases (scenarios) and also by tag remote scope. Specifying a configuration key is optional, but recommended. See Configuration key for details. (#1387)

  • The command line options supports completion using the "dotnet-suggest" tool. See How to enable command line completion for details. (#1487)

  • Allow suppressing warnings with either the toolSettings/suppressWarnings configuration setting. You an specify warning codes in an array, e.g. "suppressWarnings": ["W5113", "W5126"] (#1158)

  • Support for license keys as an alternative of license files. The license key can be specified in the toolSettings/license configuration setting or with the --license command line option. The configuration setting supports loading the value from environment variable with {env:ENV_VAR}, but SpecSync will automatically load the license key from the environment variable SPECSYNC_LICENSE_KEY. You can use the specsync upgrade command to convert your license file to license key, but you can also ask support to get your license key. License files are still supported. (#1266, #1758)

  • Allow treating warnings as errors for greater security by setting the toolSettings/treatWarningsAsErrors setting to true or using the --treatWarningsAsErrors command line option. (#1477)

  • The publishTestResults/publishAttachmentsForPassingTests configuration setting controls which attachments should be published for passing tests. In the prior releases both the files attached to the test results (e.g. screenshots) and the test output were attached to the result. Publishing the test output attachments causes performance issues therefore, you can control now which attachments of the passing tests should be published. The possible options are none - no attachments are published for passing tests, files - only file attachments are published (default), all - both file attachments and test output are attached. In prior releases all attachments were published, so the default behavior is a breaking change. Note: this setting and the changes of defaults does not affect publishing failing tests. (#1485)

Improvements

  • Allow glob pattern syntax for test result file names (--testResult, -r option) (#1071)

  • Improve logging to better explain why test results are filtered out for publish-test-results command (#1588)

  • Improve usability of file pattern conditions by hinting potential causes for not processing any local test cases (#1262)

  • Automatically enable verbose mode if --diag is used, except for timing diag category (#1524)

  • Console options that supported multiple values and so far allowed semicolon-separated list now allow using the options multiple times. The semicolon-separated value list is still supported. Affected options: --testResult, -r, --diag, --attachFile, --hierarchy (#1487)

  • Improved error handling if Git URL is used for project URL. (#1484)

  • Add error codes for all errors (#1476)

  • Allow using {test-case-id} placeholder in field updates and test result settings. (#1358)

  • The repeating step keywords in synchronized Test Case steps can be normalized to the first step keyword by setting the synchronization/format/normalizeRepeatingStepKeywords setting to true. For example, if a Given step is followed by another Given step, the second step will be changed to And. (#1622)

  • The --testRunSetting and --testResultSetting command line options are now available for publish-test-result command. (#974)

  • Added skipWithWarning and overwriteWithWarning conflict resolution methods, skip will not report a warning (#1395, #1391)

  • The support for wildcard conditions has been extended to multiple wildcards and wildcards at any position in a single predicate (so far only a single tail wildcard was supported). For example @env-*-db-* matches @env-prod-db-mysql. The literal * characters in tags has to be escaped as \*. (#1600)

  • Improve display of warning summary (#1393)

  • Detect when local and remote test case have been updated in the same way and reset hash (#1389)

  • Include configuring Test Case hierarchy synchronization and remote scope in the configuration wizards (init, upgrade). (#1388, #1386)

  • Improved coloring of console output (#1385)

  • Improved performance of Test Suite operations (#1320)

  • Text values can be converted to fields that accept Markdown (e.g. test result comment) using the TextToMarkdown value loader. (#1475)

  • Creating new scenarios from Test Cases is now also available for Jira. In order to use the feature, set the synchronization/pull/enableCreatingNewLocalTestCases configuration setting to true. (#1620)

  • The dependency to the deprecated BoDi NuGet package has been removed. (#1786)

  • The Cucumber Messages support has been updated to support Cucumber Messages v31. (#1783)

  • The Gherkin support has been updated to support Gherkin v37. (#1783)

  • Various stability and maintainability improvements (#1548, #1587, #1588, #1535, #1534, #1533, #1532, #1525, #1511, #1512, #1480, #1467, #1468, #816, #1205, #1394, #1384, #1280, #1513, #1797, #1795, #1526, #1765, #1667, #1739, #1738, #1656)

  • Plugin API improvements (see details below) (#1752, #1769, #1463, #1162, #1751, #1790, #1161, #1321, #1502, #1711)

Deprecation notices

The following settings are still usable with the new version but provide a warning. The deprecated settings will be removed in the next major release.

Most of the deprecated settings are automatically fixed by invoking the new specsync upgrade command (see Configuration wizards – Upgrade).

  • The configuration setting customizations/customFieldUpdates has been replaced by synchronization/fieldUpdates. See Update Test Case fields feature description for details. (#1589)

  • The configuration setting local/tags has been renamed to local/condition (#1529)

  • The configuration settings publishTestResult/testResult/filePath and publishTestResult/testResult/files have been replaced by publishTestResults/testResults/sources (#1462)

  • The configuration setting publishTestResult/testResult/fileFormat has been replaced by publishTestResults/testResults/resultFormat (#1462)

  • The configuration setting customizations/synchronizeLinkedArtifactTitles has been renamed to customizations/synchronizeLinkedResourceTitles (#1526)

  • The configuration setting local/defaultFeatureLanguage has been renamed to local/defaultDocumentLanguage (#1526)

  • The configuration setting synchronization/pull/enableCreatingScenariosForNewTestCases has been renamed to synchronization/pull/enableCreatingNewLocalTestCases (#1602)

  • The ignore and forceOverride conflict resolution method has been replaced with with overwrite (#1392, #1390)

  • The command line option --diagCategories has been replaced by --diag (#1523)

  • The command line option --tagFilter has been replaced by --filter (#1491)

  • The command line option --attachedFiles has been replaced by --attachFile (#1495, #1657)

  • The command line option --testResultFile has been replaced by --testResult (#1461)

  • The command line option --testResultFileFormat has been replaced by --testResultFormat (#1461)

  • The command line option --testPlanId has been replaced by --testPlan. With the --testPlan option you can specify ID values using the #1234 format. (#1459)

  • The command line options --buildId, --buildNumber, --buildFlavor, --buildPlatform have been removed. Use --testRunSetting buildId=123 (and buildNumber, buildFlavor, buildPlatform) instead. (#1460)

  • When SpecSync is used with .NET 6 or .NET 7, a warning is displayed. The official support of these frameworks have been ended by Microsoft so it is not recommended to use SpecSync with these. Please note that the .NET framework used for SpecSync does not have to be the same as the .NET framework of the synchronized project. (#1471)

Breaking changes

  • The SpecSync command line execution support for .NET 3.1 and .NET 5 has been removed. The official support of these frameworks have been ended by Microsoft so it is not recommended to use SpecSync with these. To continue using these frameworks, please use SpecSync

1.3. Please note that the .NET framework used for SpecSync does not have to be the same as the .NET framework of the synchronized project. (#872) * SpecSync v5 does not fail the Azure DevOps pipelines automatically for warnings, but registers the warning to the pipeline result instead. If you want to fail the pipelines for warnings, use the `--treatWarningsAsErrors` option. * The default behavior for publishing attachments for the passing tests has been changed (see #1485). In the prior releases both the files attached to the test results (e.g. screenshots) and the test output were attached to the result. According to the new default, only file attachments are published for passing tests. The behavior can be controlled by the `publishTestResults/publishAttachmentsForPassingTests` setting. The setting `all` represents the previous default. *Note: this setting and the changes of defaults does not affect publishing failing tests.* (#1485) * The exit code of the command line tool for errors is not always `10` anymore, but in range between `10` and `19`. (#1478) * In previous releases when a Test Suite name was used without specifying a Test Plan, SpecSync used the Test Suite from the first Test Plan it found. This has been changed to search through all Test Plans and report an error if the Test Suite name is not unique. To avoid name collisions, it is recommended to specify the Test Plan and refer to the Test Suite by `path`. (#1474) * The `name` setting of the Test Suite specification has been replaced by `titleSearch` to avoid confusion. If the Test Suite is a top-level suite in the Test Plan, consider using the `path` setting instead. (#1724) * The escaping behavior in conditions have changed: the escape character (`\`) escapes only a specified set of characters (`\`, `(`, `)`, `;`, `'`, `"`, ` `, `*`) instead of all characters. This helps for specifying Windows paths in conditions. (#1473) * The literal `*` characters of tags in conditional has to be escaped as `\*`, because now they are treated as wildcard. (#1600) * The configuration settings that have been deprecated earlier are removed now: `publishTestResults/ignoreNotExecutedTests`, `local/featureFileSource/*`, `synchronization/link[]/targetWorkItemType`, `toolSettings/testCaseWorkItemName`, `toolSettings/testSuiteWorkItemName`, `customizations/synchronizeLinkedArtifactTitles/linkLabelSeparator`, `publishTestResults/createSubResults`, `publishTestResults/runName`, `publishTestResults/runComment`, `publishTestResults/runType`, `publishTestResults/testResultComment` (#1457) * The support for including the newly created feature files to legacy non-SDK style .NET projects has been removed. The created feature files has to be included to the project manually. This does not impact usual, SDK-style .NET projects. (#1755)

Plugin Improvements

  • All plugins have been updated to be compatible with SpecSync v5. According to the new versioning policy, the plugins that are compatible with SpecSync v5 have their major version set to 5. Please make sure to update to the latest plugin versions. See plugin list for details. (#1547)

Plugin API improvements

The plugin API has been significantly changed to reflect the improvements related to synchronizing requirement issues, non-BDD projects and projects that use source document that are not physical files (e.g. a Postman collection). The official plugins have been upgraded to v5 already. If you need help with upgrading your custom plugins, please contact us.

The main changes are:

  • The "BDD Project" concept has been replaced by "Sync Project" or "Project" in short. The Sync Project represents the project that is synchronized by SpecSync with a SpecSync configuration file. The sync project is typically a folder that contains the source documents to be processed (e.g. Feature Files).

  • The "Source File" concept (e.g. a file path within a project to be processed) has been replaced by "Source Reference", the "Source File" is a specialization of "Source Reference".

  • The "Local Test Case Container" concept (e.g. a Feature File) has been replaced by "Source Document".

  • The common term for local test cases (e.g. Scenarios) or local requirements (e.g. a Rule section of a feature file) is now "Local Artifact" and "Local Test Case" and "Local Requirement" are specializations of it.

  • The generic term "Resource" has been introduced to represent any remote artifact (e.g. issue, pull request, URL) that can be linked from a local artifact.

  • The term "Managed Work Item" has been introduced to represent a remote work item (e.g. Test Case, Requirement) that is synchronized by SpecSync.

  • The data structure that represents the synchronization input is renamed to "Artifact Sync Data" from "Test Case Source Data".

  • The representation of tests results ("Local Test Run") has been consolidated. The local test runs contain "Local Test Results". The abstraction "Test Definition" has been removed and its details have been merged into "Local Test Result". If the local test result represents a parametrized test execution (e.g. scenario outline example), the concrete arguments are available as "Invocation Arguments".

  • The general processing context has been renamed to "Command Context" from "Synchronization Context" and the context specific to synchronization of an artifact is renamed to "Artifact Sync Context" from "Local Test Test Case Context".

  • The plugin API can now represent parametrized local test cases, where the parameter values are not known at synchronization time, but the parameter names are known (e.g. when a parametrized coded test receives the parameter values from an external source). The parameter values are provided later when test results are published.

  • The test results store all date/time values in DateTimeOffset class to properly represent date and time with time zone information.

  • The local test runs can store environment details and the local test results can store step level attachments and categories as well. These are not yet processed by SpecSync but the plugin API supports these.

Bug Fixes

  • Fix: Source file filter conditions might not work with mixed path separators (#1528)

  • Fixes for earlier previews:

    • Fix: Unhandled error when filter criteria ands with \ (#1527)

    • Transform underscores in hierarchy wildcard matches to spaces (#1601)

    • Allow capturing wildcards from path specifiers (#1599)

    • Allow --runComment and --testResultComment for Jira (#1458)

    • Offer 'tag' as remote scope for Jira, use it as default for init (#1281)

    • Plugins: Make TestRunTestResult.ExecutionId public (#1609)

1.3.4 - 2025/12/12

Improvements

  • Show an error when the configuration requires newer SpecSync version (#1792)

  • Detect if a plugin is not compatible with the current SpecSync version (#1522)

  • Various stability and maintainability improvements (#1787)

  • Support for .NET 10: The .NET tool installation now works on systems that only have .NET 10.0 SDK. (#1782)

  • SpecSync packages are signed with a new code signing certificate as the previous one expired. The new certificate is issued by DigiCert and stored in a HSM. The certificate thumbprint is 11fbb9b817a08bef74474f35dc2240d6ff23dc7f. (#1776)

  • Allow loading plugins released after 1/11/2025 (using the new code sign certificate) without trust acknowledgment (#1777)

  • Improve grammar of text messages (#1748)

  • The direct SpecSync package download URLs have changed and served now from https://content.specsolutions.eu/specsync/release/. The old download URLs are automatically redirected to the new ones. (#1783)

  • Support for attachments in NUnit 3 XML result files (#1727)

  • Allow specifying the behavior of how the overall outcome of flaky tests (tests that were rerun multiple times and at least one attempt failed) should be calculated. Use publishTestResults/flakyTestOutcome to configure the behavior. (See details in Publishing test result files page.) (#1665)

  • Reqnroll v3 support (#1643)

  • Support for multiple link-on-change setting for requirement customization (#1637)

  • Improve Cucumber JSON compatibility for Behave (#1605)

  • Cucumber JSON result: Improve detection of base64 embedding data (#1612)

  • Cucumber JSON result: Do not treat JSON embeddings as stdout (#1611)

  • Various stability and maintainability improvements (#1638)

Plugin Improvements

  • SpecSync.Plugin.OnlyPublishTestResults v1.1.0:

    • SpecSync.Plugin.OnlyPublishTestResults: Allow extracting ID from core fields like "TestMethod" using regex (#1714)

Bug fixes

  • Fix: The 'Search for issues using JQL' Jira API has been deprecated (#1741)

  • Fix: Duration of Python steps is calculated incorrectly when PythonBehaveJson format is used (#1663)

  • Fix: Personal Access Token (PAT) defined in the user-specific configuration file does not get overwritten with --password (#1666)

  • Fix: NUnit test parameters might not be recognized correctly with NUnit v4+ (#1644)

  • Fix: Unhandled error: "An item with the same key has already been added." in case of duplicated examples column names (#1635)

  • Fix: Python Behave: Step error messages are repeated in Jira results when Behave continue_after_failed_step option is enabled and multiple steps fail (#1604)

v1.3.3 - 2025/03/12

Improvements

  • The files and folders are processed in a usual file processing order (folders first, underscore has priority) (#1275)

  • MarkdownToHtml value loader that can be used to update custom fields from Markdown source (e.g. feature description). Usage: {feature-description:MarkdownToHtml} or {{!MarkdownToHtml}}A *Markdown* feature description: {feature-description}. The relevant Markdown extras from Markdig can be also used. (#1277)

  • Support for Scenario Outline parameter values in Behave test results (#1293)

  • Support for processing JUnit XML TestCase Properties (#1355)

  • Include app version in log file (#1314)

  • Improve error messages for filter condition errors (E1051) when used for file path filtering (#1290)

  • Improve handling invalid credentials from user config (#1285)

  • Support for .NET 9: The .NET tool installation now works on systems that only have .NET 9.0 SDK. (#1239)

  • Support for establishing non-tracked links via synchronization/links[]/disableTracking setting. With that you can disable automatic removal of links when the corresponding tag has been deleted. (#1370)

  • Force processing removals of local test cases on filter with toolSettings/forceProcessingRemovalsOnFilter. (#1373)

  • Update dependencies to address know vulnerabilities (#1375)

  • Support for Cucumber messages .ndjson result files. Use -f cucumberMessagesNdjson to load result from this files. (#351, #1509)

  • Allow publishing test results even if no matching results found. To enable this feature set publishTestResults/publishEmptyResults to true in the configuration. (#1345)

  • Improve logging of Jira file uploads (#1383)

  • Various stability and maintainability improvements (#1359, #1318, #1319, #1375, #1369)

Bug fixes

  • Fix: Unhandled exception (NullReferenceException) during synchronization of scenarios (#1366)

  • Fix: The "Processing [file]" heading is displayed twice in log (#1361)

  • Fix: Scenario outline test results are not published with Reqnroll 2.1 and MsTest (#1466)

  • Fix: The "removed" Test Cases are not included in the sync summary line (#1330)

  • Fix: Plugin versions are displayed with commit info (#1333)

  • Fix: Source file filtering does not work on Windows when * or . is used after backslash (e.g. $sourceFile ~ Features\*.feature) (#1371)

  • Fix: Unhandled error when scenario outline examples does not have a table (#1483)

New Plugins

See all plugins available on nuget.org in the plugin list

  • SpecSync.Plugin.OnlyPublishTestResults: This plugin can be used in cases when you would like to publish test results with SpecSync for Test Cases that were not synchronized by SpecSync, but created manually or with other tools. See plugin GitHub repository (#1360)

Plugin Improvements

  • SpecSync.Plugin.GenericTestResultMatcher v1.2.0 (since v1.1.0):

    • Allow matching standard console output (StdOut) by general matcher plugin (#1286)

    • Match for test result properties and allow using {test-case-id} placeholder (#1356)

  • SpecSync.Plugin.ExcelTestSource v1.1.0:

    • allow defining first step in test case row, improve flexibility, allow id prefix (#1342)

    • make column names parametrizable (#1341)

  • SpecSync.Plugin.ExcelTestResults v1.3.0 (since v1.0.0):

    • If ID matches, should not try matching other fields (#1362)

    • Support for custom outcome values (#1348)

    • Support for non-Gherkin test source (#1347)

    • Treat empty outcome as "NotExecuted" (#1346)

    • Support ID prefix (#1338)

    • Fix: ID values formatted as number are not supported (#1337)

    • Skip rows without identifier (#1336)

    • Test result format error & parameter errors are reported incorrectly (#1335)

    • Fix: Error loading ExcelTestResults plugin (#1332)

v1.3.2 - 2024/03/06

Improvements

  • Support for Reqnroll, a fork & reboot of the SpecFlow project (#1272)

  • Allow specifying authentication method explicitly in configuration (#1265)

  • Display a note in the verbose log if an environment variable was resolved to empty because it did not exist (#1270)

  • Allow using {env:ENV_VAR} placeholders in authentication related configuration settings (#1269)

  • Various stability and maintainability improvements (#1252, #1254, #1256, #1255)

v1.3.1 - 2023/12/05

Improvements

  • Support for .NET 8: The .NET tool installation now works on systems that only have .NET 8.0 SDK. (#1239)

  • Warning when SpecSync is used with .NET Core 3.1 (#1243)

  • NUnit Plugin v1.0.2: Allow loading NUnit XML (v2, v3) test result files (#1232)

  • Support for step-level results with Python Behave XML result files (#1226)

  • Various stability and maintainability improvements (#1224, #1230, #1240, #1242)

Bug fixes

  • Fix: xUnit test result loading error for wider scenario outline examples tables when using xunit.runner.visualstudio v2.5 (#1236)

  • Fix: Configuration load error: "Unable to find a constructor to use for type SpecSync.Configuration.ConfigurationWarning" when --configOverride is used (#1238)

  • Fix: NUnit Plugin: NullReferenceException when loading NUnit XML v3 result files (#1231)

  • Fix: Test result files are processed multiple times with --configOverride, --testRunSetting or --testResultSetting command line options used (#1247)

  • Fix: Jira.ZephyrScale: publishing multiple test executions for the same test might fail with attachment error E0018 (#1248)

  • Fix: Jira.ZephyrScale: error when result attachment is published for a non-existing iteration instead of warning (#1249)

v1.3.0 - 2023/03/01

New features

  • Zephyr Scale: Support for setting version & iteration field on test results (#1223)

  • Interactive conflict resolution when both the local and the remote test case has changed. The change provides an improved resolution for pull and the interactive resolution can also be used (optionally) for push. You can configure the conflict resolution method separately for push and pull and you can choose from methods: forceOverride, interactive, error, skip, ignore. Choosing for the error method can be also used to get notified about conflicting changes on the server. With the --force command line option, the changes can be overridden independently of the configured method. See Synchronization conflict resolution for details. (#613)

  • Additional template placeholders for accessing remote server related settings ({remote-project-url}, {remote-server-url}, {remote-project-name}). See full list of available placeholders in the field updates reference. (#1024)

  • Local test case conditions can be filtered for test name using $name = 'My scenario' syntax. The name filter can be combined with tag or source file filter as well, e.g. $sourceFile ~ **/Addition.feature and $name = 'Basic addition' This applies to all condition configuration settings and can also be used with the -tagFilter command line option. See all possibilities for local test case conditions in the feature description. (#1025)

  • Introducing the generic --filter command line option that can be used to filter for both tags, names and source files. E.g. --filter '@basic and $sourceFile ~ **/Addition.feature and $name = "Basic addition"'.

  • Allow not synchronizing Test Case title. See Customization: Do not synchronize title for details. (#1081)

  • Errors are reported with an error ID that can be shared with support and used to search solutions in the documentation (#1082)

  • SpecSync Feed: SpecSync can load and display important service outages, hints and tips (#1037)

New Plugins

See all plugins available on nuget.org in the plugin list

Improvements

  • Zephyr Scale: Support for step-level attachments in test results (#1207)

  • Display unique ID for warnings. Now all warnings can be better identified with a code, like W5133. The next major release (v3.5) will provide further features for fine-grain handling of warnings. (#1210)

  • Reminder of using preview release that is released more than 90 days ago. Do not allow using preview releases released more than 180 days ago. (#1202)

  • Support "Ignore non-supported local tags customization" for pull. The not supported tags are not removed. (#1107)

  • List test case synchronization errors at the end, like warnings (#1086)

  • Report output warnings and errors in a way that is recognized by Azure DevOps pipeline as error/warning (#1039)

  • Improvements for plugin development:

    • Allow plugin Test Result Loaders to set DataRow on the test result (#1209)

    • Base classes for REST API connections (#1017, #1023)

    • More flexibility for JUnitXml parsers (#1018, #1031)

    • More flexibility for JsonFileWriter (#1019)

    • Allow registering value loaders (#1020)

    • Allow deserializing parameters (#1021)

  • Better error messages for errors reported as "unhandled" (#1149, #1155, #1152, #1157, #1151, #1154, #1153, #1156)

  • Various stability and maintainability improvements (#995, #1000, #1013, #1033, #1035, #1089, #1078, #1067, #1040, #1159, #1148, #1147, #1140, #1204, #1211, #1170, #1172, #1206, #1215, #1206, #1215)

Bug fixes

  • Fix: Zephyr Scale: JsonSerializationException with Jira Zephyr Scale version 9.15.3

  • Fix: Jira TCM Customization: Test case links might be removed (#1218)

  • Fix: Jira TCM Customization: Jira Test Case might not be updated when scenario changed (#1217)

  • Fix: NUnit v2 XML test result "Failure" is not recognized (#1001)

  • Fix: Using tags with different casing might cause unnecessary updates (#936)

  • Fix: Config source types might be calculated incorrectly (#1015)

  • Fix: PyTest: Incorrect outcome published for dynamically skipped tests (#1027)

  • Fix: The error message does not contain the file name if test results are specified with a folder name, the format is not specified and there is a non-TRX file in the folder (#1070)

  • Fix: Improve error handling of various unhandled errors (#1085)

  • Fix: Not supported local tags are removed during pull if they are used in fieldUpdates (#1119)

  • Fix: Pull removes tags that have been handled by field updates (#1111)

  • Fix: IndexOutOfRangeException when Scenario Outline is pulled without data iterations (#1120)

  • Fix: JUnitXml: Test Cases (represented as <testsuite>) with failing steps (represented as <testcase>) should be treated as failed, even if the last one passed (#1083)

  • Fix: Unhandled error when Gherkin file contains comments in description (#1157)

v1.2.0 - 2023/03/01

New features

  • In all expressions where you can specify a local test case condition (synchronization/fieldUpdates[]/condition, synchronization/fieldUpdates[]/conditionalValue/*), you can now also specify source file path conditions as well using the $sourceFile ~ path/**/myfeature.feature format. E.g. the following expression selects scenarios that are tagged with @important and are in the pricing folder or any sub-folders below: @important and $sourceFile ~ pricing/. See Local test case conditions for details. (#878)

  • SpecSync plugins can now be loaded from NuGet packages that are distributed through nuget.org, custom NuGet feeds or from local folders. See SpecSync plugin documentation for examples and further details. There is also a list of plugins available on nuget.org. (#814)

  • Allow skip publishing test results for local test cases using the --tagFilter and --sourceFileFilter options. (#853)

  • Support Cucumber Java Surefire XML report (#795)

  • Support for SpecFlow v4.0 (#901)

  • Support for .NET 7: The .NET tool installation now works on systems that only have .NET 7.0 SDK. (#868)

New Plugins

See all plugins available on nuget.org in the plugin list

  • SpecSync.Plugin.ScenarioOutlinePerExamplesTestCase: Plugin that synchronizes scenario outline examples blocks as individual test cases. See plugin details and usage in the SpecSync plugins GitHub repository (#718)

  • SpecSync.Plugin.ExcelTestSource: This plugin can be used to synchronize a local test cases from Excel file using the format that Azure DevOps uses when you export Test Cases to CSV. See plugin details and usage in the SpecSync plugins GitHub repository

  • SpecSync.Plugin.MsTestTestSource: Allows synchronizing "C# MsTest Tests" and publish results from TRX result files. See plugin details and usage in the SpecSync plugins GitHub repository

  • SpecSync.Plugin.NUnitTestSource: Allows synchronizing "C# NUNit Tests" and publish results from TRX result files. See plugin details and usage in the SpecSync plugins GitHub repository (#962)

  • SpecSync.Plugin.GenericTestResultMatcher: A SpecSync plugin that can be used to override test result matching rules. See plugin details and usage in the SpecSync plugins GitHub repository

Deprecation notices

  • The configuration setting publishTestResults/ignoreNotExecutedTests has been deprecated as ignoring test results with not executed outcome is the default now. (In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used.) (#894)

  • The configuration settings local/featureFileSource/type, local/featureFileSource/filePath and local/featureFileSource/folder are moved to the local group and renamed to local/projectType, local/projectFilePath, local/folder. The old settings work, but show a warning. (#873)

  • The .NET 5 framework is out of support and will not receive security updates in the future (see https://aka.ms/dotnet-core-support). SpecSync versions released after 31/3/2023 will not run with .NET 5. Please use SpecSync with .NET 6 or any of the other supported platforms. (#919)

  • The configuration setting synchronization/link[]/targetWorkItemType has been renamed to synchronization/link[]/targetType and SpecSync now verifies if a valid issue type has been specified when linking. The old setting works, but shows a warning. The additional verification is performed even if the old setting was used. (#822)

  • The configuration setting customizations/synchronizeLinkedArtifactTitles/linkLabelSeparator has been moved to synchronization/linkLabelSeparator. The old settings work, but show a warning. (#850)

Breaking changes

  • The test results with not executed outcome are not published by default. In order to force including them (the earlier behavior), the publishTestResults/includeNotExecutedTests setting can be used. (#894)

  • The default link-label separator has been changed from : to ;. To continue using : as link label separator, please set synchronization/linkLabelSeparator. (#861)

  • The tag prefixes specified in synchronization/links[]/tagPrefix are ensured to contain only word characters (letters, numbers, underscore, dash). (#849, #930)

  • Tag predicates in local test case conditions must start with '@'. Earlier tag predicates were also accepted, but from this version the @ has to be used, even if the local test case does not use this prefix for tags. (#877)

  • The value specified as synchronization/link[]/targetWorkItemType (now renamed to synchronization/link[]/targetType) is used now to verify if the type of the linked issue. The verification is performed when establishing new links. To disable the verification and allow linking any issue types please remove this setting. (#822)

  • The "pull" command now adds and removes local test case tags based on the links of the Test Case. (#863)

Improvements

  • Allow newline ({br}) and environment variable ({env:MY_ENV}) placeholders in run/result parameters (#957)

  • Handle TAB characters for C# test sources (#950)

  • No harmful change should be performed when the license file is not specified, but the number of scenarios exceed the free license limit (#944)

  • Use provided file name for Cucumber JSON attachments (#960)

  • Include suite name in the "unable to create suite" error message. (#933)

  • The user-specific configuration file got a separate JSON schema: https://schemas.specsolutions.eu/specsync-user-config-latest.json (#859)

  • The type of the linked issues can be restricted using the synchronization/link[]/targetType or the customizations/linkOnChange/links[]/targetType configuration settings. (#822, #867, #866)

  • Filter, scope and other conditions are satisfied if there is at least one scenario outline examples that satisfies the condition. (#717)

  • Improve performance of filtered / scoped command executions (#856)

  • Detecting non-interactive user session for operations require user input and automatically cancel to avoid blocking process (#891)

  • Improved publish-test-results reporting (#915)

  • The test results with not executed outcome (NotExecuted, NotApplicable, NotRunnable, NotImpacted) are not published by default as they might hide earlier results of the Test Case. In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used. (#894, #893)

  • The product name (SpecSync for Azure DevOps or SpecSync for Jira) is visible in the license file when opened in a text editor. (#876)

  • SpecSync NuGet packages are signed with Code Signing Certificate of Spec Solutions Kft., thumbprint: dd5e69f05ef38c016508380ca0d2294dbbe1ba69 (#882)

  • JiraTCM: Perform temporary change for the summary field to be able to track local hash when only relations changed (#676)

  • Improve error message for unknown file source (#977)

  • Detect Base64 text embeddings in CucumberJson results (#991)

  • Various stability and maintainability improvements (#691, #973, #955, #942, #911, #837, #808, #898, #880, #874, #887, #981, #982, #995, #981, #982)

Bug fixes

  • Fix: Environment variables are not allowed anymore for local/featureFileSource/filePath and local/featureFileSource/folder (#947)

  • Fix: Text file attachments ("text/plain" mime type) are included in STDOUT instead of publishing as attachment with Cucumber JSON format (Cypress) (#959)

  • Fix: Attachments from before/after hooks are not published with Cypress (#958)

  • Fix: Incorrect Gherkin might be produced when Test Case step action or expected result field contains multiple lines (#923)

  • Fix: Custom field with value null is overridden unnecessarily to '' (#934)

  • Fix: Test Case might be updated always although it is up-to-date (#935)

  • Fix: Wait on stats calls block execution (#914)

  • Fix: Plugins cannot be loaded with console app (#910)

  • Fix: File paths in tags are not normalized on Linux and macOS (#912)

  • Fix: Different hash is calculated when running on Windows or macOS/Linux causing unnecessary updates (#881)

  • Fix: Plugins might be reported twice (#864)

  • Fix: Scenario test results of pytest-playwright are not found (#917)

  • Fix: Cucumber JUnit XML report does not find additional scenario outline examples with recent Cucumber version (#796)

  • Fix: Cucumber JSON scenario outline results are incorrectly detected as re-runs (#904)

  • Fix: Cucumber test results might not be detected when scenario outline placeholder is used in the title (#472)

  • Fix: Configuration files are reported twice when configOverride is used (#890)

  • Fix: SpecSync might ask for password when version command is used (#848)

  • Fix: SpecSync Plugins: NoKeywordParser produces invalid TestSourceData (#791)

  • Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)

  • Fix: Gherkin comments are not allowed in description (#978)

  • Fix: Unhandled error when Scenario Outline examples list contains column multiple times (#996)

  • Fix: Using tags with different casing might cause unnecessary updates (#936)

  • Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)

v1.1.1 - 2022/11/14

Improvements

  • The SpecSync Docker images are now based on the ubuntu:22.04 image and the Linux binaries are compatible with Ubuntu 22.04 and do not depend on the ICU packages. (#696)

Bug Fixes

  • Fix: Cannot authenticate to Jira server with Personal Access Token (PAT) (#858)

  • Fix: SpecSync version command shows missing config error when there is no configuration file (#860)

v1.1.0 - 2022/10/31

New features

  • Support for Jira Server/Data Center - SpecSync can now also be used for Server or Data Center installation of Jira with the Jira TCM Customization solution.

  • Support for Zephyr Scale on Jira Server/Data Center - The support requires minimal configuration and supports both Test Case synchronization (push, pull) and test result publishing. (#767, #774)

  • Support updating Test Case fields that are not updated automatically by SpecSync using the fieldUpdates configuration. See feature description for details. (#737)

Breaking changes

  • In the remote configuration section the server access has to be specified with a separate serverUrl and project setting instead of a single projectUrl. The projectUrl setting is deprecated but still accepted by SpecSync.

v1.0.1 - 2022/04/05

Initial version supporting Jira Test Case management.

PreviousMigrating from SpecSync v1 to v5NextRelease Model and Roadmap

Last updated

Was this helpful?