Changelog
Last updated
Was this helpful?
Last updated
Was this helpful?
The files and folders are processed in a usual file processing order (folders first, underscore has priority) (#1275)
MarkdownToHtml value loader that can be used to from Markdown source (e.g. feature description). Usage: {feature-description:MarkdownToHtml} or {{!MarkdownToHtml}}A *Markdown* feature description: {feature-description}. The relevant can be also used. (#1277)
Support for Scenario Outline parameter values in Behave test results (#1293)
Support for processing JUnit XML TestCase Properties (#1355)
Include app version in log file (#1314)
Improve error messages for filter condition errors (E1051) when used for file path filtering (#1290)
Improve handling invalid credentials from user config (#1285)
Support for .NET 9: The now works on systems that only have .NET 9.0 SDK. (#1239)
Support for establishing non-tracked links via synchronization/links[]/disableTracking setting. With that you can disable automatic removal of links when the corresponding tag has been deleted. (#1370)
Force processing removals of local test cases on filter with toolSettings/forceProcessingRemovalsOnFilter. (#1373)
Update dependencies to address know vulnerabilities (#1375)
Support for Cucumber messages .ndjson result files. Use -f cucumberMessagesNdjson to load result from this files. (#351, #1509)
Allow publishing test results even if no matching results found. To enable this feature set publishTestResults/publishEmptyResults to true in the configuration. (#1345)
Improve logging of Jira file uploads (#1383)
Various stability and maintainability improvements (#1359, #1318, #1319, #1375, #1369)
Fix: Unhandled exception (NullReferenceException) during synchronization of scenarios (#1366)
Fix: The "Processing [file]" heading is displayed twice in log (#1361)
Fix: Scenario outline test results are not published with Reqnroll 2.1 and MsTest (#1466)
Fix: The "removed" Test Cases are not included in the sync summary line (#1330)
Fix: Plugin versions are displayed with commit info (#1333)
Fix: Source file filtering does not work on Windows when * or . is used after backslash (e.g. $sourceFile ~ Features\*.feature) (#1371)
Fix: Unhandled error when scenario outline examples does not have a table (#1483)
SpecSync.Plugin.GenericTestResultMatcher v1.2.0 (since v1.1.0):
Allow matching standard console output (StdOut) by general matcher plugin (#1286)
Match for test result properties and allow using {test-case-id} placeholder (#1356)
SpecSync.Plugin.ExcelTestSource v1.1.0:
allow defining first step in test case row, improve flexibility, allow id prefix (#1342)
make column names parametrizable (#1341)
SpecSync.Plugin.ExcelTestResults v1.3.0 (since v1.0.0):
If ID matches, should not try matching other fields (#1362)
Support for custom outcome values (#1348)
Support for non-Gherkin test source (#1347)
Treat empty outcome as "NotExecuted" (#1346)
Support ID prefix (#1338)
Fix: ID values formatted as number are not supported (#1337)
Skip rows without identifier (#1336)
Test result format error & parameter errors are reported incorrectly (#1335)
Fix: Error loading ExcelTestResults plugin (#1332)
Allow specifying authentication method explicitly in configuration (#1265)
Display a note in the verbose log if an environment variable was resolved to empty because it did not exist (#1270)
Allow using {env:ENV_VAR} placeholders in authentication related configuration settings (#1269)
Various stability and maintainability improvements (#1252, #1254, #1256, #1255)
Warning when SpecSync is used with .NET Core 3.1 (#1243)
NUnit Plugin v1.0.2: Allow loading NUnit XML (v2, v3) test result files (#1232)
Support for step-level results with Python Behave XML result files (#1226)
Various stability and maintainability improvements (#1224, #1230, #1240, #1242)
Fix: xUnit test result loading error for wider scenario outline examples tables when using xunit.runner.visualstudio v2.5 (#1236)
Fix: Configuration load error: "Unable to find a constructor to use for type SpecSync.Configuration.ConfigurationWarning" when --configOverride is used (#1238)
Fix: NUnit Plugin: NullReferenceException when loading NUnit XML v3 result files (#1231)
Fix: Test result files are processed multiple times with --configOverride, --testRunSetting or --testResultSetting command line options used (#1247)
Fix: Jira.ZephyrScale: publishing multiple test executions for the same test might fail with attachment error E0018 (#1248)
Fix: Jira.ZephyrScale: error when result attachment is published for a non-existing iteration instead of warning (#1249)
Zephyr Scale: Support for setting version & iteration field on test results (#1223)
Introducing the generic --filter command line option that can be used to filter for both tags, names and source files. E.g. --filter '@basic and $sourceFile ~ **/Addition.feature and $name = "Basic addition"'.
Errors are reported with an error ID that can be shared with support and used to search solutions in the documentation (#1082)
SpecSync Feed: SpecSync can load and display important service outages, hints and tips (#1037)
Zephyr Scale: Support for step-level attachments in test results (#1207)
Display unique ID for warnings. Now all warnings can be better identified with a code, like W5133. The next major release (v3.5) will provide further features for fine-grain handling of warnings. (#1210)
Reminder of using preview release that is released more than 90 days ago. Do not allow using preview releases released more than 180 days ago. (#1202)
Support "Ignore non-supported local tags customization" for pull. The not supported tags are not removed. (#1107)
List test case synchronization errors at the end, like warnings (#1086)
Report output warnings and errors in a way that is recognized by Azure DevOps pipeline as error/warning (#1039)
Improvements for plugin development:
Allow plugin Test Result Loaders to set DataRow on the test result (#1209)
Base classes for REST API connections (#1017, #1023)
More flexibility for JUnitXml parsers (#1018, #1031)
More flexibility for JsonFileWriter (#1019)
Allow registering value loaders (#1020)
Allow deserializing parameters (#1021)
Better error messages for errors reported as "unhandled" (#1149, #1155, #1152, #1157, #1151, #1154, #1153, #1156)
Various stability and maintainability improvements (#995, #1000, #1013, #1033, #1035, #1089, #1078, #1067, #1040, #1159, #1148, #1147, #1140, #1204, #1211, #1170, #1172, #1206, #1215, #1206, #1215)
Fix: Zephyr Scale: JsonSerializationException with Jira Zephyr Scale version 9.15.3
Fix: Jira TCM Customization: Test case links might be removed (#1218)
Fix: Jira TCM Customization: Jira Test Case might not be updated when scenario changed (#1217)
Fix: NUnit v2 XML test result "Failure" is not recognized (#1001)
Fix: Using tags with different casing might cause unnecessary updates (#936)
Fix: Config source types might be calculated incorrectly (#1015)
Fix: PyTest: Incorrect outcome published for dynamically skipped tests (#1027)
Fix: The error message does not contain the file name if test results are specified with a folder name, the format is not specified and there is a non-TRX file in the folder (#1070)
Fix: Improve error handling of various unhandled errors (#1085)
Fix: Not supported local tags are removed during pull if they are used in fieldUpdates (#1119)
Fix: Pull removes tags that have been handled by field updates (#1111)
Fix: IndexOutOfRangeException when Scenario Outline is pulled without data iterations (#1120)
Fix: JUnitXml: Test Cases (represented as <testsuite>) with failing steps (represented as <testcase>) should be treated as failed, even if the last one passed (#1083)
Fix: Unhandled error when Gherkin file contains comments in description (#1157)
Allow skip publishing test results for local test cases using the --tagFilter and --sourceFileFilter options. (#853)
Support Cucumber Java Surefire XML report (#795)
Support for SpecFlow v4.0 (#901)
The configuration setting publishTestResults/ignoreNotExecutedTests has been deprecated as ignoring test results with not executed outcome is the default now. (In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used.) (#894)
The configuration settings local/featureFileSource/type, local/featureFileSource/filePath and local/featureFileSource/folder are moved to the local group and renamed to local/projectType, local/projectFilePath, local/folder. The old settings work, but show a warning. (#873)
The .NET 5 framework is out of support and will not receive security updates in the future (see https://aka.ms/dotnet-core-support). SpecSync versions released after 31/3/2023 will not run with .NET 5. Please use SpecSync with .NET 6 or any of the other supported platforms. (#919)
The configuration setting synchronization/link[]/targetWorkItemType has been renamed to synchronization/link[]/targetType and SpecSync now verifies if a valid issue type has been specified when linking. The old setting works, but shows a warning. The additional verification is performed even if the old setting was used. (#822)
The configuration setting customizations/synchronizeLinkedArtifactTitles/linkLabelSeparator has been moved to synchronization/linkLabelSeparator. The old settings work, but show a warning. (#850)
The test results with not executed outcome are not published by default. In order to force including them (the earlier behavior), the publishTestResults/includeNotExecutedTests setting can be used. (#894)
The default link-label separator has been changed from : to ;. To continue using : as link label separator, please set synchronization/linkLabelSeparator. (#861)
The tag prefixes specified in synchronization/links[]/tagPrefix are ensured to contain only word characters (letters, numbers, underscore, dash). (#849, #930)
Tag predicates in local test case conditions must start with '@'. Earlier tag predicates were also accepted, but from this version the @ has to be used, even if the local test case does not use this prefix for tags. (#877)
The value specified as synchronization/link[]/targetWorkItemType (now renamed to synchronization/link[]/targetType) is used now to verify if the type of the linked issue. The verification is performed when establishing new links. To disable the verification and allow linking any issue types please remove this setting. (#822)
The "pull" command now adds and removes local test case tags based on the links of the Test Case. (#863)
Allow newline ({br}) and environment variable ({env:MY_ENV}) placeholders in run/result parameters (#957)
Handle TAB characters for C# test sources (#950)
No harmful change should be performed when the license file is not specified, but the number of scenarios exceed the free license limit (#944)
Use provided file name for Cucumber JSON attachments (#960)
Include suite name in the "unable to create suite" error message. (#933)
The user-specific configuration file got a separate JSON schema: https://schemas.specsolutions.eu/specsync-user-config-latest.json (#859)
The type of the linked issues can be restricted using the synchronization/link[]/targetType or the customizations/linkOnChange/links[]/targetType configuration settings. (#822, #867, #866)
Filter, scope and other conditions are satisfied if there is at least one scenario outline examples that satisfies the condition. (#717)
Improve performance of filtered / scoped command executions (#856)
Detecting non-interactive user session for operations require user input and automatically cancel to avoid blocking process (#891)
Improved publish-test-results reporting (#915)
The test results with not executed outcome (NotExecuted, NotApplicable, NotRunnable, NotImpacted) are not published by default as they might hide earlier results of the Test Case. In order to force including them, the publishTestResults/includeNotExecutedTests setting can be used. (#894, #893)
The product name (SpecSync for Azure DevOps or SpecSync for Jira) is visible in the license file when opened in a text editor. (#876)
SpecSync NuGet packages are signed with Code Signing Certificate of Spec Solutions Kft., thumbprint: dd5e69f05ef38c016508380ca0d2294dbbe1ba69 (#882)
JiraTCM: Perform temporary change for the summary field to be able to track local hash when only relations changed (#676)
Improve error message for unknown file source (#977)
Detect Base64 text embeddings in CucumberJson results (#991)
Various stability and maintainability improvements (#691, #973, #955, #942, #911, #837, #808, #898, #880, #874, #887, #981, #982, #995, #981, #982)
Fix: Environment variables are not allowed anymore for local/featureFileSource/filePath and local/featureFileSource/folder (#947)
Fix: Text file attachments ("text/plain" mime type) are included in STDOUT instead of publishing as attachment with Cucumber JSON format (Cypress) (#959)
Fix: Attachments from before/after hooks are not published with Cypress (#958)
Fix: Incorrect Gherkin might be produced when Test Case step action or expected result field contains multiple lines (#923)
Fix: Custom field with value null is overridden unnecessarily to '' (#934)
Fix: Test Case might be updated always although it is up-to-date (#935)
Fix: Wait on stats calls block execution (#914)
Fix: Plugins cannot be loaded with console app (#910)
Fix: File paths in tags are not normalized on Linux and macOS (#912)
Fix: Different hash is calculated when running on Windows or macOS/Linux causing unnecessary updates (#881)
Fix: Plugins might be reported twice (#864)
Fix: Scenario test results of pytest-playwright are not found (#917)
Fix: Cucumber JUnit XML report does not find additional scenario outline examples with recent Cucumber version (#796)
Fix: Cucumber JSON scenario outline results are incorrectly detected as re-runs (#904)
Fix: Cucumber test results might not be detected when scenario outline placeholder is used in the title (#472)
Fix: Configuration files are reported twice when configOverride is used (#890)
Fix: SpecSync might ask for password when version command is used (#848)
Fix: SpecSync Plugins: NoKeywordParser produces invalid TestSourceData (#791)
Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)
Fix: Gherkin comments are not allowed in description (#978)
Fix: Unhandled error when Scenario Outline examples list contains column multiple times (#996)
Fix: Using tags with different casing might cause unnecessary updates (#936)
Fix: TRX test attachments are not found when test execution has been performed on a different OS than the publishing (#984)
The SpecSync Docker images are now based on the ubuntu:22.04 image and the Linux binaries are compatible with Ubuntu 22.04 and do not depend on the ICU packages. (#696)
Fix: Cannot authenticate to Jira server with Personal Access Token (PAT) (#858)
Fix: SpecSync version command shows missing config error when there is no configuration file (#860)
Support for Zephyr Scale on Jira Server/Data Center - The support requires minimal configuration and supports both Test Case synchronization (push, pull) and test result publishing. (#767, #774)
Initial version supporting Jira Test Case management.
See all plugins available on nuget.org in the
SpecSync.Plugin.OnlyPublishTestResults: This plugin can be used in cases when you would like to publish test results with SpecSync for Test Cases that were not synchronized by SpecSync, but created manually or with other tools. See (#1360)
Support for , a fork & reboot of the SpecFlow project (#1272)
Support for .NET 8: The now works on systems that only have .NET 8.0 SDK. (#1239)
Interactive conflict resolution when both the local and the remote test case has changed. The change provides an improved resolution for pull and the interactive resolution can also be used (optionally) for push. You can configure the conflict resolution method separately for push and pull and you can choose from methods: forceOverride, interactive, error, skip, ignore. Choosing for the error method can be also used to get notified about conflicting changes on the server. With the --force command line option, the changes can be overridden independently of the configured method. See for details. (#613)
Additional template placeholders for accessing remote server related settings ({remote-project-url}, {remote-server-url}, {remote-project-name}). See full list of available placeholders in the . (#1024)
Local test case conditions can be filtered for test name using $name = 'My scenario' syntax. The name filter can be combined with tag or source file filter as well, e.g. $sourceFile ~ **/Addition.feature and $name = 'Basic addition' This applies to all condition configuration settings and can also be used with the -tagFilter command line option. See all possibilities for local test case conditions in the . (#1025)
Allow not synchronizing Test Case title. See for details. (#1081)
See all plugins available on nuget.org in the
SpecSync.Plugin.PostmanTestSource: This plugin can be used to synchronize a tests from a collection and publish results executed with Newman. See and the (#1032, #1079)
SpecSync.Plugin.TestNGTestSource: This plugin can be used to synchronize Java TestNG test methods and publish test result files. See and the (#1143)
In all expressions where you can specify a local test case condition (synchronization/fieldUpdates[]/condition, synchronization/fieldUpdates[]/conditionalValue/*), you can now also specify source file path conditions as well using the $sourceFile ~ path/**/myfeature.feature format. E.g. the following expression selects scenarios that are tagged with @important and are in the pricing folder or any sub-folders below: @important and $sourceFile ~ pricing/. See for details. (#878)
SpecSync plugins can now be loaded from NuGet packages that are distributed through nuget.org, custom NuGet feeds or from local folders. See for examples and further details. There is also a available on nuget.org. (#814)
Support for .NET 7: The now works on systems that only have .NET 7.0 SDK. (#868)
See all plugins available on nuget.org in the
SpecSync.Plugin.ScenarioOutlinePerExamplesTestCase: Plugin that synchronizes scenario outline examples blocks as individual test cases. See plugin details and usage (#718)
SpecSync.Plugin.ExcelTestSource: This plugin can be used to synchronize a local test cases from Excel file using the format that Azure DevOps uses when you export Test Cases to CSV. See plugin details and usage
SpecSync.Plugin.MsTestTestSource: Allows synchronizing "C# MsTest Tests" and publish results from TRX result files. See plugin details and usage
SpecSync.Plugin.NUnitTestSource: Allows synchronizing "C# NUNit Tests" and publish results from TRX result files. See plugin details and usage (#962)
SpecSync.Plugin.GenericTestResultMatcher: A SpecSync plugin that can be used to override test result matching rules. See plugin details and usage
Support for Jira Server/Data Center - SpecSync can now also be used for Server or Data Center installation of Jira with the solution.
Support updating Test Case fields that are not updated automatically by SpecSync using the . See for details. (#737)
In the the server access has to be specified with a separate serverUrl and project setting instead of a single projectUrl. The projectUrl setting is deprecated but still accepted by SpecSync.