Skip to content

feat: Add --scan-dryrun-results flag for Maven test runner#1278

Open
psakthivel04 wants to merge 1 commit intov1from
feat/scan-dryrun-results
Open

feat: Add --scan-dryrun-results flag for Maven test runner#1278
psakthivel04 wants to merge 1 commit intov1from
feat/scan-dryrun-results

Conversation

@psakthivel04
Copy link
Copy Markdown
Contributor

@psakthivel04 psakthivel04 commented Apr 14, 2026

Why:

Add support for scanning Maven Surefire reports generated by mvn test -DdryRun=true. This allows users to leverage Maven's own test discovery and filtering logic for 100% accuracy.

Changes:

  • Add parse_surefire_reports() function to parse XML reports from target/surefire-reports/
  • Add --scan-dryrun-results CLI option to maven subset command
  • Add integration with Launchable subset API
  • Add comprehensive test coverage (unit and integration tests)
  • Add test data files for testing

Fixes issue where CLI would include filtered tests that Maven excludes, leading to 'missing test report' warnings in the UI.

local testing:

Uploading image.png…

Comment thread launchable/test_runners/maven.py Outdated

if classname:
test_paths.append({"type": "class", "name": classname})
click.echo(f" Discovered: {classname}", err=True)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to remove this, as having many classes could result in an excessive amount of logs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes removed it

@@ -0,0 +1,4 @@
-------------------------------------------------------------------------------
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you create these test files??

Since others may need to make changes as well, could you please document how to create these files in the tests/data/mave/README.md or a similar place?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes create new README file

Comment thread launchable/test_runners/maven.py Outdated
):

# Handle --scan-dryrun-results: parse Maven surefire reports
if is_scan_dryrun_results:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the --scan-dryrun-results path is an early return block that bypasses the rest of the subset() function. This makes it a separate code path with its own client.run() call, while the other modes (--test-compile-created-file, --scan-test-compile-lst, source_roots) all converge into a single client.run() at the end.

This could be unified into the existing if / elif / else structure:

-
-    # Handle --scan-dryrun-results: parse Maven surefire reports
-    if is_scan_dryrun_results:
-        click.echo("Scanning Maven dry-run results...", err=True)
-        tests = parse_surefire_reports()
-
-        if not tests:
-            click.secho(
-                "Warning: No surefire reports found. Did you run 'mvn test -DdryRun=true'?",
-                fg='yellow',
-                err=True
-            )
-            click.secho(
-                "Please run: mvn test -DdryRun=true",
-                fg='cyan',
-                err=True
-            )
-            return
-
-        # Add tests to client (each test path must be a list)
-        for test in tests:
-            client.test_paths.append([test])
-
-        client.same_bin_formatter = lambda s: [{"type": "class", "name": s}]
-        client.run()
-        return
-
     # Compile exclude rules
     compiled_exclude_rules = []
     ...

-    files_to_read = list(test_compile_created_file)
-    if is_scan_test_compile_lst:
+    if is_scan_dryrun_results:
+        click.echo("Scanning Maven dry-run results...", err=True)
+        tests = parse_surefire_reports()
+
+        if not tests:
+            click.secho(
+                "Warning: No surefire reports found. Did you run 'mvn test -DdryRun=true'?",
+                fg='yellow',
+                err=True
+            )
+            click.secho(
+                "Please run: mvn test -DdryRun=true",
+                fg='cyan',
+                err=True
+            )
+            return
+
+        for test in tests:
+            client.test_paths.append([test])
+
+    elif is_scan_test_compile_lst:
+        files_to_read = list(test_compile_created_file)
         if len(test_compile_created_file) > 0:
             ...
         ...
-
-    if files_to_read:
+    elif files_to_read:
         for file in files_to_read:
             ...
     else:
         for root in source_roots:
             client.scan(root, '**/*', file2test)

     client.run()

WDYT??

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I have refactored it

@launchable-app

This comment has been minimized.

@psakthivel04 psakthivel04 force-pushed the feat/scan-dryrun-results branch from eae7c2a to 8f78d8f Compare April 16, 2026 06:35
Comment thread launchable/test_runners/maven.py Outdated
@click.option(
'--scan-dryrun-results',
'is_scan_dryrun_results',
required=False,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nits: required=False is a default. So you don't need to set it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes updated it

Comment thread launchable/test_runners/maven.py Outdated
"Warning: No .lst files. Please run after executing `mvn test-compile`", fg="yellow"),
err=True)
return
client.same_bin_formatter = lambda s: [{"type": "class", "name": s}]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to set this property here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes updated it

cd tests/data/maven/dryrun-test

# Run Maven dry-run to generate Surefire reports
mvn test -DdryRun=true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it correct? pom.xml file and other files don't exist. And I've faced this error

$ mvn test -DdryRun=true
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.023 s
[INFO] Finished at: 2026-04-16T17:30:14+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/Users/ryabuki/src/github.com/cloudbees-oss/smart-tests-cli-wt/review-1278/tests/data/maven/dryrun-test). Please verify you invoked Maven from the correct directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes without pom.xml . will get this error. This is fine right.

@psakthivel04 psakthivel04 force-pushed the feat/scan-dryrun-results branch from 8f78d8f to 2c3cf81 Compare April 16, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants