feat(containertest): add `standardRun` YAML check for unmodified container startup by Copilot · Pull Request #147 · trueforge-org/forgetool

Copilot · 2026-02-19T10:58:53Z

This adds a standardRun option to container-test YAML so configs can validate a plain container startup path (default entrypoint/args, no overrides). The runner now treats standardRun as a first-class check type alongside http, tcp, filePaths, and commands.

YAML runner support
- Added standardRun to ContainerTestYAML.
- Updated “no checks configured” validation to accept configs that only specify standardRun: true.
- Wired RunChecksFromYAML to execute a standard-run check when enabled.
Standard run check implementation
- Added CheckStandardRun(ctx, image, config) in pkg/containertest/testhelpers.go.
- Behavior: starts container with the image defaults (no entrypoint/command mutation), applies optional env config, then tears down container.
Schema and config examples
- Updated pkg/containertest/container-test.schema.json:
  - Added standardRun boolean property.
  - Included standardRun in accepted check groups (anyOf).
- Updated sample config in testdata/container_test_configs/schema.yaml to demonstrate usage.
Unit coverage updates
- Extended YAML runner tests to cover:
  - parsing standardRun,
  - error propagation from standard-run execution,
  - execution ordering/coverage when combined with other check types.

Example YAML:

timeoutSeconds: 180
standardRun: true
http:
  - port: "8080"
    path: /health
    statusCode: 200

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/repos/trueforge-org/cluster-template/releases/latest
- Triggering command: /tmp/go-build2354131493/b001/forgetool.test /tmp/go-build2354131493/b001/forgetool.test -test.run=TestMainHelperProcess -stringintconv l/linux_amd64/vet /tmp/go-build2354131493/b2654/vet.cfg -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags (http block)
invalid.domain.that.does.not.exist.example
- Triggering command: /tmp/go-build2354131493/b2799/helper.test /tmp/go-build2354131493/b2799/helper.test -test.testlogfile=/tmp/go-build2354131493/b2799/testlog.txt -test.paniconexit0 -test.timeout=10m0s --64�� -o t 0.1-go1.25.5.lin-nilfunc elf_x86_64 --hash-style=gnu-test.run=TestHelperProcessMain t 0.1-go1.25.5.lin-tests -ato�� -bool t.cfg ux-amd64/pkg/tool/linux_amd64/compile -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags (dns block)
invalid.invalid.invalid
- Triggering command: /tmp/go-build2354131493/b2799/helper.test /tmp/go-build2354131493/b2799/helper.test -test.testlogfile=/tmp/go-build2354131493/b2799/testlog.txt -test.paniconexit0 -test.timeout=10m0s --64�� -o t 0.1-go1.25.5.lin-nilfunc elf_x86_64 --hash-style=gnu-test.run=TestHelperProcessMain t 0.1-go1.25.5.lin-tests -ato�� -bool t.cfg ux-amd64/pkg/tool/linux_amd64/compile -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags (dns block)
- Triggering command: /tmp/go-build2354131493/b2799/helper.test /tmp/go-build2354131493/b2799/helper.test -test.run=TestHelperProcessExitPaths --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-Wl,-z,now (dns block)
oci.trueforge.org
- Triggering command: /tmp/go-build2354131493/b001/forgetool.test /tmp/go-build2354131493/b001/forgetool.test -test.run=TestMainHelperProcess -stringintconv l/linux_amd64/vet /tmp/go-build2354131493/b2654/vet.cfg -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags (dns block)
- Triggering command: /tmp/go-build2354131493/b2799/helper.test /tmp/go-build2354131493/b2799/helper.test -test.run=TestHelperProcessCheckReqDomains pviM5nxT1ez3f21FXFdb/pviM5nxT1ez3f21FXFdb -goversion go1.25.5 -c=4 -nolocalimports -importcfg /tmp/go-build2354131493/b2795/importcfg -pack /home/REDACTED/work/forgetool/forgetool/pkg/gencmd/apply.go /home/REDACTED/work/forgetool/forgetool/pkg/gencmd/bootstrap.go -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-test.timeout=10m0s (dns block)
pool.ntp.org
- Triggering command: /tmp/go-build2354131493/b001/forgetool.test /tmp/go-build2354131493/b001/forgetool.test -test.run=TestMainHelperProcess -stringintconv l/linux_amd64/vet /tmp/go-build2354131493/b2654/vet.cfg -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert t ux-amd64/pkg/too-buildtags (dns block)
- Triggering command: /tmp/go-build2354131493/b2799/helper.test /tmp/go-build2354131493/b2799/helper.test -test.run=TestHelperProcessExitPaths /tmp/go-link-1533295041/go.o /tmp/go-link-1533295041/000000.o /tmp/go-link-1533295041/000001.o /tmp/go-link-1533295041/000002.o /tmp/go-link-1533295041/000003.o /tmp/go-link-1533295041/000004.o /tmp/go-link-1533295041/000005.o /tmp/go-link-1533295041/000006.o /tmp/go-link-1533295041/000007.o /tmp/go-link-1533295041/000008.o -ato�� -bool t ux-amd64/pkg/tool/linux_amd64/compile -errorsas -ifaceassert t ux-amd64/pkg/too-plugin-opt=/usr/libexec/gcc/x86_64-linux-gnu/13/lto-wrapper (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: Crow-Control <7613738+Crow-Control@users.noreply.github.com>

Crow-Control · 2026-02-19T11:44:51Z

@copilot fix issues

Copilot

Pull request overview

Adds a new standardRun YAML check to the pkg/containertest runner so configs can validate that an image starts successfully using its default entrypoint/args (no command overrides).

Changes:

Extended ContainerTestYAML + runner validation/execution to support standardRun.
Implemented CheckStandardRun container lifecycle check in testhelpers.go.
Updated JSON schema and sample YAML config to document/validate the new option, plus expanded unit tests around the YAML runner.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
testdata/container_test_configs/schema.yaml	Adds an example `standardRun: true` usage to the sample config.
pkg/containertest/yaml_runner_test.go	Extends YAML runner tests to parse/call/propagate errors for `standardRun`.
pkg/containertest/yaml_runner.go	Adds `standardRun` field + runner logic to treat it as a valid check type.
pkg/containertest/testhelpers.go	Introduces `CheckStandardRun` implementation for starting/tearing down a default container run.
pkg/containertest/container-test.schema.json	Adds `standardRun` property and includes it in accepted check groups.

Copilot · 2026-02-20T14:04:40Z

pkg/containertest/container-test.schema.json

  "$id": "https://trueforge-org.github.io/containerforge/container-test.schema.json",
  "title": "Container test configuration",
-  "description": "Schema for apps/*/container-test.yaml consumed by testhelpers.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths) for a container image.",
+  "description": "Schema for apps/*/container-test.yaml consumed by testhelpers.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths, standardRun) for a container image.",


The schema description says configs are consumed by testhelpers.RunChecksFromYAML, but the runner entrypoint in this repo is containertest.RunChecksFromYAML (pkg/containertest/yaml_runner.go). Updating this string will prevent readers from searching for a non-existent API.

Suggested change

"description": "Schema for apps/*/container-test.yaml consumed by testhelpers.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths, standardRun) for a container image.",

"description": "Schema for apps/*/container-test.yaml consumed by containertest.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths, standardRun) for a container image.",

Copilot · 2026-02-20T14:04:40Z

pkg/containertest/container-test.schema.json

@@ -140,6 +147,11 @@
      "required": [
        "filePaths"
      ]
+    },
+    {
+      "required": [
+        "standardRun"
+      ]
    }


anyOf only requires the presence of standardRun, so a config like {standardRun: false} will pass schema validation but still fail at runtime with "no checks configured" (because the Go runner treats standardRun as enabled only when true). Consider making standardRun a const: true in the schema (or otherwise constraining it) so schema validation matches runtime behavior.

Copilot · 2026-02-20T14:04:41Z

pkg/containertest/testhelpers.go

+			err = fmt.Errorf("failed to terminate container: %w", termErr)
+		}
+	}()
+


CheckStandardRun currently treats a successful testcontainers.Run as a pass without verifying the container is still running (or exited cleanly). This can produce false positives for images that start and immediately crash/exit. Consider checking container.State(ctx) after startup and failing if it’s not running (or if it exited with a non-zero code) before logging success.

Suggested change

state, stateErr := container.State(ctx)

if stateErr != nil {

return fmt.Errorf("failed to inspect container state for standard run check: %w", stateErr)

}

if !state.Running {

if state.ExitCode != 0 {

return fmt.Errorf("container exited with non-zero exit code %d during standard run check", state.ExitCode)

}

return fmt.Errorf("container is not running during standard run check (exited with code 0)")

}

Copilot · 2026-02-20T14:04:41Z

pkg/containertest/testhelpers.go

+// CheckStandardRun verifies the container can be started without altering entrypoint or args.
+func CheckStandardRun(ctx context.Context, image string, config *ContainerConfig) (err error) {
+	logInfo("🧪 Standard run check: image=%s", image)
+	opts := applyContainerConfig(config)
+
+	container, err := runContainer(ctx, image, opts...)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if shouldDumpContainerLogs(err != nil) {
+			dumpContainerLogs(ctx, container, "standard run check")
+		} else {
+			logDebug("Skipping container logs for standard run check (mode=%q, failed=%t)", strings.TrimSpace(strings.ToLower(os.Getenv("TESTHELPERS_CONTAINER_LOGS"))), err != nil)
+		}
+		termErr := terminateContainer(ctx, container, "standard run check")
+		if err == nil && termErr != nil {
+			err = fmt.Errorf("failed to terminate container: %w", termErr)
+		}
+	}()
+
+	logInfo("Standard run check completed successfully for image=%s", image)
+	return nil
+}


CheckStandardRun is new behavior but doesn’t have direct unit coverage in testhelpers_unit_test.go (unlike the other high-level checks). Adding tests using the existing runContainerBackend seam would help lock in expected behavior (e.g., starts container with only env customizer, propagates run errors, and wraps terminate failures).

Initial plan

8daf510

Copilot AI assigned Copilot and Crow-Control Feb 19, 2026

Copilot started work on behalf of Crow-Control February 19, 2026 10:59 View session

Copilot AI and others added 3 commits February 19, 2026 11:07

feat(containertest): add standardRun yaml option

1514161

Co-authored-by: Crow-Control <7613738+Crow-Control@users.noreply.github.com>

chore(containertest): update progress after validation

f7a21c3

Co-authored-by: Crow-Control <7613738+Crow-Control@users.noreply.github.com>

fix(containertest): restore unintended module file edits

db560b9

Co-authored-by: Crow-Control <7613738+Crow-Control@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add container test option in standardRun YAML~~ feat(containertest): add standardRun YAML check for unmodified container startup Feb 19, 2026

Copilot AI requested a review from Crow-Control February 19, 2026 11:14

Copilot finished work on behalf of Crow-Control February 19, 2026 11:14

Copilot started work on behalf of Crow-Control February 19, 2026 11:45 View session

Copilot stopped work on behalf of Crow-Control due to an error February 19, 2026 11:45
The session was cancelled by the user.

Crow-Control marked this pull request as ready for review February 20, 2026 14:00

Crow-Control requested a review from alfi0812 as a code owner February 20, 2026 14:00

Copilot AI review requested due to automatic review settings February 20, 2026 14:00

Crow-Control merged commit 29ad9d3 into main Feb 20, 2026
10 of 13 checks passed

Copilot started reviewing on behalf of Crow-Control February 20, 2026 14:00 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comments

feat(containertest): add `standardRun` YAML check for unmodified container startup#147

feat(containertest): add `standardRun` YAML check for unmodified container startup#147
Crow-Control merged 4 commits intomainfrom
copilot/add-container-test-option

Copilot AI commented Feb 19, 2026 •

edited

Loading

Uh oh!

Crow-Control commented Feb 19, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Copilot AI Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	"description": "Schema for apps/*/container-test.yaml consumed by testhelpers.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths, standardRun) for a container image.",
	"description": "Schema for apps/*/container-test.yaml consumed by containertest.RunChecksFromYAML. Define one or more check groups (http, tcp, commands, filePaths, standardRun) for a container image.",

+	state, stateErr := container.State(ctx)
+	if stateErr != nil {
+		return fmt.Errorf("failed to inspect container state for standard run check: %w", stateErr)
+	}
+	if !state.Running {
+		if state.ExitCode != 0 {
+			return fmt.Errorf("container exited with non-zero exit code %d during standard run check", state.ExitCode)
+		}
+		return fmt.Errorf("container is not running during standard run check (exited with code 0)")
+	}

Uh oh!

Comments

Conversation

Copilot AI commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Crow-Control commented Feb 19, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Feb 19, 2026 •

edited

Loading