bug: kosli attest junit reports cryptic error without filename for non-UTF-8 XML files

[dangrondahl]

Summary

kosli attest junit fails with a cryptic error when a JUnit XML file declares a non-UTF-8 encoding (e.g. ISO-8859-15):

Error: xml: encoding "ISO-8859-15" declared but Decoder.CharsetReader is nil

The error does not indicate which file caused the failure, making it difficult to debug — especially when --results-dir defaults to . and the CLI picks up all .xml files recursively.

Root Cause

Two independent issues in cmd/kosli/attestJunit.go:

1. No filename in error messages

ingestJunitDir() (line 217) delegates to junit.IngestDir() from the joshdk/go-junit library. The library's call chain (IngestDir → IngestFiles → IngestFile → IngestReader → parse) never wraps errors with the filename. Same issue in getJunitFilenames() (line 264) which calls junit.IngestFile(path) without wrapping the error.

This affects all XML parsing errors, not just encoding issues — malformed XML, truncated files, permission errors, etc.

2. No support for non-UTF-8 encodings

The go-junit library creates a bare xml.NewDecoder() without setting Decoder.CharsetReader. Go's encoding/xml only handles UTF-8 natively — any other declared encoding (ISO-8859-15, ISO-8859-1, Windows-1252, etc.) requires a custom CharsetReader.

Fix (PR #895)

• Replace junit.IngestDir() with our own directory walk that wraps errors with the filename
• Encoding errors get a user-friendly message: what's wrong, what to do
• Consolidate the two directory walks into a single pass (performance improvement)

Decision: no charset support

We considered adding golang.org/x/net/html/charset to transparently handle non-UTF-8 encodings. We decided against it because the CLI walks all .xml files in --results-dir (default .), so non-JUnit XML files (pom.xml, config files) get parsed too. We can't assume a non-UTF-8 XML file is a JUnit file with incorrect encoding — it may just be an unrelated file swept up by the walk. The right fix for users is to use --results-dir to narrow scope, and the error message now guides them to do that.

Reproduction

1. Create a JUnit XML file with a non-UTF-8 encoding declaration:

   <?xml version="1.0" encoding="ISO-8859-15"?>
   <testsuites><testsuite name="test" tests="1">
     <testcase name="pass"/>
   </testsuite></testsuites>

2. Run kosli attest junit --name junit --flow <flow> --trail <trail> --results-dir .
3. Observe: Error: xml: encoding "ISO-8859-15" declared but Decoder.CharsetReader is nil — no filename.

Workaround

• Re-save the offending XML as UTF-8, or
• Use --results-dir to point at a directory containing only JUnit XML files (avoids picking up unrelated XML with non-UTF-8 encodings)

[dangrondahl]

Reproduced locally with --dry-run:

$ ./kosli attest junit --dry-run --api-token DRY_RUN --org test-org \
    --flow test-flow --trail test-trail --name junit-test \
    --results-dir /tmp/junit-test

[warning] Repo information will not be reported as ID, Name and URL are required.
Error: xml: encoding "ISO-8859-15" declared but Decoder.CharsetReader is nil

Test fixture: a valid JUnit XML file with <?xml version="1.0" encoding="ISO-8859-15"?> in the prolog. With multiple XML files in the directory, the error gives no indication which file caused the failure.

[mbevc1]

Seems upstream is not handling non-default encodings ☹️

It's also quite unmaintained - last commit is late 2022... I've opened an issue asking if they still maintain this, but seems upstream change is a on-liner to properly handle encoding from the XML header:

func parse(reader io.Reader) ([]xmlNode, error) {
 	var (
 		dec  = xml.NewDecoder(reparentXML(reader))
 		root xmlNode
 	)
+	dec.CharsetReader = charset.NewReaderLabel

 	if err := dec.Decode(&root); err != nil {
 		return nil, err
 	}

 	return root.Nodes, nil
 }