[AURON #1952] Add sort writer test for Celeborn integration CI

[yew1eb]

Which issue does this PR close?

Closes #1952

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

How was this patch tested?

[yew1eb]

depends on #1932

[Copilot]

Pull request overview

This PR adds testing for the "sort" shuffle writer in addition to the existing "hash" writer for Celeborn integration CI. The PR expands the test matrix to cover both writer types across both Celeborn versions (0.5 and 0.6).

Changes:

• Expanded the Celeborn CI matrix from 2 jobs to 4 jobs (2 Celeborn versions × 2 writer types)
• Added overwrite: true parameter to artifact upload actions to handle artifact naming collisions when multiple matrix combinations reuse the same build artifacts
• Updated job naming to include the writer parameter for better visibility

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
.github/workflows/celeborn.yml	Expanded matrix to test both "hash" and "sort" shuffle writers for each Celeborn version, parametrized the writer configuration
.github/workflows/tpcds-reusable.yml	Added overwrite: true to artifact uploads to handle multiple workflow calls with identical artifact names

Comments suppressed due to low confidence (1)

.github/workflows/tpcds-reusable.yml:177

• The overwrite: true parameter is missing for this upload-artifact action. This is inconsistent with the other upload-artifact actions in this job (lines 171, 186, and 195) which all have overwrite: true set. Since the celeborn workflow now runs with a matrix that includes multiple writer configurations for the same celeborn version, this could lead to artifact name collisions. While the unit-tests-log upload only happens on failure, it's better to be consistent with the other uploads.

      - name: Upload unit tests log
        if: failure()
        uses: actions/upload-artifact@v6
        with:
          name: unit-tests-logs-${{ inputs.sparkver }}_${{ inputs.scalaver }}-jdk-${{ inputs.javaver }}${{ inputs.celebornver && format('-{0}', inputs.celebornver) || '' }}${{ inputs.unifflever && format('-{0}', inputs.unifflever) || '' }}
          path: "**/target/unit-tests.log"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[richox]

currently we dont depend on celeborn's shuffle writer implementation (only use its inner partition writer), so we dont have to do integration tests with different celeborn shuffle writers, what do you think?

[yew1eb]

currently we dont depend on celeborn's shuffle writer implementation (only use its inner partition writer), so we dont have to do integration tests with different celeborn shuffle writers, what do you think?

Thanks for the clarification. Closing this PR.