GH-546: Migrate Sphinx documentation

[lidavidm]

Fixes #546.

[lidavidm]

This doesn't combine the Javadocs and Sphinx docs. I'd like to extract the plugin from arrow-adbc (that lets you reference Javadocs via Intersphinx) for that purpose eventually.

[kou]

Could you rename this to sphinx-docs.tar.gz or rename existing docs.tar.gz to api-docs.tar.gz?
If we use duplicated names, we can't upload both of them to GitHub Releases.

Or how about combining them something like the following?

diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
index 821ad083..46afe8f3 100644
--- a/.github/workflows/rc.yml
+++ b/.github/workflows/rc.yml
@@ -399,9 +399,8 @@ jobs:
           ci/scripts/jni_full_build.sh . jni binaries
       - name: Prepare docs
         run: |
-          mkdir -p docs
-          cp -a target/site/apidocs docs/reference
-          tar -cvzf docs.tar.gz docs
+          cp -a target/site/apidocs reference
+          tar -cvzf reference.tar.gz reference
       - name: Upload binaries
         uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
         with:
@@ -410,12 +409,12 @@ jobs:
       - name: Upload docs
         uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
         with:
-          name: release-docs
-          path: docs.tar.gz
+          name: reference
+          path: reference.tar.gz
   docs:
     name: Docs
     needs:
-      - source
+      - binaries
     runs-on: ubuntu-latest
     permissions:
       contents: read
@@ -427,7 +426,10 @@ jobs:
       - name: Download source archive
         uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
         with:
-          name: release-source
+          name: |
+            release-source
+            reference
+          merge-multiple: true
       - name: Extract source archive
         run: |
           tar -xf apache-arrow-java-*.tar.gz --strip-components=1
@@ -438,13 +440,14 @@ jobs:
           source venv/bin/activate
           pip install -r requirements.txt
           make html
+          tar -xf ../reference.tar.gz -C docs/build/html
       - name: Compress into single artifact to keep directory structure
-        run: tar -cvzf docs.tar.gz -C docs/build html
+        run: tar -cvzf html.tar.gz -C docs/build html
       - name: Upload artifacts
         uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
         with:
-          name: release-sphinx-docs
-          path: docs.tar.gz
+          name: release-html
+          path: html.tar.gz
   verify:
     name: Verify
     needs:

[lidavidm]

Ah, I'll combine them.

[kou]

+1

I confirmed that release-html asset has expected HTML.