[Docker] Fix networking and startup issues with Docker Compose scripts. Update scripts to align with best practices

[tdonohue]

References

• Fixes "Try out DSpace 9" Docker instructions no longer work. The UI never starts. #4986

Description

This PR updates the existing Dockerfiles and Docker Compose scripts in the following ways:

• Dockerfile
  • Updates to use Node v22
  • Replaces npm install with npm ci in containers. npm ci = "clean install" and is recommended for Docker images
  • Refactors to use a staged build process so that dependency installation can be cached
  • Updated to use node user instead of root (reverted this because it caused errors when testing in GitHub actions)
  • Properly use ENTRYPOINT and CMD per Docker best practices.
• Dockerfile.dist
  • Updates to use Node v22 containers
  • Replaced npm install with npm ci in containers. npm ci = "clean install" and is recommended for Docker images
  • Added max_old_space_size=4096 to increase memory to 4GB for build process. (This helped solve out of memory errors when rebuilding this container locally)
  • Properly use ENTRYPOINT and CMD per Docker best practices.
• Docker Compose scripts
  • Fixes networking issues between several scripts by updating the "network" settings
  • Removed unnecessary tty and stdin_open configurations from scripts. This fixed the bug with starting up the UI in developer mode exactly as suggested in this comment. See ffc53c3
  • Re-sync latest backend settings to these compose scripts. For example, updates Solr core settings in several scripts based on the latest code on main
  • Updated Production scripts to properly use SSR URLs. This makes it possible to run the UI in Docker using localhost URLs. See 3e11b3c
• Added a new docker-deploy step to our GitHub actions which automatically tests both the production and development images to verify they startup properly. See 06feb29

Instructions for Reviewers

Verify it's now possible to run the UI and backend via Docker in either Production mode or Developer Mode.

How to run in Developer Mode

1. Install this PR
2. Start the backend and frontend in Docker Compose via:

   docker compose -p d10 -f docker/docker-compose.yml -f docker/docker-compose-rest.yml up -d
   

3. Optionally watch the logs as everything starts up:

   docker compose -p d10 -f docker/docker-compose.yml -f docker/docker-compose-rest.yml logs -f
   

4. Be a bit patient, as the UI code is compiled at runtime...so it may take 4-5 minutes for you to see to log that says:

   Angular Live Development Server is listening on 0.0.0.0:4000, open your browser on http://localhost:4000/
   

5. Once the above log appears, the UI should be available at http:/localhost:4000/ and the backend at http://localhost:8080/server/

How to run in Production Mode

1. Install this PR
2. Start the backend and frontend in Docker Compose via: (NOTICE: the docker-compose-dist.yml script is used here, which is the production mode script)

   docker compose -p d10 -f docker/docker-compose-dist.yml -f docker/docker-compose-rest.yml up -d
   

3. Optionally watch the logs as everything starts up:

   docker compose -p d10 -f docker/docker-compose-dist.yml -f docker/docker-compose-rest.yml logs -f
   

4. The UI should start immediately. The backend will take about 30 seconds or so to boot up (check the logs). Once the backend is started, the UI will be available at http:/localhost:4000/ and the backend at http://localhost:8080/server/

[tdonohue]

This PR is now fully working for me. I've removed the work in progress label. I can now use this PR to run the UI in either production mode or development mode (see instructions in description).

On Monday, I plan to also look into whether we can add some automatic deployment / basic testing for these Angular Docker images, as that'd help ensure they don't break unexpectedly again. But, this PR could be reviewed without those changes (as those automated deployment/tests won't touch our code, they'd just touch the GitHub actions).

NOTE: While I'd like to backport some of these changes to all branches, this PR will only cleanly backport to dspace-9_x and the other backports will need to be done manually (as both 8.x and 7.x used yarn and not npm)

[tinsch]

Thanks for the fix! I tested the "Try out DSpace 9" manual on main and can confirm that the frontend did not start on my Mac with Docker-Desktop. With your fix everything started and ran smoothly, both the dev and production version.
I just have some minor questions/comments you could check. Everything else looks good 👍

[tdonohue]

This PR has been updated with a new docker-deploy step in our GitHub docker build actions which tests the dspace-angular images to ensure they startup properly.

This PR is now ready to merge. As it's already had another test (besides myself) and has new automated tests, I plan to merge it tomorrow and backport it to all active maintenance branches.

[tdonohue]

Merging with one approval. I'll work on backporting this to all active branches (most of those likely will need to be done manually, as minor changes will be needed for each branch).

[dspace-bot]

Backport failed for dspace-9_x, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin dspace-9_x
git worktree add -d .worktree/backport-4987-to-dspace-9_x origin/dspace-9_x
cd .worktree/backport-4987-to-dspace-9_x
git switch --create backport-4987-to-dspace-9_x
git cherry-pick -x 24329251da035dbac5ed1cb1d492e474894ae20d 4065f06f43c1128ac449ee5e61ba9c4188958116 6ca170f7deca013dd5dafb6a81ae1bb9fc0994e5 3e11b3c048d836a935f5f5edee7f9be157494950 ffc53c33453c25ee58ec14340fd8ae2d61c7ec83 334e5a2f102b9c742b483ec7c1da547f4c02a1a7 d6f3f8f89e56e025dadd2da1aa54f724e0e09e63 cf3f01dbb63b5d535c008ba45c5c3da47bc7464c 06feb29bd71fec38951b9c356d83ff6be9600d46

[tdonohue]

Had to manually port this to all branches:

• 9.x version: [Port dspace-9_x] [Docker] Fix networking and startup issues with Docker Compose scripts. Update scripts to align with best practices #4994
• 8.x version: [Port dspace-8_x] [Docker] Fix networking and startup issues with Docker Compose scripts. Update scripts to align with best practices #4995
• 7.x version: [Port dspace-7_x] [Docker] Fix networking and startup issues with Docker Compose scripts. Update scripts to align with best practices #4996