Add geoserver_data directory to this repository

[ridoo]

Currently, the geoserver data directory is downloaded when the geoserver_data container is being created:

geonode-docker/docker/geoserver_data/Dockerfile

Lines 14 to 16 in cf60723

	ADD download.sh ${TEMP_DOWNLOADED}
	RUN chmod +x ${TEMP_DOWNLOADED}/download.sh
	RUN ${TEMP_DOWNLOADED}/download.sh $GEOSERVER_VERSION $TEMP_DOWNLOADED

The data dir is a zip hosted at https://artifacts.geonode.org/geoserver/:

geonode-docker/docker/geoserver_data/download.sh

Lines 14 to 16 in cf60723

	echo "GeoServer Data Directory is going to be downloaded"
	artifact_url="https://artifacts.geonode.org/geoserver/$GEOSERVER_VERSION/geonode-geoserver-ext-web-app-data.zip"
	echo "Downloading: $artifact_url"

Furthermore, for what I can see, the geoserver_data service's only job is to download that zip to provide it as shared volume to be available for other services, like django and geoserver.

geonode-docker/docker/geoserver_data/Dockerfile

Line 25 in cf60723

VOLUME ${BASE_GEOSERVER_DATA_DIR}/data

This however, forces the container to be up and running. Otherwise, the volume is not available anymore.

I have several questions:

• Is there any reason (I cannot see), why we do not host the data dir in this repository?
• Do you have particular requirements which argue for a dedicated geoserver_data container?
• What do you think about
  • adding the data dir to the geoserver container
  • removing the geoserver_data image completely
  • move shared volume provisioning to geoserver container

Anything else, I miss?

/cc @mwallschlaeger

[ridoo]

I found this location where the geoserver-geonode-ext-data.zip is created. The zip is that one which is downloaded on the geoserver_data image build:

geonode-docker/docker/geoserver_data/download.sh

Lines 14 to 18 in 56a1734

	echo "GeoServer Data Directory is going to be downloaded"
	artifact_url="https://artifacts.geonode.org/geoserver/$GEOSERVER_VERSION/geonode-geoserver-ext-web-app-data.zip"
	echo "Downloading: $artifact_url"
	curl -k -L "$artifact_url" --output data.zip && unzip -x -d ${TEMP_DOWNLOADED} data.zip
	echo "GeoServer Data Directory download has been completed"

However, this seems to be a test for pushing things to S3 in the future. I think, the workflow is a similar one on geosolutions Jenkins.

[giohappy]

@ridoo I agree with you. The geoserver-data service is there since a long time and we also considered several times the option to remove it.

Nothing against it, but we want to maintain the data directory as a dedicated volume, that lasts beyond the upgrades of the geoserver container.

We could make the geoserver container populate the volume the very first time, and we have to options here:

• it uncompresses geoserver.war and copies the data folder to the volume (NOTICE: the war contains the data directory. It's there in case someone wants to run Geoserver outside the standard deployment, where the data directory is normally mounted in an external folder).
• it downloads the data directory from the AWS artifact and copies it to the volume, which is the operation currently performed by the geoserver-data container

[ridoo]

@giohappy do you see a chance to let the GeoNode/geoserver image provide the geoserver_data dir?

As far as I can see, both options you provided come with some kind of searching, where is the actual source of the data dir. In deployments, I assume, most people do a volume overlay on this directory anyway.

What do you think?

[giohappy]

@ridoo as I said in my previous comment, the geoserver image already ships with a copy of the GeoServer data dir, so we could let it "mount" it inside the volume. See the first bullet point.
At the moment it's inside the WAR file but we could also ship it outside the WAR, ready to be mounted with having to uncompress the WAR first.

BTW do you mean keeping the data-dir inside the container without using external volumes? That could work for "read-only" scenarios, where you have immutable data and configurations.

[ridoo]

@giohappy well, what I have in mind is a deployment which configures an empty mountpoint where GeoServer can unpack its data dir into. Always thought, GeoNode needs some specific data dir, so I never checked, if the data dir shipped with geonode/geoserver is the same as the data dir downloaded.

Regarding separate GeoServer deployment, we could do the following (not tested):

docker compose -f /srv/geonode/docker-compose.yml up -d  ## without geoserver
docker run -p8080:8080 -v /srv/geoserver_data:/geoserver_data geonode/geoserver:latest

The /srv/geoserver_data would be a local directory which can be mounted to both GeoNode and GeoServer.

Hope, I did not miss the actual point your are trying to make.