Add CI/CD for extensions#38
Open
denniskuijs wants to merge 35 commits into
Open
Conversation
…image with all extensions
denniskuijs
changed the title
Member
Author
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4841e684b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+58
to
+59
| if: ${{ !steps.is_main_repo.outputs.value || github.event_name == 'pull_request' }} | ||
| timeout-minutes: 20 |
There was a problem hiding this comment.
This condition skips the Gradle build for openremote/extensions push runs, but later steps still try to build Docker from context: deployment/build and include copied extension JARs. On a normal push to main, no prior step creates deployment/build or deployment/build/extensions, so the docker/build-push-action step can fail due to missing context/artifacts and block the publish pipeline.
Useful? React with 👍 / 👎.
| DOCKER_BUILD_RECORD_UPLOAD: false | ||
|
|
||
| - name: Build manager image | ||
| if: ${{ steps.is_main_repo.outputs.value && github.event_name == 'release' }} |
There was a problem hiding this comment.
This step is gated on github.event_name == 'release', but this workflow is only configured for push, pull_request, and workflow_dispatch events, so the manager image path never executes. As written, neither the tag-push run nor the gh workflow run ci_cd.yml --ref <tag> dispatch from release.yml can satisfy this condition, so versioned/latest manager images are never published.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
This PR adds CI/CD automation for building/testing/publishing the extensions repository, and introduces deployment support for packaging extension JARs into Docker images, alongside migrating in the energy and demo-setup extensions.
Changes:
- Adds a CI/CD workflow to build, run tests, publish Maven artifacts, and build/push Docker images (develop + release).
- Introduces/migrates
energyanddemo-setupextensions (models, services, demo data, and tests). - Adds Gradle tasks to copy built extension JARs into a
deploymentbuild context and enables JaCoCo reporting across subprojects.
Reviewed changes
Copilot reviewed 74 out of 76 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| gradle.properties | Disables Gradle parallelism; bumps OpenRemote version; adds Camel version property. |
| entsoe/src/test/groovy/org/openremote/extension/entsoe/agent/protocol/EntsoeProtocolTest.groovy | Adjusts Spock ignore behavior to re-enable most tests in CI. |
| entsoe/build.gradle | Adds task to copy built extension JAR into the deployment build context. |
| energy/build.gradle | New energy extension Gradle build + publishing + copy-to-deployment task. |
| energy/src/main/resources/META-INF/services/org.openremote.model.ContainerService | Registers energy container services via ServiceLoader. |
| energy/src/main/resources/META-INF/services/org.openremote.model.AssetModelProvider | Registers energy asset model provider. |
| energy/src/main/java/org/openremote/extension/energy/manager/ForecastWindService.java | Adds wind forecast service that updates attributes/predicted datapoints. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityAsset.java | Adds base electricity asset model and common attributes. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityBatteryAsset.java | Adds battery asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityChargerAsset.java | Adds charger asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityConsumerAsset.java | Adds consumer asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityProducerAsset.java | Adds producer base asset model (forecast attribute). |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityProducerSolarAsset.java | Adds solar producer asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityProducerWindAsset.java | Adds wind producer asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricityStorageAsset.java | Adds storage asset base model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricitySupplierAsset.java | Adds supplier asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricVehicleAsset.java | Adds EV asset model. |
| energy/src/main/java/org/openremote/extension/energy/model/ElectricVehicleFleetGroupAsset.java | Adds EV fleet group model. |
| energy/src/main/java/org/openremote/extension/energy/model/EnergyModelProvider.java | Enables auto-scan model provider for energy assets. |
| energy/src/main/java/org/openremote/extension/energy/model/EnergyOptimisationAsset.java | Adds energy optimisation asset model. |
| energy/src/main/java/org/openremote/extension/energy/storage/StorageSimulatorAgent.java | Adds a simulated storage agent entity. |
| energy/src/main/java/org/openremote/extension/energy/storage/StorageSimulatorAgentLink.java | Adds agent link for the storage simulator protocol. |
| energy/src/main/java/org/openremote/extension/energy/storage/StorageSimulatorProtocol.java | Adds a protocol that periodically simulates storage behavior. |
| energy/src/test/resources/META-INF/services/org.openremote.model.setup.SetupTasks | Registers energy test setup tasks. |
| energy/src/test/groovy/org/openremote/extension/energy/TestSetupTasks.groovy | Defines test setup tasks for energy integration tests. |
| energy/src/test/groovy/org/openremote/extension/energy/KeycloakTestSetup.groovy | Sets up a Keycloak realm for energy tests. |
| energy/src/test/groovy/org/openremote/extension/energy/ManagerTestSetup.groovy | Creates assets used by energy tests. |
| energy/src/test/groovy/org/openremote/extension/energy/ForecastSolarServiceTest.groovy | Adds a forecast.solar integration-style test (currently ignored). |
| ems/build.gradle | Adds task to copy built extension JAR into the deployment build context. |
| deployment/build.gradle | Adds Gradle logic to populate the Docker build context. |
| deployment/Dockerfile | Adds Dockerfile for building a manager image with extension JARs included. |
| demo-setup/build.gradle | New demo-setup extension build + publishing + copy-to-deployment task. |
| demo-setup/src/main/resources/META-INF/services/org.openremote.model.setup.SetupTasks | Registers demo setup tasks via ServiceLoader. |
| demo-setup/src/main/resources/META-INF/services/org.openremote.model.AssetModelProvider | Registers demo asset model provider. |
| demo-setup/src/main/resources/demo/rules/smartcity/TotalSolarProduction.flow | Adds demo rules flow. |
| demo-setup/src/main/resources/demo/rules/smartcity/TotalPowerConsumption.flow | Adds demo rules flow. |
| demo-setup/src/main/resources/demo/rules/smartcity/StationCrowded.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/RotterdamPowerBalance.flow | Adds demo rules flow. |
| demo-setup/src/main/resources/demo/rules/smartcity/RotterdamBatteryUse.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/ParkingFull.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/OnsParkDimLightWinds.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/OnsParkBrightStrongWinds.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/MarkthalChargersInUse.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/Markthal.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/LightGroupOnOff.flow | Adds demo rules flow. |
| demo-setup/src/main/resources/demo/rules/smartcity/Euromast.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/EnvironmentAlerts.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/smartcity/DeKuip.json | Adds demo ruleset. |
| demo-setup/src/main/resources/demo/rules/manufacturer/TotalFlow.flow | Adds manufacturer demo rules flow. |
| demo-setup/src/main/resources/demo/rules/manufacturer/SalinityLessThan3.json | Adds manufacturer demo ruleset. |
| demo-setup/src/main/resources/demo/rules/manufacturer/SalinityGreaterThan25.json | Adds manufacturer demo ruleset. |
| demo-setup/src/main/resources/demo/rules/manufacturer/SalinityBetween20And25.json | Adds manufacturer demo ruleset. |
| demo-setup/src/main/resources/demo/rules/manufacturer/IrrigationTankLow.json | Adds manufacturer demo ruleset. |
| demo-setup/src/main/resources/demo/rules/manufacturer/FlowPerMeter.flow | Adds manufacturer demo rules flow. |
| demo-setup/src/main/resources/demo/dashboards/smartcity/parking.json | Adds SmartCity demo dashboard definition. |
| demo-setup/src/main/resources/demo/dashboards/manufacturer/harvesting.json | Adds Manufacturer demo dashboard definition. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/DemoSetupTasks.java | Wires demo setup tasks selection by setup type. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/KeycloakDemoSetup.java | Adds Keycloak demo realm/user setup. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/ManagerDemoAgentSetup.java | Adds optional KNX/Velbus demo agent provisioning. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/ManagerDemoDashboardSetup.java | Imports demo dashboards at startup. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/RulesDemoSetup.java | Imports demo rulesets/flows at startup. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/model/HarvestRobotAsset.java | Adds demo domain asset model. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/model/IrrigationAsset.java | Adds demo domain asset model. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/model/ManufacturerAssetModelProvider.java | Enables auto-scan model provider for demo assets. |
| demo-setup/src/main/java/org/openremote/extension/demosetup/model/SoilSensorAsset.java | Adds demo domain asset model. |
| build.gradle | Enables JaCoCo reporting across Java subprojects. |
| .github/workflows/release.yml | Minor YAML quoting consistency. |
| .github/workflows/ci_cd.yml | Adds release trigger, Docker build/push, Maven publish, artifact archival improvements. |
Comments suppressed due to low confidence (2)
energy/build.gradle:62
- The POM
scmconnection/developerConnection/url still referenceopenremote/openremote. Since the extension was moved here, these SCM fields should point toopenremote/extensions(consistent withems/build.gradle) so Maven consumers get correct source links.
demo-setup/build.gradle:56 - The POM
scmconnection/developerConnection/url still referenceopenremote/openremote. Since the code lives inopenremote/extensions, update these SCM values so published metadata links to the correct repository (seeems/build.gradlefor the convention).
scm {
connection = 'scm:git:git://github.com/openremote/openremote.git'
developerConnection = 'scm:git:ssh://github.com:openremote/openremote.git'
url = 'https://github.com/openremote/openremote/tree/master'
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0 | ||
|
|
||
| - name: Build develop image | ||
| if: ${{ steps.is_main_repo.outputs.value && github.event_name == 'push' }} |
Comment on lines
+79
to
+83
| - name: Install QEMU | ||
| uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0 | ||
| with: | ||
| platforms: linux/amd64,linux/aarch64 | ||
|
|
| load: false | ||
| push: true | ||
| tags: | | ||
| openremote/manager:${{ inputs.tag }} |
Comment on lines
+1
to
+4
| copy { | ||
| into layout.buildDirectory | ||
| from "Dockerfile" | ||
| } No newline at end of file |
Comment on lines
+74
to
+80
| electricityWindAsset.setWindSpeedMax(18d) | ||
| electricityWindAsset.setWindSpeedMin(2d) | ||
| electricityWindAsset.setWindSpeedReference(12d) | ||
| electricityWindAsset.setPowerExportMax(9000d) | ||
| electricityWindAsset.setEfficiencyExport(100) | ||
| electricityWindAsset.setPowerExportMax(2.5) | ||
| electricityWindAsset.setLocation(new GeoJSONPoint(9.195285, 48.787418)) |
Comment on lines
+62
to
+66
| } | ||
|
|
||
| public Optional<Double> getPowerForecast() { | ||
| return getAttributes().getOrCreate(POWER_FORECAST).getValue(); | ||
| } |
Comment on lines
+71
to
+73
| @JsonProperty("temp") | ||
| protected double tempature; | ||
|
|
Comment on lines
+29
to
+30
| @Ignore | ||
| class ForecastSolarServiceTest extends Specification implements ManagerContainerTrait { |
Comment on lines
+39
to
+43
| pom { | ||
| name = 'OpenRemote energy extension' | ||
| description = 'Adds the energy domain extension' | ||
| url = 'https://github.com/openremote/openremote' | ||
| licenses { |
Comment on lines
+33
to
+37
| pom { | ||
| name = 'OpenRemote Demo setup' | ||
| description = 'Adds the OpenRemote Demo setup' | ||
| url = 'https://github.com/openremote/openremote' | ||
| licenses { |
Comment on lines
+19
to
+21
| # Release | ||
| release: | ||
|
|
Member
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3c9e7838d5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| -PsigningKey=$MAVEN_SIGNING_KEY -PsigningPassword=$MAVEN_SIGNING_PASSWORD -PpublishUsername=$MAVEN_USERNAME -PpublishPassword=$MAVEN_PASSWORD \ | ||
| -PkeystoreKeyAlias=$KEYSTORE_KEY_ALIAS -PkeystoreKeyPassword=$KEYSTORE_KEY_PASSWORD -PkeystoreFile=$PWD/keystore -PkeystorePassword=$KEYSTORE_PASSWORD | ||
| - name: Build manager image | ||
| if: ${{ steps.is_main_repo.outputs.value && github.event_name == 'release' }} |
There was a problem hiding this comment.
This workflow is only registered for push, pull_request, and workflow_dispatch, so github.event_name is never release; release tags configured at the top of this file arrive as push events. Because release.yml was deleted in this commit, the manager image release step here and the Maven Sonatype publish step below are now skipped for every tagged release.
Useful? React with 👍 / 👎.
| context: deployment/build | ||
| platforms: linux/amd64,linux/arm64 | ||
| load: false | ||
| push: false |
There was a problem hiding this comment.
The job logs in to DockerHub and applies registry tags, but the build-push-action invocation sets push: false while also using load: false; per docker/build-push-action semantics, that means the build result is not exported to the registry. On main pushes this leaves openremote/manager:develop unchanged, and the release image step has the same issue for version/latest tags.
Useful? React with 👍 / 👎.
Comment on lines
+190
to
+191
| if ((attributeName.equals(ElectricityChargerAsset.VEHICLE_CONNECTED.getName()) || attributeName.equals(ElectricVehicleAsset.CHARGER_CONNECTED.getName())) | ||
| && (Boolean)attributeEvent.getValue().orElse(false)) { |
There was a problem hiding this comment.
This branch runs only when vehicleConnected/chargerConnected is true, but storageAssetConnected() treats true as connected and the log says the flag should be cleared after disconnection. In the scenario where a force-charged EV/charger reconnects or reports connected again, this removes it from forceChargeAssetIds while FORCE_CHARGE can remain RUNNING, so the next optimisation cycle can overwrite the forced charge setpoint; the false disconnect event is the one that should clear it.
Useful? React with 👍 / 👎.
Comment on lines
+100
to
+103
| - name: Build manager image | ||
| if: ${{ steps.is_main_repo.outputs.value && github.event_name == 'release' }} | ||
| uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0 | ||
| timeout-minutes: 20 |
Comment on lines
+105
to
+109
| context: deployment/build | ||
| platforms: linux/amd64,linux/arm64 | ||
| load: false | ||
| push: false | ||
| tags: | |
Comment on lines
+118
to
+122
| - name: Publish to Maven Sonatype | ||
| if: ${{ steps.is_main_repo.outputs.value && github.event_name == 'release' }} | ||
| timeout-minutes: 20 | ||
| run: ./gradlew publishToSonatype closeAndReleaseSonatypeStagingRepository -PsigningKey=$MAVEN_SIGNING_KEY -PsigningPassword=$MAVEN_SIGNING_PASSWORD -PpublishUsername=$MAVEN_USERNAME -PpublishPassword=$MAVEN_PASSWORD | ||
| env: |
Comment on lines
+77
to
+79
| electricityWindAsset.setPowerExportMax(9000d) | ||
| electricityWindAsset.setEfficiencyExport(100) | ||
| electricityWindAsset.setPowerExportMax(2.5) |
Comment on lines
+89
to
+93
| context: deployment/build | ||
| platforms: linux/amd64,linux/arm64 | ||
| load: false | ||
| push: false | ||
| tags: openremote/manager:develop |
Comment on lines
+278
to
+282
| assetProcessingService.sendAttributeEvent(new AttributeEvent(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER_FORECAST.getName(), -currentPower), getClass().getSimpleName()); | ||
|
|
||
| if (electricityProducerWindAsset.isSetActualWindValueWithForecast().orElse(false)) { | ||
| assetProcessingService.sendAttributeEvent(new AttributeEvent(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER.getName(), -currentPower), getClass().getSimpleName()); | ||
| } |
Comment on lines
+287
to
+289
| LocalDateTime timestamp = Instant.ofEpochMilli(weatherForecastModel.getTimestamp()).atZone(ZoneId.systemDefault()).toLocalDateTime(); | ||
| assetPredictedDatapointService.updateValue(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER_FORECAST.getName(), -powerForecast, timestamp); | ||
| assetPredictedDatapointService.updateValue(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER.getName(), -powerForecast, timestamp); |
Comment on lines
+292
to
+295
| timestamp = timestamp.plusMinutes(15); | ||
| assetPredictedDatapointService.updateValue(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER_FORECAST.getName(), -powerForecast, timestamp); | ||
| assetPredictedDatapointService.updateValue(electricityProducerWindAsset.getId(), ElectricityProducerAsset.POWER.getName(), -powerForecast, timestamp); | ||
| } |
Comment on lines
+168
to
+170
| double deltaHours = seconds / 3600d; | ||
| double efficiency = power > 0 ? ((double) storageAsset.getEfficiencyImport().orElse(100)) / 100d : (1d / (((double) storageAsset.getEfficiencyExport().orElse(100)) / 100d)); // Export efficiency < 1 means more energy is consumed to produce requested power | ||
| double energyDelta = power * deltaHours * efficiency; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR includes the following changes:
demo-setupandenergyextensions from openremote/openremote to extensions.developimage on push to main and publish it to DockerHub with tagopenremote/manager:developincluding ALL extensions.managerimage when a new release is created an publish it to DockerHub with tag'openremote/manager:latestandopenremote/manager:$versionincluding ALL extensions.deploymentfolder so they can be included when building a new image.Tests to be improved: