From 59931f59a69feb3dd5a44c1b68bcecadbb2a86ca Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Tue, 29 Apr 2025 11:01:11 +0200 Subject: [PATCH 1/3] feat: add deployment --- deployment/README.md | 161 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 deployment/README.md diff --git a/deployment/README.md b/deployment/README.md new file mode 100644 index 0000000..c691050 --- /dev/null +++ b/deployment/README.md @@ -0,0 +1,161 @@ +# Best Practices on deploying DSP-Endpoints + +DSP spec states +that [authorization](https://eclipse-dataspace-protocol-base.github.io/DataspaceProtocol/HEAD/#authorization) is +optional. However, participants usually require their endpoints to be protected and publicly reachable. Clients (usually +Consumers) have an interest to automatically discover DSP-endpoints and infer knowledge about their expected behavior. +For this, the metadata endpoint is particularly relevant because it hints to the Consumer what version of the DSP and +profiles to expect allowing the Consumer to terminate the interaction early. + +Deployment of endpoints is a detail that is at the discretion of each participant. The DSP-spec accounts for this with +the concept of the `` URL which can include subdomains and URL-segments. For example +`https://connector.mycorp.com/some/path` is a valid `` URL. However, the spec places implicit constraints on it. +If precautions are not taken under certain circumstances, there are two effects that impede interoperability. + +1. A DSP-endpoint is self-aware of its exposure location, because it needs to send protocol messages that include its + own callback endpoints. Examples are the `ContractRequestMessage`'s `callbackAddress` property and the `endpointURL` + the `Dataset`. +2. The `version` endpoint is bound to the root of the domain as per RFC8615. This is very much unlike the other DSP + endpoints that are allowed to contain arbitrary URL-segments (both for the `` and `/:callback` endpoints). + +## Scenarios + +### Scenario 1: Participant has single Agent serving single version + +In this case, the subprotocols' endpoints (Catalog Protocol, Contract Negotiation Protocol, Transfer Process Protocol) +are all served from the same ``. The component implementing the endpoints is self-contained and only needs its own +`` as input to construct the `endpointURL` and `callbackAddress`. The metadata document can be hosted if the +application exposing the DSP-endpoint is deployed with direct access to the domain. + +*`domain/.well-known/dspace-version` serving directions to the single DSP base endpoint.* + +```json +{ + "protocolVersions": [ + { + "version": "2025-1", + "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", + "binding": "HTTPS" + } + ] +} +``` + +If the `CatalogService` is registered in the did document, the `service` entry looks like + +```json +{ + "@context": [ + "https://w3id.org/dspace/2025/1/context.jsonld", + "https://www.w3.org/ns/did/v1" + ], + "id": "some:id", + "type": "CatalogService", + "serviceEndpoint": "https:///catalog" +} +``` + +### Scenario 2: Participant has single Agent serving multiple versions + +This scenario is similar to [scenario 1](#scenario-1-participant-has-single-agent-serving-single-version) with the +difference that two entries are returned to two separate `` URLs - one for each protocol version. + +- `domain/.well-known/dspace-version` serving directions to navigate to endpoints for specific versions. + +```json +{ + "protocolVersions": [ + { + "version": "2025-1", + "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", + "binding": "HTTPS" + }, + { + "version": "2024-1", + "path": "/arbitrary/difference/between/domain/and/2024-1/endpoint/", + "binding": "HTTPS" + } + ] +} +``` + +If the `CatalogService` is registered in the did document, the `service` entry still looks like in scenario 1. The +Participant indicates that this is the preferred catalog endpoint version. As the term `CatalogService` is only defined +in the context of DSP-version `2025/1`, the Consumer should assume that it serves that. Regardless, the Client still has +the option to navigate back to the well-known endpoint, filter by their preferred version and navigate back. + +```json +{ + "@context": [ + "https://w3id.org/dspace/2025/1/context.jsonld", + "https://www.w3.org/ns/did/v1" + ], + "id": "some:id", + "type": "CatalogService", + "serviceEndpoint": "https:///catalog" +} +``` + +### Scenario 3: Participant has single DSP-endpoint behind an API-Gateway + +API Gateways are infrastructure components that serve as single public entry point to an organization's internal +network which is by default not exposed to the public internet. An API Gateway thus incorporates multiple applications +and differentiates them via subpaths. Two example configuration rules of such an API gateway would be: + +1. `https://gateway.mycorp.com/first-business-application/` redirects to `https://my-business-app.mycorp.com/api/` +2. `https://gateway.mycorp.com/dsp-participant-agent/` redirects to `https://dsp-participant-agent.mycorp.com/` + +To the public internet, the well-known metadata endpoint would (if running in the Participant Agent itself) +be exposed to the public at`https://gateway.mycorp.com/dsp-participant-agent/.well-known/dspace-version` violating +RFC8615. Dataspace Participants would be required to configure an additional redirect: + +3. `https://gateway.mycorp.com/.well-known/dspace-version` redirects to + `https://dsp-participant-agent.mycorp.com/well-known/dspace-version` + +The returned version endpoint information would be: + +```json +{ + "protocolVersions": [ + { + "version": "2025-1", + "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", + "binding": "HTTPS" + }, + { + "version": "2024-1", + "path": "/arbitrary/difference/between/domain/and/2024-1/endpoint/", + "binding": "HTTPS" + } + ] +} +``` + +indicating paths that are relative to the domain of the URL this payload was served by. A client will assume that the +`2025-1` endpoint is available at `https://gateway.mycorp.com/arbitrary/difference/between/domain/and/2025-1/endpoint/` +which would be unavailable as the DSP-endpoints are registered with a first segment `dsp-participant-agent`. So +configuring an additional redirect rule (see rule 3) is not feasible. + +In that case, the did doc would be + +```json +{ + "@context": [ + "https://w3id.org/dspace/2025/1/context.jsonld", + "https://www.w3.org/ns/did/v1" + ], + "id": "some:id", + "type": "CatalogService", + "serviceEndpoint": "https://gateway.mycorp.com/dsp-participant-agent/arbitrary/difference/between/domain/and/2025-1/endpoint/catalog" +} +``` + +with the `serviceEndpoint` internally redirecting to +`https://dsp-participant-agent.mycorp.com/arbitrary/difference/between/domain/and/2025-1/endpoint/` +(see redirect rule 1) which succeeds. + +Exposing a separate microservice implementing the metadata endpoint or a static document are options too. Their API +response would have to be aware of the Participant Agent's deployment on an API Gateway. + +### Scenario 4: Participant has multiple DSP-endpoints behind an API-Gateway + From ef40f46dd898c8fb197851edb82946eabd45dbba Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Wed, 30 Apr 2025 10:53:49 +0200 Subject: [PATCH 2/3] fix: add note on connector deployment awareness --- deployment/README.md | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/deployment/README.md b/deployment/README.md index c691050..2ba12b1 100644 --- a/deployment/README.md +++ b/deployment/README.md @@ -1,8 +1,6 @@ # Best Practices on deploying DSP-Endpoints -DSP spec states -that [authorization](https://eclipse-dataspace-protocol-base.github.io/DataspaceProtocol/HEAD/#authorization) is -optional. However, participants usually require their endpoints to be protected and publicly reachable. Clients (usually +Participants usually require their endpoints to be protected and publicly reachable. Clients (usually Consumers) have an interest to automatically discover DSP-endpoints and infer knowledge about their expected behavior. For this, the metadata endpoint is particularly relevant because it hints to the Consumer what version of the DSP and profiles to expect allowing the Consumer to terminate the interaction early. @@ -107,10 +105,12 @@ and differentiates them via subpaths. Two example configuration rules of such an To the public internet, the well-known metadata endpoint would (if running in the Participant Agent itself) be exposed to the public at`https://gateway.mycorp.com/dsp-participant-agent/.well-known/dspace-version` violating -RFC8615. Dataspace Participants would be required to configure an additional redirect: +RFC8615. + +Dataspace Participants could try to heal that divergence by configuring an additional redirect: 3. `https://gateway.mycorp.com/.well-known/dspace-version` redirects to - `https://dsp-participant-agent.mycorp.com/well-known/dspace-version` + `https://dsp-participant-agent.mycorp.com/.well-known/dspace-version` The returned version endpoint information would be: @@ -121,11 +121,6 @@ The returned version endpoint information would be: "version": "2025-1", "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", "binding": "HTTPS" - }, - { - "version": "2024-1", - "path": "/arbitrary/difference/between/domain/and/2024-1/endpoint/", - "binding": "HTTPS" } ] } @@ -134,7 +129,22 @@ The returned version endpoint information would be: indicating paths that are relative to the domain of the URL this payload was served by. A client will assume that the `2025-1` endpoint is available at `https://gateway.mycorp.com/arbitrary/difference/between/domain/and/2025-1/endpoint/` which would be unavailable as the DSP-endpoints are registered with a first segment `dsp-participant-agent`. So -configuring an additional redirect rule (see rule 3) is not feasible. +configuring an additional redirect rule (see rule 3) is not enough but **has to be** accompanied by making the connector +deployment aware of its location on the proxy. It has to be aware of that anyway because of the `callbackAddress` +properties that are part of the Contract Negotiation and Transfer Process Protocols. This would change the version info +like this: + +```json +{ + "protocolVersions": [ + { + "version": "2025-1", + "path": "dsp-participant-agent/arbitrary/difference/between/domain/and/2025-1/endpoint/", + "binding": "HTTPS" + } + ] +} +``` In that case, the did doc would be @@ -156,6 +166,3 @@ with the `serviceEndpoint` internally redirecting to Exposing a separate microservice implementing the metadata endpoint or a static document are options too. Their API response would have to be aware of the Participant Agent's deployment on an API Gateway. - -### Scenario 4: Participant has multiple DSP-endpoints behind an API-Gateway - From db16beae4cb3274363df73c8e0efcdd3a5da0919 Mon Sep 17 00:00:00 2001 From: arnoweiss Date: Thu, 10 Jul 2025 16:40:13 +0200 Subject: [PATCH 3/3] chore: rename deployment to discovery, adapt to RC4 --- deployment/README.md | 168 ------------------------------------------- discovery/README.md | 26 +++++++ 2 files changed, 26 insertions(+), 168 deletions(-) delete mode 100644 deployment/README.md create mode 100644 discovery/README.md diff --git a/deployment/README.md b/deployment/README.md deleted file mode 100644 index 2ba12b1..0000000 --- a/deployment/README.md +++ /dev/null @@ -1,168 +0,0 @@ -# Best Practices on deploying DSP-Endpoints - -Participants usually require their endpoints to be protected and publicly reachable. Clients (usually -Consumers) have an interest to automatically discover DSP-endpoints and infer knowledge about their expected behavior. -For this, the metadata endpoint is particularly relevant because it hints to the Consumer what version of the DSP and -profiles to expect allowing the Consumer to terminate the interaction early. - -Deployment of endpoints is a detail that is at the discretion of each participant. The DSP-spec accounts for this with -the concept of the `` URL which can include subdomains and URL-segments. For example -`https://connector.mycorp.com/some/path` is a valid `` URL. However, the spec places implicit constraints on it. -If precautions are not taken under certain circumstances, there are two effects that impede interoperability. - -1. A DSP-endpoint is self-aware of its exposure location, because it needs to send protocol messages that include its - own callback endpoints. Examples are the `ContractRequestMessage`'s `callbackAddress` property and the `endpointURL` - the `Dataset`. -2. The `version` endpoint is bound to the root of the domain as per RFC8615. This is very much unlike the other DSP - endpoints that are allowed to contain arbitrary URL-segments (both for the `` and `/:callback` endpoints). - -## Scenarios - -### Scenario 1: Participant has single Agent serving single version - -In this case, the subprotocols' endpoints (Catalog Protocol, Contract Negotiation Protocol, Transfer Process Protocol) -are all served from the same ``. The component implementing the endpoints is self-contained and only needs its own -`` as input to construct the `endpointURL` and `callbackAddress`. The metadata document can be hosted if the -application exposing the DSP-endpoint is deployed with direct access to the domain. - -*`domain/.well-known/dspace-version` serving directions to the single DSP base endpoint.* - -```json -{ - "protocolVersions": [ - { - "version": "2025-1", - "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", - "binding": "HTTPS" - } - ] -} -``` - -If the `CatalogService` is registered in the did document, the `service` entry looks like - -```json -{ - "@context": [ - "https://w3id.org/dspace/2025/1/context.jsonld", - "https://www.w3.org/ns/did/v1" - ], - "id": "some:id", - "type": "CatalogService", - "serviceEndpoint": "https:///catalog" -} -``` - -### Scenario 2: Participant has single Agent serving multiple versions - -This scenario is similar to [scenario 1](#scenario-1-participant-has-single-agent-serving-single-version) with the -difference that two entries are returned to two separate `` URLs - one for each protocol version. - -- `domain/.well-known/dspace-version` serving directions to navigate to endpoints for specific versions. - -```json -{ - "protocolVersions": [ - { - "version": "2025-1", - "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", - "binding": "HTTPS" - }, - { - "version": "2024-1", - "path": "/arbitrary/difference/between/domain/and/2024-1/endpoint/", - "binding": "HTTPS" - } - ] -} -``` - -If the `CatalogService` is registered in the did document, the `service` entry still looks like in scenario 1. The -Participant indicates that this is the preferred catalog endpoint version. As the term `CatalogService` is only defined -in the context of DSP-version `2025/1`, the Consumer should assume that it serves that. Regardless, the Client still has -the option to navigate back to the well-known endpoint, filter by their preferred version and navigate back. - -```json -{ - "@context": [ - "https://w3id.org/dspace/2025/1/context.jsonld", - "https://www.w3.org/ns/did/v1" - ], - "id": "some:id", - "type": "CatalogService", - "serviceEndpoint": "https:///catalog" -} -``` - -### Scenario 3: Participant has single DSP-endpoint behind an API-Gateway - -API Gateways are infrastructure components that serve as single public entry point to an organization's internal -network which is by default not exposed to the public internet. An API Gateway thus incorporates multiple applications -and differentiates them via subpaths. Two example configuration rules of such an API gateway would be: - -1. `https://gateway.mycorp.com/first-business-application/` redirects to `https://my-business-app.mycorp.com/api/` -2. `https://gateway.mycorp.com/dsp-participant-agent/` redirects to `https://dsp-participant-agent.mycorp.com/` - -To the public internet, the well-known metadata endpoint would (if running in the Participant Agent itself) -be exposed to the public at`https://gateway.mycorp.com/dsp-participant-agent/.well-known/dspace-version` violating -RFC8615. - -Dataspace Participants could try to heal that divergence by configuring an additional redirect: - -3. `https://gateway.mycorp.com/.well-known/dspace-version` redirects to - `https://dsp-participant-agent.mycorp.com/.well-known/dspace-version` - -The returned version endpoint information would be: - -```json -{ - "protocolVersions": [ - { - "version": "2025-1", - "path": "/arbitrary/difference/between/domain/and/2025-1/endpoint/", - "binding": "HTTPS" - } - ] -} -``` - -indicating paths that are relative to the domain of the URL this payload was served by. A client will assume that the -`2025-1` endpoint is available at `https://gateway.mycorp.com/arbitrary/difference/between/domain/and/2025-1/endpoint/` -which would be unavailable as the DSP-endpoints are registered with a first segment `dsp-participant-agent`. So -configuring an additional redirect rule (see rule 3) is not enough but **has to be** accompanied by making the connector -deployment aware of its location on the proxy. It has to be aware of that anyway because of the `callbackAddress` -properties that are part of the Contract Negotiation and Transfer Process Protocols. This would change the version info -like this: - -```json -{ - "protocolVersions": [ - { - "version": "2025-1", - "path": "dsp-participant-agent/arbitrary/difference/between/domain/and/2025-1/endpoint/", - "binding": "HTTPS" - } - ] -} -``` - -In that case, the did doc would be - -```json -{ - "@context": [ - "https://w3id.org/dspace/2025/1/context.jsonld", - "https://www.w3.org/ns/did/v1" - ], - "id": "some:id", - "type": "CatalogService", - "serviceEndpoint": "https://gateway.mycorp.com/dsp-participant-agent/arbitrary/difference/between/domain/and/2025-1/endpoint/catalog" -} -``` - -with the `serviceEndpoint` internally redirecting to -`https://dsp-participant-agent.mycorp.com/arbitrary/difference/between/domain/and/2025-1/endpoint/` -(see redirect rule 1) which succeeds. - -Exposing a separate microservice implementing the metadata endpoint or a static document are options too. Their API -response would have to be aware of the Participant Agent's deployment on an API Gateway. diff --git a/discovery/README.md b/discovery/README.md new file mode 100644 index 0000000..c288f93 --- /dev/null +++ b/discovery/README.md @@ -0,0 +1,26 @@ +# Discovery of DSP endpoints in Practice + +Participants usually require their endpoints to be protected and publicly reachable. Clients (usually +Consumers) have an interest to automatically discover DSP-endpoints and infer knowledge about their expected behavior. +For this, the [metadata endpoint](https://eclipse-dataspace-protocol-base.github.io/DataspaceProtocol/2025-1-RC4/#exposure-of-dataspace-protocol-versions) +is particularly relevant because it hints to the Consumer what version of the DSP and profiles to expect, allowing the +Consumer to terminate the interaction early if there's no commonly understood protocol. + +Deployment of endpoints is a detail that is at the discretion of each participant. That's why it's necessary to +differentiate URLs and their relationship with each other. + +- `` is the naked domain without URL segments or fragments. +- `` is the URL on which the `/.well-known/dspace-version` endpoint is hosted. It should be equivalent to +`` to satisfy RFC8615. If it does, clients can always find the metadata endpoint. Else, a Dataspace should +have conventions to enable discovery of that endpoint in a different manner - for instance by [registering a +`DataService`](https://eclipse-dataspace-protocol-base.github.io/DataspaceProtocol/2025-1-RC4/#example-data-service-did-service-example) +in their did document. +- `` is the URL on which the protocol endpoints are hosted. These endpoints host specific profiles and versions +of the DSP. They are constructable from the metadata endpoint by concatenating `` with the `path` property fetched +from the document. + +A DSP-endpoint must be self-aware of its exposure location, because it needs to send protocol messages that include its +own callback endpoints. Examples are the `ContractRequestMessage`'s `callbackAddress` property and the `endpointURL` +the `DataService` exposes. Both serve the `` URL of the relevant services. In deployment scenarios where the +DSP-endpoints are proxied through the network, the application logic must be made aware of this as it impacts the +payloads it will emit.