CCM-13295 add basic status reporter component

[Ian-Hodges]

Description

This PR adds the Digital Letters status reporter component. This component subscribes to all Digital Letters events and uses a Firehose stream (with associated transformer lambda) to record them to S3. There is then an event_record Glue table which is backed by these files in S3 and a corresponding Athena workspace which can be used to query the table.

Note: While the Glue table is configured to point to the S3 bucket it does not automatically load the data from S3. A follow-up PR will be produced to implement the step function shown in the above diagram to do this. In the meantime, we can manually use the MSCK REPAIR TABLE event_record command in Athena to prompt data to be reloaded from S3.

Context

This component allows us to record details of all events sent as part of the Digital Letters application, so that other components can use Athena to query the Glue table in order to build the reports that will be sent back to the customer.

Validation

Ran the following query in Athena:

SELECT * FROM "AwsDataCatalog"."nhs-pr190-dl-reporting"."event_record"
WHERE senderId = 'pr-validation-1'
limit 10;

Sent a uk.nhs.notify.digital.letters.mesh.inbox.message.downloaded.v1 event to my DE's event bus with a sender ID of pr-validation-1:

Verified that the event had been recorded to S3:

Ran the MSCK REPAIR TABLE event_record command in Athena:

Re-ran the original Athena query:

A CloudWatch query of the event bus logs (filtered for pr-validation-1), for comparison:

Type of changes

• Refactoring (non-breaking change)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• I am familiar with the contributing guidelines
• I have followed the code style of the project
• I have added tests to cover my changes
• I have updated the documentation accordingly
• This PR is a result of pair or mob programming
• If I have used the 'skip-trivy-package' label I have done so responsibly and in the knowledge that this is being fixed as part of a separate ticket/PR. - CCM-13475: Resolve fast-xml-parser security vulnerability #196 has been created to resolve the current Trivy failure

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.