feat: tnc connectors implementation [CM-1010]

[mbani01]

This pull request adds support for the TNC platform in the Snowflake connectors integration. It introduces new activity types, data sources, SQL queries, and transformers to ingest and process TNC certification, enrollment, and course action data. The changes include both backend database updates and significant additions to the data ingestion pipeline.

Key changes:

1. TNC Platform Integration:

• Added TNC as a supported platform in the Snowflake connectors, registering three new data sources: enrollments, certificates, and course actions, each with their own query builders and transformers. [1] [2] [3]

2. Data Source Query Builders:

• Implemented buildSourceQuery functions for TNC certificates, enrollments, and course actions, providing SQL logic to extract and join relevant data from Snowflake, including handling environment-specific filters and incremental syncs. [1] [2] [3]

3. Data Transformation Logic:

• Added transformer classes for TNC certificates, enrollments, and course actions, mapping raw query results into the unified activity schema, resolving member identities, organizations, and attributes, and handling special cases or missing data. [1] [2] [3]

4. Database Schema Update:

• Inserted new TNC-related activity types into the activityTypes table to support tracking of certification enrollments, training enrollments, issued certifications, and course/exam attempts.

These changes lay the groundwork for ingesting and processing TNC data, enabling richer analytics and reporting for certification and training activities.

---

Note

Medium Risk
Adds a new Snowflake ingestion path (queries + transformers) and new persisted activity types, so incorrect mapping or SQL could lead to missing/duplicated activities or mis-attributed org/member data.

Overview
Adds TNC support to the Snowflake connectors pipeline. Registers PlatformType.TNC with three new data sources (enrollments, certificates, courses) and implements their Snowflake buildSourceQuery SQL plus transformers to emit unified activities (including member identities, org enrichment, segment routing, and incremental export logic).

Introduces new TNC activity taxonomy and org sourcing. Adds a DB migration inserting five tnc activityTypes, exports TncActivityType/TNC_GRID from @crowd/integrations, and wires TNC into organization source/attribute-source enums and org attribute source priority. Also improves TransformerBase.safeTransformRow logging to include error message/stack and row key metadata for debugging skipped rows.

^{Written by Cursor Bugbot for commit 00c3f1a. This will update automatically on new commits. Configure here.}

[github-actions]

Conventional Commits FTW!

[joanagmaia]

Nitpick: Can we add a comment here just to explain that this is being added for staging/testing purposes?

[joanagmaia]

If we were to fallback to the email, could we remove the domain? And simply add the initial part of the email?

[joanagmaia]

Same thing here

[joanagmaia]

Same here

[cursor]

Missing sourceId validation in certificates and enrollments transformers

Medium Severity

The certificates transformer extracts certificateId via (row.CERTIFICATE_ID as string)?.trim() and the enrollments transformer extracts enrollmentId via (row.ENROLLMENT_ID as string)?.trim() — neither validates the value before assigning it to activity.sourceId. If the row value is null/undefined, the result is undefined, violating the IActivityData interface where sourceId: string is required (not optional). The courses transformer correctly validates courseActionId and returns null if missing, but the other two transformers skip this check, risking activities with undefined sourceId being passed downstream.

Additional Locations (1)

• services/apps/snowflake_connectors/src/integrations/tnc/enrollments/transformer.ts#L26-L27

 

[cursor]

SQL CTE constants duplicated across three files

Low Severity

The CDP_MATCHED_SEGMENTS and ORG_ACCOUNTS SQL CTE constants are identically copy-pasted across all three TNC buildSourceQuery files (certificates, courses, enrollments). LFID_COALESCE is also duplicated between certificates and enrollments. These could live in a shared TNC SQL fragments module to avoid divergence during future updates.

Additional Locations (2)

• services/apps/snowflake_connectors/src/integrations/tnc/certificates/buildSourceQuery.ts#L9-L30
• services/apps/snowflake_connectors/src/integrations/tnc/enrollments/buildSourceQuery.ts#L10-L31

 

[Copilot]

Pull request overview

Adds first-class TNC support to the Snowflake connectors pipeline, enabling exports/transforms for TNC enrollments, certificates, and course actions into the unified activity model, plus the corresponding activity type registration and org-source wiring.

Changes:

• Registers TNC as a Snowflake connector platform with three new data sources (enrollments, certificates, course actions) including Snowflake query builders and transformers.
• Introduces TNC activity typing/scoring (enrolled-*, issued-certification, attempted-*) and a DB migration to register these activity types.
• Extends organization source/attribute source enums and priority ordering to include TNC, and improves transformer error logging.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
services/libs/types/src/enums/organizations.ts	Adds TNC to organization source enums.
services/libs/integrations/src/integrations/tnc/types.ts	Defines TNC activity types + scoring grid.
services/libs/integrations/src/integrations/index.ts	Exports TNC integration types.
services/libs/data-access-layer/src/organizations/attributesConfig.ts	Adds TNC to org attribute source priority list.
services/apps/snowflake_connectors/src/integrations/types.ts	Adds TNC data source names.
services/apps/snowflake_connectors/src/integrations/tnc/tncTransformerBase.ts	Adds shared org-building logic for TNC transformers.
services/apps/snowflake_connectors/src/integrations/tnc/enrollments/transformer.ts	Transforms enrollment rows into unified activities.
services/apps/snowflake_connectors/src/integrations/tnc/enrollments/buildSourceQuery.ts	Snowflake SQL for TNC enrollments export + incremental logic.
services/apps/snowflake_connectors/src/integrations/tnc/courses/transformer.ts	Transforms course-action rows into attempted-course/exam activities.
services/apps/snowflake_connectors/src/integrations/tnc/courses/buildSourceQuery.ts	Snowflake SQL for TNC course actions export + incremental logic.
services/apps/snowflake_connectors/src/integrations/tnc/certificates/transformer.ts	Transforms certificate rows into issued-certification activities.
services/apps/snowflake_connectors/src/integrations/tnc/certificates/buildSourceQuery.ts	Snowflake SQL for TNC certificates export + incremental logic.
services/apps/snowflake_connectors/src/integrations/index.ts	Registers TNC platform + sources in Snowflake connector registry.
services/apps/snowflake_connectors/src/core/transformerBase.ts	Adjusts error logging in safeTransformRow.
backend/src/database/migrations/V1772556158__addTncActivityTypes.sql	Inserts TNC activity types into activityTypes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

lfUsername is sourced from the LFID column, but it’s being added as a TNC MemberIdentityType.USERNAME. That makes the LFID username look like a TNC username and can lead to incorrect member identity matching across platforms. Recommend keeping lfUsername only as a PlatformType.LFID identity, and only emitting a TNC username identity if you have a real TNC username field.

Suggested change

	platform: PlatformType.TNC,
	value: lfUsername,
	type: MemberIdentityType.USERNAME,
	verified: true,
	sourceId,
	},
	{

[Copilot]

Identities are marked verified: true but don’t set verifiedBy. Other integrations (including the Cvent Snowflake transformer) consistently include verifiedBy to capture verification provenance. Consider adding verifiedBy: PlatformType.TNC for verified TNC identities (and the derived LFID identity if applicable).

[Copilot]

Verified identities here omit verifiedBy. The rest of the codebase typically sets verifiedBy whenever verified: true (e.g., verifiedBy: PlatformType.CVENT in the Cvent Snowflake transformer). Suggest adding verifiedBy: PlatformType.TNC to keep provenance consistent.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

SQL filter captures "started" events but descriptions say "completed"

Medium Severity

The courses SQL query filters ca.source = 'course_started', capturing only course-start events. However, the migration descriptions for attempted-course and attempted-exam both say "completed" ("Certification course is completed" / "Certification exam is completed"). Either the SQL filter needs 'course_completed' or the migration descriptions are wrong — either way, the data captured contradicts the activity type semantics. The PR reviewer's comment was about course-vs-exam classification, not this started-vs-completed discrepancy.

Additional Locations (1)

• backend/src/database/migrations/V1772556158__addTncActivityTypes.sql#L4-L6