Skip to content

ENG-1475 Export relations to imported nodes#817

Open
maparent wants to merge 1 commit intoeng-1476-repair-lint-ci-task-for-databasefrom
eng-1475-export-relations-to-imported-nodes
Open

ENG-1475 Export relations to imported nodes#817
maparent wants to merge 1 commit intoeng-1476-repair-lint-ci-task-for-databasefrom
eng-1475-export-relations-to-imported-nodes

Conversation

@maparent
Copy link
Collaborator

@maparent maparent commented Feb 23, 2026

https://linear.app/discourse-graphs/issue/ENG-1475/export-relations-to-imported-nodes

https://www.loom.com/share/527b312dca4f441083f9ae25987e5f57

Code note: the main work was to send the original node's koi identifier in upsertConcept, and have that function resolve the identifier to the original node.


Open with Devin

@linear
Copy link

linear bot commented Feb 23, 2026

@supabase
Copy link

supabase bot commented Feb 23, 2026

Updates to Preview Branch (eng-1475-export-relations-to-imported-nodes) ↗︎

Deployments Status Updated
Database Sat, 28 Feb 2026 03:15:10 UTC
Services Sat, 28 Feb 2026 03:15:10 UTC
APIs Sat, 28 Feb 2026 03:15:10 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations Sat, 28 Feb 2026 03:15:11 UTC
Migrations Sat, 28 Feb 2026 03:15:11 UTC
Seeding Sat, 28 Feb 2026 03:15:11 UTC
Edge Functions Sat, 28 Feb 2026 03:15:14 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

@maparent maparent marked this pull request as draft February 23, 2026 15:07
@maparent maparent changed the base branch from main to eng-1344-upload-relation-and-rel-schema February 23, 2026 15:07
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

View 8 additional findings in Devin Review.

Open in Devin Review

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from ad7af74 to ce78692 Compare February 23, 2026 16:29
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch from 30e40b2 to bb38514 Compare February 23, 2026 16:30
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from ce78692 to b87afad Compare February 23, 2026 17:17
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch 2 times, most recently from 6784bf7 to 91b4992 Compare February 23, 2026 19:02
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 3 times, most recently from 9d6b14b to 1b31245 Compare February 24, 2026 20:25
@maparent maparent force-pushed the eng-1344-upload-relation-and-rel-schema branch from cc52d2f to a6f2d9e Compare February 24, 2026 20:26
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 2 times, most recently from ce150f3 to 3169b1e Compare February 25, 2026 17:30
Base automatically changed from eng-1344-upload-relation-and-rel-schema to main February 25, 2026 18:26
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch 3 times, most recently from 7489026 to 2df080d Compare February 27, 2026 18:16
@maparent maparent changed the base branch from main to eng-1476-repair-lint-ci-task-for-database February 27, 2026 18:17
@maparent maparent marked this pull request as ready for review February 27, 2026 18:17
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 12 additional findings in Devin Review.

Open in Devin Review

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Missing legacy importedFromSpaceUri check in collectDiscourseNodesFromPaths allows imported files to be synced

Same root cause as in shouldSyncFile: collectDiscourseNodesFromPaths only checks frontmatter.importedFromRid but not the legacy frontmatter.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault at apps/obsidian/src/utils/getDiscourseNodes.ts:32-38 correctly checks both fields, but this function was not updated consistently.

Root Cause and Impact

This function is used by syncDiscourseNodeChanges (called from FileChangeListener). During the migration window or if migration fails, any file change event on a file that still has only the legacy importedFromSpaceUri will pass through both shouldSyncFile and collectDiscourseNodesFromPaths guards, causing the imported node to be synced to Supabase as a locally-created node. This is a second guard that should also check the legacy field for defense-in-depth.

(Refers to lines 717-720)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@maparent maparent force-pushed the eng-1476-repair-lint-ci-task-for-database branch from 9121ca2 to 6aa3c39 Compare February 28, 2026 00:12
@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 2df080d to 21b378d Compare February 28, 2026 00:16
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 15 additional findings in Devin Review.

Open in Devin Review

Comment on lines +426 to 429
const allNodes = await collectDiscourseNodesFromVault(plugin, true);

const changedNodeInstances = relationsOnly
? []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Imported nodes are unintentionally synced as Content and Concepts to the user's own space

Changing collectDiscourseNodesFromVault(plugin) to collectDiscourseNodesFromVault(plugin, true) causes allNodes to include imported nodes. When relationsOnly is false, these imported nodes are passed directly to buildChangedNodesFromNodes at line 430. Since imported nodes were never synced as Content to this space, detectNodeChanges (syncDgNodesToSupabase.ts:298-301) treats them as brand-new files (returning ["title", "content"]), so they end up in changedNodeInstances. They then flow into:

  1. upsertNodesToSupabaseAsContentWithEmbeddings (line 444) — creating Content rows and fetching embeddings for nodes that belong to another space
  2. discourseNodeInstanceToLocalConcept via convertDgToSupabaseConcepts (line 452) — creating duplicate Concept rows in the user's space
Root Cause and Impact

The intent of passing includeImported: true was to populate allNodesById in convertDgToSupabaseConcepts so that relationInstanceToLocalConcept can look up imported nodes and use their importedFromRid for local_reference_content. However, the same allNodes array is also fed into buildChangedNodesFromNodes, which has no filter for imported nodes.

Impact: Every sync (when relationsOnly is false) will re-upload all imported nodes as Content with embeddings (wasting API calls and storage), and create duplicate Concept entries in the user's own space for nodes that are supposed to be cross-space references. This could also cause unique constraint violations on (space_id, name) if the imported node's file path collides with an existing concept name.

The fix should separate the two uses: pass only non-imported nodes to buildChangedNodesFromNodes, while passing all nodes (including imported) to convertDgToSupabaseConcepts for the allNodesById lookup.

(Refers to lines 426-434)

Prompt for agents
In apps/obsidian/src/utils/syncDgNodesToSupabase.ts, in the syncAllNodesAndRelations function (lines 406-469), the allNodes variable (line 426) now includes imported nodes via collectDiscourseNodesFromVault(plugin, true). This is needed for convertDgToSupabaseConcepts (line 452-459) but NOT for buildChangedNodesFromNodes (line 430-434). Fix by collecting two sets of nodes:

1. allNodes (with imported) for passing to convertDgToSupabaseConcepts
2. localNodes (without imported) for passing to buildChangedNodesFromNodes

For example, replace lines 426-434 with something like:

    const allNodes = await collectDiscourseNodesFromVault(plugin, true);
    const localNodes = allNodes.filter(n => !n.frontmatter.importedFromRid);

    const changedNodeInstances = relationsOnly
      ? []
      : await buildChangedNodesFromNodes({
          nodes: localNodes,
          supabaseClient,
          context,
        });

This ensures imported nodes are available in allNodesById for relation reference resolution but are not synced as Content/Concepts to the user's own space.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 21b378d to 2df8ee0 Compare February 28, 2026 01:08
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

resourceIds.add(triple.id);
}
});
if (!resourceIds) return;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 !resourceIds is always false because a Set object is always truthy

The guard if (!resourceIds) return; at line 204 is intended to skip the Supabase upsert when there are no resource IDs to publish. However, resourceIds is a Set<string> (initialized on line 165), and a Set object is always truthy in JavaScript regardless of its size. This means the check never triggers an early return.

Root Cause and Impact

The condition !resourceIds evaluates the truthiness of the Set object itself, not its size. Since any object (including an empty Set) is truthy in JavaScript, !resourceIds is always false.

The correct check should be if (resourceIds.size === 0) return;.

Impact: When no matching relation triples are found, the function proceeds to call client.from("ResourceAccess").upsert(...) with an empty array ([...resourceIds.values()] yields []). Depending on how Supabase handles an empty upsert, this could be a no-op or could cause an unnecessary network request. More importantly, the function then proceeds to loadRelations, iterate over relations, and potentially call saveRelations — all unnecessary work.

Suggested change
if (!resourceIds) return;
if (resourceIds.size === 0) return;
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@maparent maparent force-pushed the eng-1475-export-relations-to-imported-nodes branch from 2df8ee0 to de9dff5 Compare February 28, 2026 03:14
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

🐛 1 issue in files not directly in the diff

🐛 Missing legacy importedFromSpaceUri check in shouldSyncFile allows imported files to be synced (apps/obsidian/src/utils/fileChangeListener.ts:101-103)

After the migration from importedFromSpaceUri to importedFromRid, shouldSyncFile only checks frontmatter?.importedFromRid but does NOT check the legacy frontmatter?.importedFromSpaceUri. The refactored collectDiscourseNodesFromVault in getDiscourseNodes.ts:32-38 correctly checks both fields:

if (
  (frontmatter.importedFromRid || frontmatter.importedFromSpaceUri) &&
  includeImported !== true
) { continue; }
Root Cause and Impact

The FileChangeListener is initialized immediately (apps/obsidian/src/index.ts:56-57) via this.fileChangeListener.initialize(), while migrateImportedFromFrontMatter runs asynchronously inside the fire-and-forget void initializeSupabaseSync(this) call at apps/obsidian/src/index.ts:47. During the migration window (or if migration fails), files that still have only importedFromSpaceUri in their frontmatter will pass shouldSyncFile, be queued, and eventually be synced to Supabase as locally-created nodes. This could create duplicate concepts or corrupt data in the remote database.

View 18 additional findings in Devin Review.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant