Refactor: use semantic_check=False for left join bypass

Instead of a special _aggregation parameter, co-opt semantic_check=False
to bypass the left join A → B constraint. When bypassed, PK = PK(A) ∪ PK(B).

This is cleaner because:
- Consistent with existing semantic_check semantics (bypass strict validation)
- User-facing parameter, not an internal hack
- Responsibility is on the caller for any invalid PK from such operations

Aggregation now uses semantic_check=False for its internal left join,
then resets PK via GROUP BY.

Co-authored-by: dimitri-yatsenko<dimitri@datajoint.com>

Author: claude

docs/src/design/semantic-matching-spec.md

@@ -338,7 +338,19 @@ Session.join(Trial, left=True)  # OK: Session → Trial
 ```
 DataJointError: Left join requires the left operand to determine the right operand (A → B).
 The following attributes from the right operand's primary key are not determined by
-the left operand: ['z']. Use an inner join or restructure the query.
+the left operand: ['z']. Use an inner join, restructure the query, or use semantic_check=False.
+```
+
+### Bypassing with `semantic_check=False`
+
+When `semantic_check=False` is used for a left join where A → B doesn't hold, the constraint is bypassed and **PK = PK(A) ∪ PK(B)** is used. This is useful when the caller will reset the primary key afterward (e.g., aggregation with GROUP BY).
+
+```python
+# Direct left join - normally blocked
+A.join(B, left=True)  # Error: A doesn't determine B
+
+# Bypass with semantic_check=False - produces PK(A) ∪ PK(B)
+A.join(B, left=True, semantic_check=False)  # Allowed, but PK may have NULLs
 ```
 
 ### Aggregation Exception
@@ -348,9 +360,10 @@ the left operand: ['z']. Use an inner join or restructure the query.
 This apparent contradiction is resolved by the `GROUP BY` clause:
 
 1. Aggregation requires B → A so that B can be grouped by A's primary key
-2. The intermediate left join `A LEFT JOIN B` would have an invalid PK under the normal left join rules (B → A case gives PK(B))
-3. However, aggregation's `GROUP BY PK(A)` clause **resets** the primary key to PK(A)
-4. The final result has PK(A), which consists entirely of non-NULL values from A
+2. The intermediate left join `A LEFT JOIN B` would have an invalid PK under the normal left join rules
+3. Aggregation uses `semantic_check=False` for its internal join, producing PK(A) ∪ PK(B)
+4. The `GROUP BY PK(A)` clause then **resets** the primary key to PK(A)
+5. The final result has PK(A), which consists entirely of non-NULL values from A
 
 **Example:**
 ```
@@ -360,13 +373,12 @@ Trial: session_id*, trial_num*, response_time    (references Session)
 # Aggregation with keep_all_rows=True
 Session.aggr(Trial, keep_all_rows=True, avg_rt='avg(response_time)')
 
-# Internally: Session LEFT JOIN Trial (B → A, would normally be invalid)
+# Internally: Session LEFT JOIN Trial with semantic_check=False
+# Intermediate PK would be {session_id} ∪ {session_id, trial_num} = {session_id, trial_num}
 # But GROUP BY session_id resets PK to {session_id}
 # Result: All sessions, with avg_rt=NULL for sessions without trials
 ```
 
-The left join constraint validation is bypassed internally for aggregation because the `GROUP BY` clause guarantees a valid primary key in the final result.
-
 ## Universal Set `dj.U`
 
 `dj.U()` or `dj.U('attr1', 'attr2', ...)` represents the universal set of all possible values and lineages.

src/datajoint/expression.py

@@ -282,18 +282,17 @@ def __matmul__(self, other):
             "The @ operator has been removed in DataJoint 2.0. " "Use .join(other, semantic_check=False) for permissive joins."
         )
 
-    def join(self, other, semantic_check=True, left=False, _aggregation=False):
+    def join(self, other, semantic_check=True, left=False):
         """
         Create the joined QueryExpression.
 
         Uses semantic matching: only attributes with the same name AND the same
         lineage (homologous namesakes) are used for joining.
 
         :param other: QueryExpression to join with
-        :param semantic_check: If True (default), raise error on non-homologous namesakes.
-            If False, bypass semantic check (use for legacy compatibility).
+        :param semantic_check: If True (default), raise error on non-homologous namesakes
+            and enforce left join A → B constraint. If False, bypass these checks.
         :param left: If True, perform a left join retaining all rows from self.
-        :param _aggregation: Internal flag to bypass left join validation for aggregation.
 
         Examples:
             a * b  is short for a.join(b)
@@ -337,10 +336,10 @@ def join(self, other, semantic_check=True, left=False, _aggregation=False):
         result._connection = self.connection
         result._support = self.support + other.support
         result._left = self._left + [left] + other._left
-        result._heading = self.heading.join(other.heading, left=left, _aggregation=_aggregation)
+        result._heading = self.heading.join(other.heading, left=left, semantic_check=semantic_check)
         result._restriction = AndList(self.restriction)
         result._restriction.append(other.restriction)
-        result._original_heading = self.original_heading.join(other.original_heading, left=left, _aggregation=_aggregation)
+        result._original_heading = self.original_heading.join(other.original_heading, left=left, semantic_check=semantic_check)
         assert len(result.support) == len(result._left) + 1
         return result
 
@@ -684,8 +683,8 @@ def create(cls, arg, group, keep_all_rows=False):
 
         if keep_all_rows and len(group.support) > 1 or group.heading.new_attributes:
             group = group.make_subquery()  # subquery if left joining a join
-        # Pass _aggregation=True to bypass left join validation (aggregation resets PK via GROUP BY)
-        join = arg.join(group, left=keep_all_rows, _aggregation=True)
+        # Use semantic_check=False to bypass left join A → B validation (aggregation resets PK via GROUP BY)
+        join = arg.join(group, semantic_check=False, left=keep_all_rows)
         result = cls()
         result._connection = join.connection
         result._heading = join.heading.set_primary_key(arg.primary_key)  # use left operand's primary key

src/datajoint/heading.py

@@ -468,7 +468,7 @@ def select(self, select_list, rename_map=None, compute_map=None):
         )
         return Heading(chain(copy_attrs, compute_attrs))
 
-    def join(self, other, left=False, _aggregation=False):
+    def join(self, other, left=False, semantic_check=True):
         """
         Join two headings into a new one.
 
@@ -480,22 +480,22 @@ def join(self, other, left=False, _aggregation=False):
         A → B holds iff every attribute in PK(B) is either in PK(A) or secondary in A.
         B → A holds iff every attribute in PK(A) is either in PK(B) or secondary in B.
 
-        For left joins (left=True), A → B is required. Otherwise, the result would not
-        have a valid primary key because:
+        For left joins (left=True), A → B is required by default. Otherwise, the result
+        would not have a valid primary key because:
         - Unmatched rows from A have NULL values for B's attributes
         - If B → A or Neither, the PK would include B's attributes, which could be NULL
         - Only when A → B does PK(A) uniquely identify all result rows
 
-        Exception: Aggregation (A.aggr(B, keep_all_rows=True)) uses a left join internally
-        but requires B → A instead. This is valid because the GROUP BY clause resets the
-        primary key to PK(A), which consists of non-NULL values from the left operand.
+        When semantic_check=False for left joins where A → B doesn't hold, the constraint
+        is bypassed and PK = PK(A) ∪ PK(B) is used. This is useful for aggregation, where
+        the GROUP BY clause resets the primary key afterward.
 
         It assumes that self and other are headings that share no common dependent attributes.
 
         :param other: The other heading to join with
-        :param left: If True, this is a left join (requires A → B unless _aggregation=True)
-        :param _aggregation: If True, skip left join validation (used by Aggregation.create)
-        :raises DataJointError: If left=True and A does not determine B (unless _aggregation)
+        :param left: If True, this is a left join (requires A → B unless semantic_check=False)
+        :param semantic_check: If False, bypass left join A → B validation (PK becomes union)
+        :raises DataJointError: If left=True, semantic_check=True, and A does not determine B
         """
         from .errors import DataJointError
 
@@ -507,17 +507,23 @@ def join(self, other, left=False, _aggregation=False):
             name in other.primary_key or name in other.secondary_attributes for name in self.primary_key
         )
 
-        # For left joins, require A → B (unless this is an aggregation context)
-        if left and not _aggregation and not self_determines_other:
-            missing = [
-                name for name in other.primary_key if name not in self.primary_key and name not in self.secondary_attributes
-            ]
-            raise DataJointError(
-                f"Left join requires the left operand to determine the right operand (A → B). "
-                f"The following attributes from the right operand's primary key are not "
-                f"determined by the left operand: {missing}. "
-                f"Use an inner join or restructure the query."
-            )
+        # For left joins, require A → B unless semantic_check=False
+        if left and not self_determines_other:
+            if semantic_check:
+                missing = [
+                    name
+                    for name in other.primary_key
+                    if name not in self.primary_key and name not in self.secondary_attributes
+                ]
+                raise DataJointError(
+                    f"Left join requires the left operand to determine the right operand (A → B). "
+                    f"The following attributes from the right operand's primary key are not "
+                    f"determined by the left operand: {missing}. "
+                    f"Use an inner join, restructure the query, or use semantic_check=False."
+                )
+            else:
+                # Bypass: use union of PKs (will be reset by caller, e.g., aggregation)
+                other_determines_self = False  # Force the "Neither" case
 
         seen = set()
         result_attrs = []