Fix LID Group Send Failure (not-acceptable error)

docs/troubleshooting/11-12-25-group-chat-not-acceptable-error.md

@@ -0,0 +1,190 @@
+# Group Chat "not-acceptable" Error - LID Group Send Failure
+
+**Date:** December 11, 2025
+**Status:** PATCH FIXED - Ready for deployment to staging
+**Severity:** HIGH - Blocking group message sends
+**Last Updated:** 2025-12-11 (Patch file format corrected)
+
+## Summary
+Instance `6287777635515` cannot send messages to LID-addressed group `120363407389509612@g.us`. Baileys includes sender's own devices in sender key distribution, causing WhatsApp to reject with "not-acceptable".
+
+## Affected Resources
+- **Group:** `120363407389509612@g.us` (Test Group Chat)
+- **Addressing Mode:** LID (`@lid` JIDs)
+- **Instances:** `6287777635515` (inbox 1), `6287785582370` (inbox 16)
+- **Conversations:** 400 (inbox 1), 401 (inbox 16)
+
+## Timeline
+- **03:49:01** - First error from `6287785582370`
+- **03:49:30** - Error from `6287777635515`
+- **04:08:36** - Error persists (both instances)
+- **04:15:46** - After removing `6287785582370` from group, error continues
+- **04:16:38** - After restarting instance, error still occurs (msg 3065674, 3065679)
+
+## Root Cause: Baileys Sender Key Bug with LID Groups
+
+### The Core Issue
+When sending to LID-addressed groups, Baileys includes **the sender's own devices** in the sender key distribution list. WhatsApp's server rejects this with "Internal Server Error" → "not-acceptable".
+
+### Evidence (Post-Removal, 04:15:50 UTC)
+```json
+{
+  "senderKeyJids": [
+    "219232478433331:30@s.whatsapp.net",  // Admin device ✅
+    "22033668821180@s.whatsapp.net",      // SENDER ITSELF ❌
+    "22033668821180:87@s.whatsapp.net",   // SENDER devices ❌
+    "22033668821180:95@s.whatsapp.net",   // SENDER devices ❌
+    "22033668821180:98@s.whatsapp.net"    // SENDER devices ❌
+  ],
+  "msg": "sending new sender key"
+}
+```
+**Sender:** `6287777635515` (LID: `22033668821180`)
+**Problem:** Baileys trying to send sender keys to sender's own JID/devices
+
+### Group Configuration
+- **Addressing Mode:** LID
+- **Participants (3):**
+  1. `219232478433331@lid` → `6287791121908@s.whatsapp.net` (admin)
+  2. `161663189794896@lid` → `6287785582370@s.whatsapp.net` (member)
+  3. `22033668821180@lid` → `6287777635515@s.whatsapp.net` (member)
+
+## What We Tested
+
+### ✅ Proven
+1. **Typing indicator works** - Connection is fine
+2. **Receiving messages works** - Decryption succeeds (with retry receipts)
+3. **Error is consistent** - Both instances (`5515`, `2370`) fail identically
+4. **Sender key issue** - Baileys sends keys to sender's own devices in LID groups
+5. **Not cached metadata** - Restart + metadata refresh didn't fix it
+
+### ❌ Disproven Theories
+1. **Encryption session loss** - Re-adding to group didn't fix (attempted with `2370`)
+2. **Stale group metadata** - Restart forced refresh, error persists
+3. **Two instances conflict** - Removing `2370` didn't fix `5515` send
+4. **Container state** - Full restart didn't resolve
+
+### 🔍 Key Evidence
+**File:** `full_send_sequence.json` (04:08:33 UTC)
+```json
+{"jidsRequiringFetch":["219232478433331:30@s.whatsapp.net","161663189794896:22@s.whatsapp.net","22033668821180:95@s.whatsapp.net","22033668821180:98@s.whatsapp.net"],"msg":"fetching sessions"}
+{"senderKeyJids":["219232478433331@s.whatsapp.net","219232478433331:30@s.whatsapp.net","161663189794896@s.whatsapp.net","161663189794896:22@s.whatsapp.net","22033668821180@s.whatsapp.net","22033668821180:87@s.whatsapp.net","22033668821180:95@s.whatsapp.net","22033668821180:98@s.whatsapp.net"],"msg":"sending new sender key"}
+```
+Sender `22033668821180` trying to establish keys with **itself**.
+
+## CONFIRMED: Baileys Sender Key Bug in Group Messages
+
+### Root Cause (Code Analysis)
+**File:** `node_modules/baileys/lib/Socket/messages-send.js`
+
+**The Bug (lines 339-372):**
+```javascript
+// Line 340: Gets ALL participants including sender
+const participantsList = groupData.participants.map(p => p.id);
+
+// Line 350: Fetches devices for ALL participants (no exclusion)
+const additionalDevices = await getUSyncDevices(participantsList, ...);
+
+// Lines 365-372: NO SENDER FILTERING - all devices go to senderKeyJids
+for (const { user, device } of devices) {
+    senderKeyJids.push(jid);  // ← SENDER'S OWN DEVICES INCLUDED
+}
+```
+
+**Why Non-Group Works:**
+In 1:1 messages (lines 407-419), Baileys properly separates meJids vs otherJids:
+```javascript
+const isMe = user === meUser;
+if (isMe) { meJids.push(jid); } else { otherJids.push(jid); }
+```
+
+**Why Group Fails:**
+Group message path has NO such filtering. All participant devices go into senderKeyJids.
+
+### Why LID Exacerbates
+- `meId` = `6287777635515@s.whatsapp.net` (phone format)
+- `participantsList[i]` = `22033668821180@lid` (LID format)
+- JID format mismatch would break any comparison even if filtering existed
+
+## Fix Implemented
+
+### Patch Applied: Baileys senderKeyJids Filtering
+**File:** `node_modules/baileys/lib/Socket/messages-send.js` (lines 363-380)
+**Patch File:** `patches/baileys+6.7.19+lid-group-send.patch`
+
+```javascript
+const senderKeyJids = [];
+// [WIDGET-WORKS] Filter sender's own devices from senderKeyJids (LID group send fix)
+// Same pattern as non-group path (line ~396) - fixes "not-acceptable" error
+const { user: meUser } = jidDecode(meId);
+const meLid = authState.creds?.me?.lid?.split(':')[0]?.split('@')[0];
+// ensure a connection is established with every device
+for (const { user, device } of devices) {
+    // [WIDGET-WORKS] Skip sender's own devices
+    if (user === meUser || user === meLid) {
+        continue;
+    }
+    const jid = jidEncode(user, groupData?.addressingMode === 'lid' ? 'lid' : 's.whatsapp.net', device);
+    if (!senderKeyMap[jid] || !!participant) {
+        senderKeyJids.push(jid);
+        senderKeyMap[jid] = true;
+    }
+}
+```
+
+### Upstream Research Summary
+- **Baileys v6.7.17+**: Has LID group send fixes but NOT this specific senderKeyJids filtering
+- **PR #1731**: Fixes sender key initialization, NOT distribution target filtering
+- **Open Issues**: #1690 (Groups Sessions – LID), #1768 (LID bugs) - ongoing
+- **Conclusion**: This bug is NOT fixed upstream; patch required
+
+### Re-applying After npm install
+```bash
+# Manual patch application
+patch -p0 < patches/baileys+6.7.19+lid-group-send.patch
+```
+
+### Deployment Issues Encountered
+
+**Issue 1: Missing `patch` binary in Alpine Linux (Commit: 38b01376)**
+- **Problem**: Docker build failed because `node:20-alpine` base image doesn't include `patch` utility
+- **Solution**: Added `patch` to `apk` install list in Dockerfile line 4
+- **Fix**: `RUN apk update && apk add --no-cache git ffmpeg wget curl bash openssl patch`
+
+**Issue 2: Malformed patch file format (Commits: b1c030c3, 518f76d2)**
+- **Problem**: Initial patch had git diff headers but incomplete content, causing "malformed patch at line 19" error
+- **Root Cause**: Patch file ended prematurely at line 21 without proper trailing context lines
+- **Solution**: Regenerated patch with complete unified diff format including 3+ lines of trailing context
+- **Result**: Patch file now properly shows lines 360-367 (original 8 lines) → 360-373 (new 14 lines)
+
+**Issue 3: Git-style a/b prefixes incompatible with -p0 (Commits: 98276760, 4ce427b8)**
+- **Problem**: Patch file had `--- a/node_modules/...` and `+++ b/node_modules/...` headers
+- **Root Cause**: With `patch -p0`, the utility looks for literal path `a/node_modules/...` which doesn't exist
+- **Solution**: Removed `a/` and `b/` prefixes so patch looks for correct path `node_modules/baileys/...`
+- **Result**: Patch now compatible with `-p0` flag used in Dockerfile
+
+**Verification**:
+```bash
+# Test patch application locally before Docker build
+cd /path/to/evolution-api
+patch -p0 --dry-run < patches/baileys+6.7.19+lid-group-send.patch
+```
+
+## Alternative Options (Not Implemented)
+
+### Option 2: Override cachedGroupMetadata in Evolution
+Use `option.cachedGroupMetadata` callback to return participants excluding sender.
+**File:** `whatsapp.baileys.service.ts:1991-1997`
+
+### Option 3: Report Upstream
+Open issue in WhiskeySockets/Baileys with this analysis.
+
+## References
+- Perplexity research confirms this is NOT a known documented bug
+- Baileys version: Check `package.json` for exact version
+- Related: Baileys issue #1854 (group send crashes, different root cause)
+
+## Related Files
+- `src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts:1977-2053` (sendMessage)
+- `node_modules/baileys/lib/Socket/messages-send.js:696` (error origin)
+- `docs/troubleshooting/08-12-25-indorama-lid-conversation-split.md` (LID normalization context)

patches/baileys+6.7.19+lid-group-send.patch

@@ -1,6 +1,7 @@
---- a/node_modules/baileys/lib/Socket/messages-send.js
-+++ b/node_modules/baileys/lib/Socket/messages-send.js
+--- node_modules/baileys/lib/Socket/messages-send.js
++++ node_modules/baileys/lib/Socket/messages-send.js
 @@ -360,8 +360,14 @@
+                     data: bytes,
                      meId
                  });
                  const senderKeyJids = [];