SDSTOR-22200

[shosseinimotlagh]

Prevent OOM during volume recovery by releasing cached btree nodes

Problem Statement

During multi-volume recovery, HomeStore experiences OOM (Out of Memory) crashes when the total cached btree nodes across all volumes exceed available memory. This occurs because:

1. Cross-volume cache accumulation: During vol_recovery_start_phase2(), each volume's verify_tree() loads all btree nodes into cache sequentially. Without eviction between volumes, total cache size = sum(all volumes).

2. Clean node waste: After recovery completes, cached nodes from verify_tree() provide zero value but continue consuming cache quota, leaving no room for dirty buffers that actually need writeback.

3. Peak memory formula: With N volumes and per-node overhead of 832 bytes (512B-4096B buffer + MemVector + MemPiece + BtreeNode), peak memory during recovery reaches:
   Total = N × (nodes_per_volume × (buffer_size + overhead))
   Example: 13 volumes × 6.86M nodes × 832 bytes ≈ 77.3GB, exceeding typical 60GB limits.

Root Cause Analysis (GDB Session Evidence)

Extensive GDB investigation on production pod revealed:

• RSS breakdown: 53.14GB total, with btree cache consuming ~40GB
• Zero dirty buffers: All 13 volumes showed dirty_buf_cnt=0, ruling out dirty buffer accumulation
• Zero LRU eviction: No evidence of cache eviction during recovery
• Verified accumulation: Cache = sum(all 13 volumes) instead of max(single volume)
• Per-node cost: 832 bytes confirmed (buffer + 320 bytes overhead for MemVector/MemPiece/BtreeNode structures)
• Primary allocator: FixedBlkAllocator MPMC queue consumed 10.06GB

Solution

This PR implements a two-pronged approach to prevent OOM:

1. Iterative Btree Verification (Performance Improvement)

• Add verify_node_fowarding() with iterative BFS traversal using std::queue
• Releases BtreeNodePtr immediately after processing each node
• Benefit: Reduces lock hold time on ancestor nodes during single-tree traversal
• Important: This is NOT a memory leak fix, only improves lock contention

2. Safe Cache Release After Recovery (OOM Prevention)

• Add release_cached_tree() to safely evict btree node buffers after each volume's recovery
• Add free_node_from_cache() in ssd_btree.hpp to evict buffers with proper safety checks:
  • Not locked (try_lock succeeds)
  • Not dirty (no pending writeback)
  • Reference count permits eviction (accounting for wb_cache permanent reference)
• Key mechanism: Cache::erase(cache_only=true) removes buffer from hash_set/eviction list while wb_cache retains reference for checkpoint flush

3. Integration & Configuration

• Call release_cached_tree() in vol_recovery_start_phase2() AFTER recovery_start_phase2() completes for each volume
• Add release_cache_after_recovery config flag (default: true) with hotswap support
• Add recovery_cache_release_latency metric to track performance overhead
• Update HomeBlksRecoveryStats to log cache release time

Technical Details

Cache Eviction Safety Checks:

static bool free_node_from_cache(bnodeid_t node_id, uint32_t node_size) {
 auto req = writeback_req_t::make_request();
 req->isSyncCall = true;  // Critical: prevents null deref on wb_req->bcp->cp_id
 auto buf = m_blkstore->read(bid, 0, node_size, req, true);
 if (!buf) { return true; }  // Already evicted
 if (!buf->try_lock()) { return false; }  // Locked, skip
 buf->unlock();
 buf.reset();
 m_blkstore->free_blk(bid, boost::none, boost::none, true);  // cache_only=true
 // Verify eviction succeeded
 auto verify_buf = m_blkstore->read(bid, 0, node_size, verify_req, true);
 return (verify_buf == nullptr);
}

Delegation Chain:
- HomeBlks::vol_recovery_start_phase2() → Volume::release_cached_tree() → mapping::release_cached_tree() → Btree::release_cached_tree() → ssd_btree::free_node_from_cache()

Benefits

1. Limits peak memory: max(single volume) instead of sum(all volumes)
  - Example: 6.86GB peak instead of 77.3GB total
2. Frees cache space for dirty buffers: Clean nodes from verify_tree() waste quota; evicting them makes room for dirty buffers needing writeback
3. Defensive against future bugs: Reduces OOM risk from ANY dirty buffer accumulation scenario
4. Zero data loss: cache_only=true ensures wb_cache retains reference for checkpoint flush
5. Configurable: Runtime hotswap via release_cache_after_recovery flag
6. Observable: recovery_cache_release_latency metric tracks performance cost

Changes Summary

Modified Files:
- src/engine/homeds/btree/btree.hpp: Add iterative verify + cache release methods
- src/engine/homeds/btree/ssd_btree.hpp: Add free_node_from_cache() primitive
- src/homeblks/volume/mapping.{hpp,cpp}: Delegation layer updates
- src/homeblks/volume/volume.{hpp,cpp}: Top-level volume interface updates
- src/homeblks/home_blks.{hpp,cpp}: Recovery orchestration + metrics + stats
- src/homeblks/homeblks_config.fbs: Config flag definition
- src/engine/blkalloc/*: Memory tracking improvements (prerequisite commits)

Metrics & Configuration:
- New gauge: recovery_cache_release_latency
- New config: release_cache_after_recovery (default: true, hotswap enabled)
- Updated: HomeBlksRecoveryStats to include m_cache_release_ms

Code Style:
- Applied clang-format across entire codebase for consistency

Testing Notes

- Requires testing on multi-volume recovery scenario (13+ volumes)
- Monitor recovery_cache_release_latency metric for performance overhead
- Verify peak RSS stays within bounds: max(single_volume) instead of sum(all_volumes)
- Confirm dirty_buf_cnt can increase without OOM after cache release
- Test config flag: verify behavior with release_cache_after_recovery=false