Implement comprehensive LRU caching to reduce Ethereum RPC and IPFS gateway calls

[Copilot]

This PR implements a robust caching solution to address performance issues with repeated Ethereum RPC and IPFS gateway calls throughout the DAppNode system. The solution introduces a new @dappnode/cache package with LRU (Least Recently Used) cache functionality and integrates it into three critical areas.

Problem

DAppNode was making unnecessary repeated calls to:

• Ethereum RPC endpoints for smart contract addresses
• IPFS gateways for dappstore package information
• Database queries for staker configuration data

This resulted in increased latency, potential rate limiting, and inefficient resource usage.

Solution

New Cache Package (@dappnode/cache)

Created a dedicated caching package built on lru-cache v11.x with:

• Generic DappnodeCache class with configurable TTL and size limits
• Pre-configured cache instances optimized for specific use cases
• Helper utilities for common cache patterns
• Comprehensive test suite with 15 test cases

Cache Integration Points

1. DAppStore Data Caching (fetchDirectory)

// Before: Direct API call on every request
const release = await dappnodeInstaller.getRelease(name);

// After: Cached with 30-minute TTL
const release = await getOrSet(
  dappstoreCache,
  `release:${name}`,
  async () => await dappnodeInstaller.getRelease(name)
);

2. Smart Contract Address Caching (updateMyPackages)

// Before: In-memory Map with no eviction
const contractAddressMap = new Map<string, string>();

// After: LRU cache with 24-hour TTL and automatic eviction
contractAddressCache.set(dnpName, contractAddress);

3. Staker Configuration Caching (stakerConfigGet)

// Before: Database query on every request
return await Promise.all([
  await execution.getAllExecutions(network),
  // ... other queries
]);

// After: Cached with 10-minute TTL and invalidation on updates
return await getOrSet(stakerConfigCache, `stakerConfig:${network}`, fetchConfig);

Cache Configuration

Each cache is optimized for its specific use case:

• dappstoreCache: 30min TTL, 500 items (package metadata changes infrequently)
• contractAddressCache: 24hr TTL, 200 items (contract addresses rarely change)
• stakerConfigCache: 10min TTL, 50 items (more frequent updates expected)

Performance Benefits

• Reduced External Calls: Eliminates redundant Ethereum RPC and IPFS gateway requests
• Lower Latency: Cached responses return immediately without network round-trips
• Improved Reliability: Less dependency on external service availability
• Memory Efficient: LRU eviction prevents unbounded memory growth
• Data Freshness: TTL ensures cached data doesn't become stale

Testing

All functionality is thoroughly tested with:

• Unit tests for cache operations (set, get, delete, eviction)
• Integration tests for real-world usage patterns
• TTL expiration and LRU eviction validation
• Cache invalidation scenarios

The implementation maintains full backward compatibility while providing significant performance improvements for high-traffic operations.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• binaries.soliditylang.org
  • Triggering command: /usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16 (dns block)
• ethereum.github.io
  • Triggering command: /usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16 (dns block)
• ns.dappnode.io
  • Triggering command: /usr/local/bin/node --experimental-specifier-resolution=node --import=tsx/esm /home/REDACTED/work/DNP_DAPPMANAGER/DNP_DAPPMANAGER/packages/utils/node_modules/mocha/lib/cli/cli.js ./test/unit --no-config --recursive --no-package --color --exit --extension ts --require dotenv/config --diff --reporter spec --slow 75 --timeout 2000 --ui bdd --watch-ignore node_modules --watch-ignore .git (dns block)
• ofcncog2cu-dsn.algolia.net
  • Triggering command: node /usr/local/bin/yarn add @dappnode/cache@workspace:^0.1.0 (dns block)
• relay.trufflesuite.com
  • Triggering command: /usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16 (dns block)
• solc-bin.ethereum.org
  • Triggering command: /usr/local/bin/node ./build/cli.bundled.js obtain --solc=0.5.16 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Implement caching to reduce unnecessary Ethereum RPC and IPFS gateway calls</issue_title>
<issue_description>Is your feature request related to a problem? Please describe.

Currently, dappmanager makes repeated Ethereum RPC and IPFS gateway calls for retrieving dappstore information and smart contract addresses, leading to inefficiencies and potentially increased latency. The staker config is also fetched repeatedly from the frontend. Existing caching for smart contract addresses is limited to an in-memory map.

Describe the solution you'd like

Implement a proper caching class in a new package module within dappmanager, leveraging a robust caching library such as @isaacs/node-lru-cache to provide controlled cache eviction and performance improvements. The caching class should support:

• Caching dappstore data to avoid unnecessary Ethereum RPC and IPFS gateway calls (used in the dappmanager module, in calls, fetchdirectory).
• Caching smart contract addresses from the auto-updates daemon (currently cached in-memory in the daemons module, updateMyPackages).
• Caching staker config when retrieved from the frontend (used in dappmanager module, calls, stakerConfig).

Describe alternatives you've considered

• Continue using in-memory maps for caching (less robust, harder to maintain and scale).
• Use Redis or other distributed caching solutions (may be overkill for current use case but could be considered if scaling needs grow).

Additional context

• The cache should be implemented as a reusable class/module.
• Integration points: fetchdirectory (dappstore), updateMyPackages (smart contract addresses), stakerConfig (staker config).
• Reference: @isaacs/node-lru-cache
  </issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #2280

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[github-actions]

🚀 Deployed on https://68ee2c83c53a200f8c3fe3dd--objective-borg-9eeae5.netlify.app