Skip to content

fix: use lazy FTP connection initialization to avoid connecting on import#244

Open
ndpvt-web wants to merge 2 commits intoAlertaDengue:mainfrom
ndpvt-web:fix/lazy-ftp-connection
Open

fix: use lazy FTP connection initialization to avoid connecting on import#244
ndpvt-web wants to merge 2 commits intoAlertaDengue:mainfrom
ndpvt-web:fix/lazy-ftp-connection

Conversation

@ndpvt-web
Copy link

Summary

Resolves #242

This PR implements lazy FTP connection initialization for all online_data database modules, preventing the library from connecting to the FTP server at import time.

Problem

When importing PySUS modules, the library immediately connects to the FTP server via module-level .load() calls. If the FTP server is unavailable, the import crashes, making the entire library unusable even for operations that don't require FTP access.

Solution

Added lazy wrapper classes for each database module that defer the FTP connection until data is actually requested:

  • SINAN - Lazy initialization for SINAN database access
  • SIM - Lazy initialization for SIM database access
  • CNES - Lazy initialization for CNES database access
  • SIA - Lazy initialization for SIA database access
  • SIH - Lazy initialization for SIH database access
  • PNI - Lazy initialization for PNI database access
  • CIHA - Lazy initialization for CIHA database access
  • SINASC - Lazy initialization for SINASC database access
  • IBGE - Lazy initialization for IBGE database access

Changes

  • 9 files modified in `pysus/online_data/`
  • 223 lines added, 9 lines removed
  • Each module now wraps its database instance in a lazy class that only calls `.load()` on first attribute access
  • Backward compatible - existing code works without modifications

Testing

  • Verified that `import pysus` succeeds without FTP server connection
  • All modified files have valid Python syntax
  • No unrelated changes included

AI Disclosure

This contribution was developed with the assistance of Claude (AI by Anthropic). The implementation approach, code, and PR description were AI-assisted. All changes are focused on resolving the specific issue described above.

Co-Authored-By: AI Assistant (Claude) ai-assistant@contributor-bot.dev

…port

Resolves AlertaDengue#242. FTP connections are now deferred until data is actually
requested, preventing import failures when the FTP server is unavailable.

Changes:
- Added lazy wrapper classes for all online_data database modules
- SINAN, SIM, CNES, SIA, SIH, PNI, CIHA, SINASC, and IBGE modules
  now use lazy initialization
- FTP connections only occur when methods are called, not at import time
- Maintains backward compatibility with existing API

Co-Authored-By: AI Assistant (Claude) <ai-assistant@contributor-bot.dev>
Signed-off-by: ndpvt-web <ndpvt-web@users.noreply.github.com>
@fccoelho fccoelho requested review from fccoelho and luabida March 10, 2026 10:46
Copy link
Collaborator

@fccoelho fccoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the implementation could be simplified by moving all methods of the _Lazy* classes to a base class and let each derived _Lazy wrapper overload only the _ensure_loaded method that is specific to each database. This will avoid a lot of code duplication.

Also, the _ensure_loaded methods is optimistic that the connection will always work. I think it should handle a potential failure in the connection to the FTP server.

- Move all duplicated lazy wrapper logic into a shared _LazyDatabase
  base class in pysus/online_data/_lazy.py
- Use __getattr__ to transparently proxy all attribute access to the
  underlying database instance, eliminating per-class method stubs
- Add FTP connection error handling: _ensure_loaded now catches
  exceptions and raises ConnectionError with a descriptive message
- Replace 9 individual _Lazy* classes with simple one-liner
  instantiations: e.g. ciha = _LazyDatabase(CIHA)

Addresses review feedback from @fccoelho requesting base class
extraction and FTP error handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ndpvt-web
Copy link
Author

Thanks for the review! I've addressed both points:

  1. Base class to eliminate duplication — Extracted a shared _LazyDatabase base class into pysus/online_data/_lazy.py. It uses __getattr__ to transparently proxy all attribute access to the underlying database instance, so there are zero duplicated method stubs. Each module now just does:

    ciha = _LazyDatabase(CIHA)

    This replaced 9 individual _Lazy* classes (removing ~160 lines of duplicated code).

  2. FTP connection error handling_ensure_loaded now wraps the db_class().load() call in a try/except, logging the error via loguru and raising a ConnectionError with a descriptive message when the FTP server is unavailable.

Updated in the latest commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid connecting to FTP server on import

2 participants