Executor Module Testing

[jeremymanning]

name: "Executor Module Testing"
status: open
created: 2025-09-04T00:46:14Z
updated: 2025-09-04T00:46:14Z
github: [Will be updated when synced to GitHub]
depends_on: []
parallel: true
conflicts_with: []

Description

Implement comprehensive test coverage for the executor module (clustrix/executor.py) focusing on job lifecycle management, SSH connections, and file transfers. The current coverage is 71% and needs to reach 85%+ by testing job submission, monitoring, and result collection workflows.

Acceptance Criteria

• Test coverage for clustrix/executor.py reaches 85%+
• Job lifecycle testing (submit → monitor → collect results)
• SSH connection management testing with paramiko mocking
• File transfer operations testing (upload/download via SFTP)
• Job status monitoring for all cluster types (SLURM, PBS, SGE, Kubernetes, SSH)
• Error handling for connection failures and timeouts
• Remote environment setup and cleanup testing
• Result serialization and deserialization testing
• All tests pass in CI/CD pipeline
• No reduction in existing test coverage for other modules

Technical Details

Current Coverage Analysis

• File: clustrix/executor.py
• Current: 71% coverage
• Target: 85%+ coverage
• Focus: Job execution workflow and SSH operations

Key Areas to Test

1. Job Lifecycle Management

   • Job submission to different cluster types
   • Job status polling and monitoring
   • Result collection and cleanup
   • Error propagation from remote execution

2. SSH Connection Management

   • Connection establishment with paramiko
   • Connection pooling and reuse
   • Authentication (key-based and password)
   • Connection failure and retry logic

3. File Transfer Operations

   • SFTP upload of serialized functions and data
   • Remote directory creation and management
   • Download of results and logs
   • File permission and cleanup handling

4. Cluster-Specific Operations

   • SLURM job submission and monitoring
   • PBS/SGE scheduler integration
   • Kubernetes pod management
   • SSH-only execution mode

5. Error Scenarios

   • Network connectivity issues
   • Authentication failures
   • Remote execution errors
   • Timeout handling

Testing Strategy

• Mock paramiko SSH/SFTP operations for predictable testing
• Use temporary directories for file operation testing
• Mock cluster scheduler commands (sbatch, qsub, kubectl)
• Test concurrent job execution scenarios
• Verify proper resource cleanup in all error paths

Dependencies

• paramiko library for SSH operations
• pytest and pytest-mock for testing framework
• tempfile for temporary directory testing
• Access to existing executor implementation

Effort Estimate

Size: M (3-4 days)

• Research: 0.5 days (understand current executor implementation)
• Test Design: 0.5 days (plan SSH mocking and job lifecycle tests)
• Implementation: 2-2.5 days (write comprehensive executor tests)
• Validation: 0.5-1 day (verify coverage and CI integration)

Definition of Done

• Executor module test coverage ≥ 85%
• Job lifecycle fully tested for all cluster types
• SSH connection and file transfer operations covered
• Error handling and edge cases tested
• All tests pass locally and in CI
• No regression in other module coverage
• Code review completed and approved
• Documentation updated if needed