Skip to content

Conversation

@hsparks-codes
Copy link

@hsparks-codes hsparks-codes commented Dec 5, 2025

feat: Add Runway video generation service

Summary

Adds RunwayService for video generation using Runway ML API. This fills a critical gap in the generative miner - the video modality now has an API-based service option.

Closes #303

Motivation

Currently, generative miners can only generate videos using local models, which requires significant GPU resources. This PR adds Runway ML as an API-based alternative, enabling miners to participate in video generation tasks without local GPU infrastructure.

This complements PR #301 (StabilityAI for images) - together they provide complete API coverage for both modalities.

Changes

File Description
neurons/generator/services/runway_service.py New video generation service
neurons/generator/services/service_registry.py Register RunwayService in SERVICE_MAP
.env.gen_miner.template Add Runway configuration options
docs/Generative-Mining.md Add Runway service documentation
tests/generator/runway_service.py Test suite matching existing patterns

Features

  • Text-to-video generation - Generate videos from text prompts
  • Image-to-video - Animate still images (via URL or bytes)
  • Gen-3 Alpha Turbo model - Fast, high-quality video generation
  • Flexible duration - 5 or 10 second video clips
  • Aspect ratios - 1280:768 (landscape), 768:1280 (portrait)
  • Configurable - Via RUNWAY_MODEL and RUNWAY_DEFAULT_DURATION env vars
  • C2PA Content Credentials - Videos include signed provenance metadata from RUNWAY AI, INC.

Configuration

# In .env.gen_miner
VIDEO_SERVICE=runway
RUNWAYML_API_SECRET=your_api_key_here

# Optional
RUNWAY_MODEL=gen3a_turbo
RUNWAY_DEFAULT_DURATION=5  # 5 or 10 seconds

Testing

All tests pass with real API:

========== Runway Service Full Test Suite ==========

=== Testing aspect ratio conversion ===
✔ 1920x1080 -> 1280:768
✔ 1080x1920 -> 768:1280
✔ 1024x1024 -> 1280:768

=== Testing service info ===
✔ Service info validated

=== Testing invalid modality ===
✔ Correctly rejected image modality

=== Running generation test for model: gen3a_turbo, duration: 5s ===
✔ Generated video in 21.37s (2577.7 KB)
✔ Metadata keys: ['model', 'provider', 'duration', 'ratio', 'generation_time']
✔ Saved to outputs/gen3a_turbo_5s.mp4
=== Model gen3a_turbo PASSED ===

=== Testing invalid API key ===
✔ Correctly failed with 401 error

========== All Tests Completed ==========
Test Result
Aspect ratio conversion ✅ PASSED
Service info validation ✅ PASSED
Invalid modality rejection ✅ PASSED
Video generation (gen3a_turbo) ✅ PASSED (2.5 MB, 21s)
Invalid API key handling ✅ PASSED

Implementation Notes

  • Follows exact same patterns as stabilityai_service.py and openai_service.py
  • Sync process() method to match existing services
  • Uses official Runway API with proper polling for task completion
  • Supports both image URL and base64 encoded image input
  • Consistent metadata structure with other services
  • C2PA credentials included - Runway embeds signed Content Credentials in generated videos

Checklist

  • Code follows existing service patterns
  • Documentation updated
  • Environment template updated
  • Tests included and passing
  • Tested with real API - video generation works
  • No breaking changes
  • C2PA verified - Generated videos include Content Credentials signed by RUNWAY AI, INC.

Note

Add Runway ML video generation service with registry integration, env/docs configuration, and tests.

  • Services:
    • New RunwayService (neurons/generator/services/runway_service.py): Implements Runway ML video generation with text-to-video and image-to-video, polling/status handling, aspect ratio selection, 5/10s durations, and env-configurable model/duration; exposes service info and API key requirements.
    • Registry Update (neurons/generator/services/service_registry.py): Adds runway to SERVICE_MAP; updates modality selection to allow VIDEO_SERVICE=runway and documents options.
  • Configuration & Docs:
    • Env Template (.env.gen_miner.template): Adds RUNWAYML_API_SECRET and optional RUNWAY_MODEL/RUNWAY_DEFAULT_DURATION; updates service selection notes and API key links.
    • Docs (docs/Generative-Mining.md): Adds Runway service section with features, pricing, and setup instructions.
  • Tests:
    • New Suite (tests/generator/runway_service.py): Tests aspect ratio conversion, service info, invalid modality, env model validation, happy-path generation, and invalid API key handling.

Written by Cursor Bugbot for commit 6a7c800. This will update automatically on new commits. Configure here.

Contribution by Gittensor, learn more at https://gittensor.io/

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@hsparks-codes hsparks-codes force-pushed the feat/video-generation-service branch 2 times, most recently from b223cf2 to 9bb6b61 Compare December 5, 2025 22:56
@hsparks-codes hsparks-codes force-pushed the feat/video-generation-service branch from 9bb6b61 to 7b2c884 Compare December 5, 2025 23:27
Adds RunwayService for video generation using Runway ML API.
This fills a gap in the generative miner - video modality now has an API service.

Features:
- Text-to-video generation
- Image-to-video (animate still images via URL or bytes)
- Gen-3 Alpha Turbo model support
- 5 or 10 second video clips
- Aspect ratios: 1280:768 (landscape), 768:1280 (portrait)
- Configurable via RUNWAY_MODEL and RUNWAY_DEFAULT_DURATION env vars

Tested with real API - video generation works correctly.

Complements PR BitMind-AI#301 (StabilityAI for images) - this PR adds video support.

Closes BitMind-AI#303
@hsparks-codes hsparks-codes force-pushed the feat/video-generation-service branch from 7b2c884 to 412472e Compare December 5, 2025 23:42
@hsparks-codes
Copy link
Author

@dylanuys Please check the PR and give me your feedback.

Copy link
Contributor

@dylanuys dylanuys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please attach a generated mp4 file (or send to @_dubm on discord if too large for github) so that I can verify c2pa metadata for Runway data.

@hsparks-codes
Copy link
Author

@dylanuys Just added the test video file.

=== C2PA Verification ===

✅ C2PA manifest found!

Signature Info:
  Issuer: RUNWAY AI, INC.
  Common Name: eng-leadership@runwayml.com
  Time: 2025-12-07T12:22:20+00:00

AI Generation Assertion:
  Action: c2pa.created
  Software Agent: RunwayML Video Generation
  Digital Source Type: trainedAlgorithmicMedia

Runway videos include proper C2PA Content Credentials signed by RUNWAY AI, INC. with AI generation assertions.

Resolves conflict in service_registry.py by including both
stabilityai and runway services.
- Re-raise AssertionError so test_invalid_api_key properly fails
  if API unexpectedly succeeds
- Delete env var in finally block when original was None,
  matching pattern from test_model_env_validation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create additional generation service wrappers for gen miners

2 participants