Skip to content

feat: BROS-833: Video vectors#9504

Open
nick-skriabin wants to merge 9 commits intodevelopfrom
fb-bros-833
Open

feat: BROS-833: Video vectors#9504
nick-skriabin wants to merge 9 commits intodevelopfrom
fb-bros-833

Conversation

@nick-skriabin
Copy link
Member

@nick-skriabin nick-skriabin commented Mar 2, 2026

This pull request introduces support for vector-based annotations on video frames by adding new VideoVector and VideoVectorLabels tags, along with their associated backend and frontend logic. It also updates documentation, schema definitions, and region handling to accommodate these new annotation types. Additionally, the PR implements keyframe-based interpolation for smooth vector transitions across video frames.

The most important changes are:

New Video Vector Annotation Support:

  • Added VideoVector and VideoVectorLabels tags to the schema (tags.json) with comprehensive attribute definitions for vector annotation on videos, including support for keyframe interpolation, skeleton mode, and customizable appearance. [1] [2]
  • Implemented the VideoVectorRegionModel in the frontend, including logic for keyframe-based interpolation of vector shapes, region actions, and registration in the region registry. [1] [2] [3] [4] [5] [6] [7] [8]

Documentation Updates:

  • Added detailed parameter documentation for the new VideoVector and VideoVectorLabels tags. [1] [2]

Backend and Data Model Adjustments:

  • Updated backend logic to recognize and correctly process VideoVectorLabels in project configuration and annotation label extraction, ensuring compatibility with existing video annotation workflows. [1] [2]

Frontend Display and Usability Improvements:

  • Improved label display logic for video regions to ensure labels are shown only when appropriate settings are enabled. [1] [2]
  • Registered the new region type for use in the editor and included it in the set of recognized result types.

Schema Cleanup:

  • Removed the curves attribute from the schema for relevant tags, possibly to avoid redundancy or confusion, as curve support is now handled differently. [1] [2]

@nick-skriabin nick-skriabin requested review from a team and hlomzik as code owners March 2, 2026 18:59
@netlify
Copy link

netlify bot commented Mar 2, 2026

Deploy Preview for label-studio-docs-new-theme canceled.

Name Link
🔨 Latest commit 723ee14
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-docs-new-theme/deploys/69a88d95b4c5f20008e0101d

@netlify
Copy link

netlify bot commented Mar 2, 2026

Deploy Preview for heartex-docs canceled.

Name Link
🔨 Latest commit 723ee14
🔍 Latest deploy log https://app.netlify.com/projects/heartex-docs/deploys/69a88d95e87a0e0008339cc6

@netlify
Copy link

netlify bot commented Mar 2, 2026

Deploy Preview for label-studio-playground ready!

Name Link
🔨 Latest commit 723ee14
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-playground/deploys/69a88d9553e23a00085ccadb
😎 Deploy Preview https://deploy-preview-9504--label-studio-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions bot added the feat label Mar 2, 2026
@netlify
Copy link

netlify bot commented Mar 2, 2026

Deploy Preview for label-studio-storybook ready!

Name Link
🔨 Latest commit 723ee14
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-storybook/deploys/69a88d95ca88f00008f80113
😎 Deploy Preview https://deploy-preview-9504--label-studio-storybook.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov
Copy link

codecov bot commented Mar 2, 2026

Codecov Report

❌ Patch coverage is 17.46032% with 364 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.99%. Comparing base (995f2fc) to head (723ee14).
⚠️ Report is 8 commits behind head on develop.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
web/libs/editor/src/tools/VideoVector.js 1.35% 146 Missing ⚠️
.../libs/editor/src/tags/object/Video/VideoVector.jsx 6.94% 134 Missing ⚠️
web/libs/editor/src/regions/VideoVectorRegion.jsx 13.79% 50 Missing ⚠️
...libs/editor/src/tags/object/Video/VideoRegions.jsx 69.38% 15 Missing ⚠️
web/libs/editor/src/tags/object/Video/Video.js 22.22% 14 Missing ⚠️
web/libs/editor/src/tags/control/VideoVector.js 71.42% 2 Missing ⚠️
web/libs/editor/src/components/Node/Node.tsx 0.00% 1 Missing ⚠️
web/libs/editor/src/regions/Area.js 83.33% 1 Missing ⚠️
...libs/editor/src/tags/control/VideoVectorLabels.jsx 80.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9504      +/-   ##
===========================================
- Coverage    68.33%   67.99%   -0.34%     
===========================================
  Files         1163     1168       +5     
  Lines        69069    69489     +420     
  Branches     11802    11932     +130     
===========================================
+ Hits         47195    47248      +53     
- Misses       21867    22234     +367     
  Partials         7        7              
Flag Coverage Δ
lsf-integration 52.44% <17.08%> (-0.46%) ⬇️
lsf-unit 43.71% <12.30%> (-0.35%) ⬇️
pytests 81.58% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nick-skriabin
Copy link
Member Author

nick-skriabin commented Mar 3, 2026

/fmt

Workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants