Update display size scale tests with video frame from video file by haoxli · Pull Request #4614 · gpuweb/cts

haoxli · 2026-03-17T07:56:31Z

For display‑size scale tests, a VideoFrame created from a video file is the right source to validate importExternalTexture because it exercises the real decode, color conversion, and display‑size metadata path. A canvas ImageData frame bypasses the video decode path, so it’s removed.

Added a focused video list to validate scaling across codec and color‑space variation, covering both codecs (VP9, H.264) and both color spaces (bt.601, bt.709).

Issue: #4605

Requirements for PR author:

All missing test coverage is tracked with "TODO" or .unimplemented().
New helpers are /** documented */ and new helper files are found in helper_index.txt.
Test behaves as expected in a WebGPU implementation. (If not passing, explain above.)
Test have be tested with compatibility mode validation enabled and behave as expected. (If not passing, explain above.)

Requirements for reviewer sign-off:

Tests are properly located.
Test descriptions are accurate and complete.
Tests provide complete coverage (including validation control cases). Missing coverage MUST be covered by TODOs.
Tests avoid over-parameterization (see case count report).

When landing this PR, be sure to make any necessary issue status updates.

For display‑size scale tests, a VideoFrame created from a video file is the right source to validate importExternalTexture because it exercises the real decode, color conversion, and display‑size metadata path. A canvas ImageData frame bypasses the video decode path, so it’s removed. Added a focused video list to validate scaling across codec and color‑space variation, covering both codecs (VP9, H.264) and both color spaces (bt.601, bt.709). Issue: gpuweb#4605

github-actions · 2026-03-17T08:03:15Z

Results for build job (at 183d872):

-webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_diff_with_coded_size:* - 3 cases, 3 subcases (~1/case)
-webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_from_textureDimensions:* - 3 cases, 3 subcases (~1/case)
+webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_scale:* - 9 cases, 9 subcases (~1/case)
+webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_from_textureDimensions:* - 9 cases, 9 subcases (~1/case)
-TOTAL: 280752 cases, 2322091 subcases
+TOTAL: 280764 cases, 2322103 subcases

shaoboyan091 · 2026-03-19T04:52:01Z

src/webgpu/web_platform/external_texture/video.spec.ts

+    const canvasContext = canvas.getContext('2d', { colorSpace: 'srgb' });
+    if (canvasContext === null) {
+      frame.close();
+      t.skip(' onscreen canvas 2d context not available');


leading space, pls remove.

shaoboyan091 · 2026-03-19T04:56:20Z

src/webgpu/web_platform/external_texture/video.spec.ts

 }

-function createVideoFrameWithDisplayScale(
+async function createVideoFrameWithDisplayScale(


Would you mind to add comments similar to your commit message to simple describe we need real video file instead of canvas created ones?

shaoboyan091 · 2026-03-19T05:03:14Z

src/webgpu/web_platform/external_texture/video.spec.ts


-  if (canvasContext === null) {
-    t.skip(' onscreen canvas 2d context not available');
+  if (sourceFrame === undefined || codedWidth === undefined || codedHeight === undefined) {


Using assert? Or skip with correct info

shaoboyan091 · 2026-03-19T05:05:09Z

src/webgpu/web_platform/external_texture/video.spec.ts

-
-  let displayWidth = kWidth;
-  let displayHeight = kHeight;
+  let displayWidth = codedWidth;


Or just assign 0 instead of codedWidth.

shaoboyan091 · 2026-03-19T05:10:36Z

src/webgpu/web_platform/external_texture/video.spec.ts

+  'four-colors-vp9-bt709.webm',
+] as const;
+type DisplayScale = 'smaller' | 'same' | 'larger';
+const kDisplayScales: DisplayScale[] = ['smaller', 'same', 'larger'];


If thit won't expand in future then I slightly lean to keep the origin format (Using 'smaller' | 'same' | 'larger' in all three places).

shaoboyan091 · 2026-03-19T05:16:58Z

src/webgpu/web_platform/external_texture/video.spec.ts

+    // Build expected sampled colors by drawing the same source frame to a 2D canvas.
+    // This makes the check robust across codecs/container metadata while still validating
+    // that importExternalTexture sampling matches browser video rendering behavior.
+    const canvas = createCanvas(t, 'onscreen', kWidth, kHeight);


Just double check, are we using kWidth, kHeight instead of using codedWidth, codedHeight for some purpose?

The origin part use kWidth and kHeight due to the generated video frame is from a kWidth and kHeight (which mapping to codedWidth and codedHeight here) canvas.

That's not entirely accurate. I only used kWidth and kHeight to avoid having to define an additional size variable here. We also can use displayWidth and displayHeight in VideoFrame for that. In the origin PR, I used canvas becuase I found it also can reporduce the issue and I can control the sizes easily.

shaoboyan091

BTW, this http://127.0.0.1:8080/standalone/?q=webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_scale:videoName=%22four-colors-h264-bt601.mp4%22;displayScale=%22same%22 fail in canary.

kainino0x · 2026-03-19T21:13:35Z

src/webgpu/web_platform/external_texture/video.spec.ts

-    format: 'RGBA',
-    codedWidth: kWidth,
-    codedHeight: kHeight,
+  const frame = new VideoFrame(sourceFrame, {


Is there a way to encode this data in a video file rather than going through VideoFrame? Doing it with VideoFrame seems kind of contrived - the way people would actually run into issues in the wild is through video files that have different dimensions encoded in them.

It doesn't seem easy to create a video file with display size scale. I see we can use setsar=2/1 to make video display size becomes 2 times.

ffmpeg.exe -loop 1 -i .\four-colors.png -c:v libx264 -pix_fmt yuv420p -frames 50 -colorspace smpte170m -color_primaries smpte170m -color_trc smpte170m -color_range tv -vf "setsar=2/1" four-colors-h264-bt601-sar2.mp4

The display height becomes two times of coded width, but the width does not, and the four colored squares are still displayed according to the coded size.

If I force DAR using setsar=2/1,setdar=4/3, the SAR will be 1:1.
ffmpeg.exe -loop 1 -i .\four-colors.png -c:v libx264 -pix_fmt yuv420p -frames 50 -colorspace smpte170m -color_primaries smpte170m -color_trc smpte170m -color_range tv -vf "setsar=2/1,setdar=4/3" four-colors-h264-bt601-sar2.mp4

Verify:
ffprobe.exe -v error -select_streams v:0 -show_entries stream=width,height,coded_width,coded_height,sample_aspect_ratio,display_aspect_ratio -of default=noprint_wrappers=1 .\four-colors-h264-bt601-sar2.mp4

width=320 height=240 coded_width=320 coded_height=240 sample_aspect_ratio=1:1 display_aspect_ratio=4:3

I will further investigate how to make it works.

If it works, I think we need to add 6 new video files at least (3 different codes/color spaces videos x 2 display scales (larger/smaller).

That should be fine as long as they're not too large. Thank you for investigating!

kainino0x · 2026-03-19T21:14:11Z

src/webgpu/web_platform/external_texture/video.spec.ts

+    // Build expected sampled colors by drawing the same source frame to a 2D canvas.
+    // This makes the check robust across codecs/container metadata while still validating
+    // that importExternalTexture sampling matches browser video rendering behavior.
+    const canvas = createCanvas(t, 'onscreen', kWidth, kHeight);


There are too many different widths and heights. Can you rename kWidth and kHeight to something more specific? (kCanvasWidth kCanvasHeight)

Ok, I will return displayWidth and displayHegith from createVideoFrameWithDisplayScale, and use them in the cases stead of kWidth and kHeight.

haoxli · 2026-03-20T07:57:41Z

BTW, this http://127.0.0.1:8080/standalone/?q=webgpu:web_platform,external_texture,video:importExternalTexture,video_frame_display_size_scale:videoName=%22four-colors-h264-bt601.mp4%22;displayScale=%22same%22 fail in canary.

I tested it pass on Intel GPU, but failed on NV GPU. It seems to be a tolerance issue or decoder differences on NVIDIA. I will check it again after we update the video file.

haoxli requested a review from kainino0x March 17, 2026 08:50

shaoboyan091 self-requested a review March 19, 2026 01:12

shaoboyan091 reviewed Mar 19, 2026

View reviewed changes

kainino0x reviewed Mar 19, 2026

View reviewed changes

Conversation

haoxli commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shaoboyan091 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haoxli commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

haoxli commented Mar 17, 2026 •

edited

Loading