Webui/prompt processing progress

[ServeurpersoCom]

Close #17079

Integrates existing backend 'return_progress' feature into WebUI to show real-time token processing statistics during both prompt preprocessing and generation phases.

Key Features

• Unified statistics UI: Same Reading/Generation tab switcher used during streaming and after completion
• Live ETA countdown: Real-time countdown updates every second during prompt processing
• Auto tab switching: Automatically switches from Reading to Generation tab when prompt processing completes
• Manual tab navigation: Users can switch between tabs at any time during generation
• Preserved stats: Reading stats are preserved and viewable even after generation starts

Implementation

• useProcessingState hook: Extended with getLiveProcessingStats() and getLiveGenerationStats() methods, plus ETA countdown logic
• ChatMessageStatistics component: Enhanced with isLive and isProcessingPrompt props for streaming mode
• ChatMessageAssistant component: Uses unified statistics component during loading phase

Demo

demo.mp4

[ngxson]

Just a nits improvement, I think showing percentage + ETA instead of elapsed time can be more useful:

Processing (123 / 456 tokens - 27% - ETA: 50s)

[ServeurpersoCom]

It can still be improved; I don't know if people have prompts that take several minutes, but adding the minutes might be a good idea! (and also we calculate the tokens/s we can display them, but it will bloat, and we already have the final value)
I also need to double-check on CPU to break down the display and see if I can't have NaN or similar. Even with the first chunk #18305

[ServeurpersoCom]

I think we're good. Now the client side message "Processing..." is no longer visible.

During first batch :

Next one :

[ngxson]

Very nice feature!

(May need approval from @allozaur too)

[ServeurpersoCom]

I made a small observation: the chunk format is very close to that expected during normal inference, which spoofs the stat bubbles displayed during inference (those with the Settings/"Keep stats visible after generation" option). It might be wise to filter at this stage if "delta.content = null".

[allozaur]

Overall good stuff, but some changes are required in order to make it ready for merging. I will handle this on my end.

[ServeurpersoCom]

I'm doing a quick re-testing and we can merge it

[allozaur]

I'm doing a quick re-testing and we can merge it

wait! haven't finished yet, im polishing it up

[ServeurpersoCom]

I'm doing a quick re-testing and we can merge it

wait! haven't finished yet, im polishing it up

No worries, I'll test it when you're finished, and I never merge myself :)

[allozaur]

Alright, last changes pushed and also updated PR description with new demo video.

[allozaur]

Alright now I think that this is ready for merging

[allozaur]

@ggerganov please take a look at this and let me know if you also think that it's production ready :)

[ServeurpersoCom]

https://github.com/user-attachments/assets/6bec9d79-5b63-4d37-9a88-11be4aa0deae
On my side, there is a weird double update like this:
50s 51s 50s 49s 50s 49s...

[allozaur]

https://github.com/user-attachments/assets/6bec9d79-5b63-4d37-9a88-11be4aa0deae On my side, there is a weird double update like this: 50s 51s 50s 49s 50s 49s...

I see... maybe in this case i will remove this client-side countdown and will leave just the default ETA value

[ngxson]

Mathematically say, I designed the progress object such that the ETA can be inferred without keeping track of a timer on client side.

The progress can be non-linear. For example, if you're doing something else (watch a video, open a new tab, etc) while prompt is processing, it can slow down the progress enough to be noticeable.

[ServeurpersoCom]

Yes, the timer creates a race with server updates, causing the jump. My original approach skipped the first chunk and recalculated from total elapsed time, avoiding both the initial error and the need for client-side countdown. I'll let allozaur finish the refactoring, the update of the existing stats fields is superb! now we also have the pre-processing tokens/s!

[allozaur]

well, I've applied 4807b0f to still keep the client-side counter as one more attempt to have this smoother waiting experience, @ngxson @ServeurpersoCom please check it and test on your ends.

In the end if it's working good for you, let's keep it, but if you think that this is better to just use the chunk-based calculations, then I will remove this browser time counter logic.

[ServeurpersoCom]

No more "ETA data race" on my side, the linearity adjusts from the beginning, and the last second is spot on -> we can merge

If we want to be meticulous, it's a good idea to retest the very first batch slowly on the processor (testing in progress)
I check if "percent = Math.round(0 / 0) = NaN" exist

Still not perfect :
https://github.com/user-attachments/assets/15a1d51e-83c9-498f-ab36-12bce67e6da8

[ServeurpersoCom]

chunk-based calculation

Yes we need this !

[ggerganov]

The ETA is quite inaccurate:

Screen.Recording.2025-12-29.at.18.18.35.mov

I would suggest to remove the ETA since it is not very useful.

[allozaur]

The ETA is quite inaccurate:
Screen.Recording.2025-12-29.at.18.18.35.mov

I would suggest to remove the ETA since it is not very useful.

can do! :D so eventually we are simplifying this and probably that's the best outcome

[ServeurpersoCom]

Now it's perfect, no more strange jitter, and 1 refresh for each batch

[ServeurpersoCom]

Merge needed to continue on 18226, which affects many files

[ngxson]

The formula seems to be off (that why the ETA was incorrect). I'll push a fix.

[ServeurpersoCom]

I could do another small front PR to test it and put it back. % and tokens/s are relatively stable during preprocessing, so the ETA should converge to accurate values after a few chunks once the average stabilizes

Done with front-only ngxson commit. batch refresh rate + linear ETA + better UI from allozaur

[ServeurpersoCom]

530831841-7c575e2f-fd79-4d3b-b0da-092ac798d769.mp4

Now it works with what everyone brought! Perfect!