TT-7360 feat: Careful Speech workflow, waveform segments, and playback engine#340
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Implements a new Careful Speech step in the renderer, including clause-based waveform segmentation, navigation/recording UI, segment persistence, and localization/tooling integration.
Changes:
- Added Careful Speech UI (player + controls) and supporting utilities/hooks for clause and clause-moves segment management.
- Extended WaveSurfer region coloring/selection logic to support a new Careful Speech segment color mode and more reliable region play-end behavior.
- Integrated the new tool end-to-end (tool slug, offline setup, workflow migration, localization strings/selectors).
Reviewed changes
Copilot reviewed 28 out of 30 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/renderer/src/utils/namedSegments.ts | Adds new named segment buckets for clause and clause-moves. |
| src/renderer/src/utils/carefulSpeechSegmentColors.ts | Introduces Careful Speech waveform region color utilities. |
| src/renderer/src/store/localization/reducers.tsx | Adds Careful Speech localized string bundle and tool label. |
| src/renderer/src/store/localization/model.tsx | Adds Careful Speech localization typings and tool string key. |
| src/renderer/src/store/localization/exported-strings-name.json | Updates exported localization bundle filename reference. |
| src/renderer/src/selector/selectors.tsx | Adds a selector for Careful Speech localization layout. |
| src/renderer/src/routes/PassageDetail.tsx | Routes mobile Careful Speech tool to the new screen component. |
| src/renderer/src/crud/useWavesurferRegions.tsx | Adds Careful Speech region-color mode + region-in/out targeting safeguards. |
| src/renderer/src/crud/useWaveSurfer.tsx | Wires new segment color mode + Careful Speech status ref into regions hook. |
| src/renderer/src/crud/useTools.ts | Registers Careful Speech in the tool slug list. |
| src/renderer/src/crud/useOfflineSetup.ts | Uses the new carefulSpeech tool slug for offline workflow creation. |
| src/renderer/src/crud/toolSlug.ts | Adds ToolSlug.CarefulSpeech. |
| src/renderer/src/components/WSAudioPlayer.tsx | Exposes additional controls (setPlay, auto-segment, load/save regions, Careful Speech color apply). |
| src/renderer/src/components/PassageDetail/transcribeDefaultSegParams.ts | Adds default auto-segmentation params used for clause-moves generation. |
| src/renderer/src/components/PassageDetail/PassageDetailPlayer.tsx | Plumbs Careful Speech status ref + segment playback end callback to WSAudioPlayer. |
| src/renderer/src/components/PassageDetail/PassageDetailGrids.tsx | Adds Careful Speech screen into the main PassageDetail layout logic. |
| src/renderer/src/components/PassageDetail/PassageDetailCarefulSpeech.tsx | New Careful Speech step screen: clause playback, boundary adjustment, recording flow, status coloring. |
| src/renderer/src/components/PassageDetail/carefulSpeech/useCarefulSpeechSegments.ts | New hook for bootstrapping/persisting clause and clause-moves segment buckets. |
| src/renderer/src/components/PassageDetail/carefulSpeech/carefulSpeechFormat.ts | Helper to format clause time ranges for display. |
| src/renderer/src/components/PassageDetail/carefulSpeech/CarefulSpeechControls.tsx | UI controls for boundary adjustment, recording, navigation, and completion. |
| src/renderer/src/components/PassageDetail/carefulSpeech/CarefulSpeechControls.cy.tsx | Cypress component test for CarefulSpeechControls basic rendering. |
| src/renderer/src/components/PassageDetail/carefulSpeech/carefulSpeechCompletion.ts | Logic for determining which clauses are completed and selecting the next incomplete clause. |
| src/renderer/src/components/PassageDetail/carefulSpeech/carefulSpeechCompletion.test.ts | Unit tests for completion and next-clause selection logic. |
| src/renderer/src/components/PassageDetail/carefulSpeech/carefulSpeechBoundary.ts | Logic for snapping clause boundary changes to clause-moves boundaries (or threshold fallback). |
| src/renderer/src/components/PassageDetail/carefulSpeech/carefulSpeechBoundary.test.ts | Unit tests for boundary adjustment and snapping behavior. |
| migration/bold-workflow/insert-bold-workflowsteps.sql | Updates Bold workflow Careful Speech step to use the new carefulSpeech tool/settings. |
| localization/TranscriberAdmin-en.xlf | Adds Careful Speech string units and tool label to localization source. |
| localization/TranscriberAdmin-en-1.2.xliff | Adds Careful Speech string units and tool label to localization source (xliff v1.2). |
gtryus
force-pushed
the
TT-7360-careful
branch
4 times, most recently
from
1499fc1 to
0c2fdb9
Compare
gtryus
force-pushed
the
TT-7360-careful
branch
3 times, most recently
from
9e1bb9e to
5c52da2
Compare
sarahentzel
requested changes
Jun 8, 2026
| BackTranslation = 'BT', | ||
| CarefulSpeech = 'CarefulSpeech', | ||
| Clause = 'clause', | ||
| ClauseMoves = 'clause-moves', |
Collaborator
There was a problem hiding this comment.
This isn't used anywhere that I could see.
| @@ -946,5 +958,6 @@ export function useWaveSurfer( | |||
| wsSetRegionColor, | |||
| wsRemoveCurrentRegion, | |||
| applyMarkVersesRegionColors, | |||
Collaborator
There was a problem hiding this comment.
I think it would be better to pass in a applyColors function and have the calling function return a color instead of having all this logic inside wavesurfer
| getCarefulSpeechPendingColor, | ||
| getCarefulSpeechRegionBaseColor, | ||
| type ICarefulSpeechColorStatus, | ||
| } from '../utils/carefulSpeechSegmentColors'; |
Collaborator
There was a problem hiding this comment.
pass in an applyColor function and put specialized login in the caller
gtryus
force-pushed
the
TT-7360-careful
branch
3 times, most recently
from
8100e88 to
fb8697a
Compare
gtryus
changed the title
gtryus
changed the title
Comment on lines
+2027
to
+2029
| const clearRecordingNode = allowRecord && | ||
| hasClearableAudio && | ||
| !dockRecordButton && ( |
Contributor
Author
There was a problem hiding this comment.
If we need it for the prompt step, we should add the control to the menu on that step. We don't want this Clear button there either.
…management
Introduce the implementation, workflow states, and user interface controls
for the BOLD Careful Speech step. This includes modular backend utilities for clause
boundary adjustments, auto-segmentation parameter configurations, and robust state
management using WaveSurfer integration.
- Core Feature & Utilities:
* Create `carefulSpeechBoundary` and `carefulSpeechCompletion` logic with full test coverage.
* Implement `useCarefulSpeechSegments` to orchestrate segment states, leveraging
`useProjectSegmentSave` for persistent local storage.
* Integrate the CarefulSpeech tool into offline configurations, tool slugs, and routing pipelines.
- Audio Player & WaveSurfer Integration:
* Enhance `WSAudioPlayer` and `useWavesurferRegions` to handle custom region
coloring, region-out listening documentation, and zoomed playback constraints.
* Introduce `isPlayRegionLocked` and a `beforePlay` callback to streamline audio execution states.
* Adjust silence threshold settings in `boldCarefulSpeechSegParams` for fine-tuned chunking.
- UI, UX, and Focus Management:
* Create `PassageDetailCarefulSpeech` and `CarefulSpeechControls` layout layers.
* Add a conditional "listen pass help" tooltip restricted to the first active segment.
* Implement auto-focus mechanics using React refs on the speaker input field when highlighted.
* Rework the "More/Fewer" auto-segment configuration actions.
- Internationalization & Metadata:
* Inject new localization strings for guided audio flows and helper prompts across relevant files.
…ctions Updated the CONTEXT.md file to provide detailed descriptions of the BOLD Careful Speech workflow, including the two-phase listen and recording passes, clause boundaries, and artifact types. Clarified the differences between BOLD Careful Speech and non-BOLD Phrase Back Translation processes. Improved definitions for key terms such as "Careful Speech artifact" and "Clause boundaries" to ensure accurate understanding and usage within the workflow. Also revised the ADR document to reflect these clarifications and emphasize the unique aspects of the BOLD Careful Speech step compared to other processes.
added 2 commits
June 15, 2026 15:24
- Added new utilities for splitting clauses at identified silence points, including `canSplitClause` and `splitClauseAt`. - Integrated clause splitting logic into the `PassageDetailCarefulSpeech` component, allowing users to split clauses during recording. - Enhanced `WSAudioPlayer` controls to support finding split points and updated UI components to include a "Split Clause" button. - Introduced localization strings for the new "Split Clause" feature and updated relevant tests to ensure functionality. - Added tests for the new clause splitting utilities to validate behavior under various conditions.
…dling - Introduced new state management for step completion in the PassageDetailCarefulSpeech component. - Implemented confirmation dialog behavior for marking steps complete and advancing to the next step. - Added functionality to clear recordings and update step completion status accordingly. - Updated tests to validate the new step completion logic and confirm dialog interactions. - Refactored related functions to improve clarity and maintainability.
- Deleted the `listenPassHelp` localization string from the XLIFF and XLF files as it is no longer needed. - Removed the corresponding entry from the localization model and initial state in the reducers. - Updated the exported strings name to reflect the new localization file structure.
added 2 commits
June 16, 2026 10:49
…ageDetailChooser - Eliminated the PassageDetailChooser import and its usage within the PassageDetailCarefulSpeech component. - Updated related tests to reflect the removal of the component, streamlining the codebase.
…ency - Adjusted padding properties for Stack and Typography components to enhance layout. - Standardized button padding for "Split Clause" and "Combine With Next Clause" actions to ensure uniformity in appearance.
gtryus
added a commit
that referenced
this pull request
Jun 16, 2026
…k engine (#340) * refactor: rename applyMarkVersesRegionColors to applyRegionColors for consistency * TT-7360 feat: implement Careful Speech workflow and waveform segment management Introduce the implementation, workflow states, and user interface controls for the BOLD Careful Speech step. This includes modular backend utilities for clause boundary adjustments, auto-segmentation parameter configurations, and robust state management using WaveSurfer integration. - Core Feature & Utilities: * Create `carefulSpeechBoundary` and `carefulSpeechCompletion` logic with full test coverage. * Implement `useCarefulSpeechSegments` to orchestrate segment states, leveraging `useProjectSegmentSave` for persistent local storage. * Integrate the CarefulSpeech tool into offline configurations, tool slugs, and routing pipelines. - Audio Player & WaveSurfer Integration: * Enhance `WSAudioPlayer` and `useWavesurferRegions` to handle custom region coloring, region-out listening documentation, and zoomed playback constraints. * Introduce `isPlayRegionLocked` and a `beforePlay` callback to streamline audio execution states. * Adjust silence threshold settings in `boldCarefulSpeechSegParams` for fine-tuned chunking. - UI, UX, and Focus Management: * Create `PassageDetailCarefulSpeech` and `CarefulSpeechControls` layout layers. * Add a conditional "listen pass help" tooltip restricted to the first active segment. * Implement auto-focus mechanics using React refs on the speaker input field when highlighted. * Rework the "More/Fewer" auto-segment configuration actions. - Internationalization & Metadata: * Inject new localization strings for guided audio flows and helper prompts across relevant files. * Enhance BOLD Careful Speech documentation and clarify workflow distinctions Updated the CONTEXT.md file to provide detailed descriptions of the BOLD Careful Speech workflow, including the two-phase listen and recording passes, clause boundaries, and artifact types. Clarified the differences between BOLD Careful Speech and non-BOLD Phrase Back Translation processes. Improved definitions for key terms such as "Careful Speech artifact" and "Clause boundaries" to ensure accurate understanding and usage within the workflow. Also revised the ADR document to reflect these clarifications and emphasize the unique aspects of the BOLD Careful Speech step compared to other processes. * Implement clause splitting functionality in Careful Speech workflow - Added new utilities for splitting clauses at identified silence points, including `canSplitClause` and `splitClauseAt`. - Integrated clause splitting logic into the `PassageDetailCarefulSpeech` component, allowing users to split clauses during recording. - Enhanced `WSAudioPlayer` controls to support finding split points and updated UI components to include a "Split Clause" button. - Introduced localization strings for the new "Split Clause" feature and updated relevant tests to ensure functionality. - Added tests for the new clause splitting utilities to validate behavior under various conditions. * Enhance PassageDetailCarefulSpeech component with step completion handling - Introduced new state management for step completion in the PassageDetailCarefulSpeech component. - Implemented confirmation dialog behavior for marking steps complete and advancing to the next step. - Added functionality to clear recordings and update step completion status accordingly. - Updated tests to validate the new step completion logic and confirm dialog interactions. - Refactored related functions to improve clarity and maintainability. * Remove unused localization strings and update references - Deleted the `listenPassHelp` localization string from the XLIFF and XLF files as it is no longer needed. - Removed the corresponding entry from the localization model and initial state in the reducers. - Updated the exported strings name to reflect the new localization file structure. * Refactor PassageDetailCarefulSpeech component by removing unused PassageDetailChooser - Eliminated the PassageDetailChooser import and its usage within the PassageDetailCarefulSpeech component. - Updated related tests to reflect the removal of the component, streamlining the codebase. * Update CarefulSpeechControls component styles for improved UI consistency - Adjusted padding properties for Stack and Typography components to enhance layout. - Standardized button padding for "Split Clause" and "Combine With Next Clause" actions to ensure uniformity in appearance. --------- Co-authored-by: Greg Trihus <gtryus@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce the end-to-end implementation, state workflow mechanics, and user interface
controls for the BOLD Careful Speech step, alongside downstream stabilization, refactoring,
and comprehensive unit testing.
Core Features & Workflow Engine:
carefulSpeechBoundaryandcarefulSpeechCompletionlogic with full test coverage.useCarefulSpeechSegmentsto orchestrate segment states, leveraginguseProjectSegmentSavefor persistent local storage.CarefulSpeechControlslayout.Audio Player & WaveSurfer Stabilization:
applyRegionColorsroutine acrossboth
PassageDetailCarefulSpeechandPassageDetailMarkVerses.WSAudioPlayerwith automated logging, visibility toggles for the clear-recordingaction button, and optimized region-out listening handlers.
immediately following auto-play cycles.
Automation & Auto-Play Enhancements:
selections, and "Next Segment" navigation button clicks.
forcefully preserve the active recording mode context.
UI/UX & Component Optimization:
CarefulSpeechControls.auto-focus pools, and the removal of development-phase console logs.
L10n, Typings, & Quality Assurance:
ICarefulSpeechStringsinterfacesand unified string providers across all subcomponents.
statusRef).CarefulSpeechControlsand the interactivebehavior/disabled states of the "Next Clause" navigation boundaries.