Implementation Spec — TASK #30: Dub → Stories cast+lines export ("Create Story from Dub")¶
TL;DR¶
Add a "Create Story from Dub" action to the Dub editor header that takes the current dubSegments + speakerClones, groups segments by speaker_id into a Stories cast (one CastMember per speaker, voice = that speaker's assigned profile_id), maps each segment {start, end, text, speaker_id, profile_id} → a Stories StoryTrack {character, text, profileId}, loads them into the Stories Editor, and navigates there (setMode('stories')). Pure transform lives in a new util frontend/src/utils/dubToStory.js; one new store action loadStoryFromDub in frontend/src/store/storiesSlice.ts; one button in the active-editor header of frontend/src/pages/DubTab.jsx. No backend changes.
No API / SSE / DB surface (read this before hunting for an endpoint). This feature is 100% client-side: there is no new HTTP request/response, no SSE event type, no WebSocket frame, no backend route, no Python code, no DB table, no alembic migration, and no Zustand-persist
versionbump. The only "data store" touched is the existinglocalStorageblobomnivoice.app(persistversion: 4), and only through shapes it already persists (cast,storyTracks,currentProjectId). The complete data contract is pinned in §"API / data shapes" below — a developer can implement every function from the signatures there without reading any other file. The only network-adjacent symbols in the touched code (API,/longform/render,/dub/audio/:jobId) are pre-existing and unchanged — this transform never calls them; downstream Stories render uses them later, untouched by this PR.Grounding note (verified against code, 2026-06-13): Every file/symbol/line reference below was checked against the working tree on branch
feat/stories-shared-render. Corrections from the prior draft: (a)loadProject/newProjectare atstoriesSlice.ts:101-106(loadProjectis 101-105,newProjectis 106); (b)setModelives instore/uiSlice.ts:62/:94, notstoriesSlice.ts; (c) the editor-headerdub-head__actionsblock to hook is the second one atDubTab.jsx:715-750(gated bydubJobId && dubStep ∈ {editing,generating,done}at line 704) — the firstdub-head__actionsat line 357 is the idle/pre-transcription skeleton with disabled buttons, do not hook there; (d) StoriesEditor'smakeTrackis atStoriesEditor.jsx:91-92,_trackIdreseed effect at136-140,slug()at173, and cast ids in the live UI are generated bygenCastId()(randomc_<rnd>, line 95-98) whileslug()is used only byautoCast; (e) there are 21 locale files underi18n/locales/; (f)castis persisted (store/index.ts:111); (g) DubTab getsprofilesatApp.jsx:1124andspeakerClonesatApp.jsx:1135. Additional verifications this round: (h) the per-speaker<select>readsdubSegments.find(s => s.speaker_id === spk)?.profile_idand writes the value to every segment of that speaker (DubTab.jsx:842,845), so within a clean job all segments of a speaker share oneprofile_id— the "read first-seen" rule below is purely defensive against externally-imported/SRT segments; (i) profile objects expose at least{ id, name }(the Stories voice<select>lists them byp.id/p.nameatStoriesEditor.jsx:557,715); (j) the Dub "Restore original" action atDubTab.jsx:1048-1049overwritesseg.textwithseg.text_original, so readingseg.textalways reflects what the user currently sees — there is no separate live "show original" view toggle; (k)nextCastColor(storyCast.js:15-18) picks the first unused palette color and, only when all 8 are used, wraps viacast.length % 8(deterministic, not an error); (l)upsertCastMember(storiesSlice.ts:73-80) dedupes cast byid— butloadStoryFromDubdoes a wholesaleset, so cast-id uniqueness must be guaranteed inside the transform, not relied on at store level. Constraint-relevant verifications (round 3): (m) theslug()rule mirrored bycastIdForisname.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '') || 'char'(StoriesEditor.jsx:173) — a single bounded negated char class[^a-z0-9]+with no nesting/overlapping quantifiers (linear-time, CodeQL-js/polynomial-redos-safe); theauto:/preset:discrimination usesString.prototype.startsWith, not regex; (n) CodeQL runs on bothpythonandjavascript-typescriptwith thesecurity-and-qualitysuite (.github/workflows/security.yml:74,95-96), which includesjs/polynomial-redos— so the new.jstransform's regex is in CodeQL's gating scope, not exempt; (o) the persistmigrateatstore/index.ts:120-130is pass-through forversion < 4and identity for4, and the persistedcast/storyTracksshapes are exactly what this transform emits — so no migrate edit and noversionbump (version: 4atindex.ts:115); (p) the CJK gate (tests/test_no_hardcoded_cjk.py) scans git-tracked source, auto-allows*.test.*files (_is_allowed, line 89 — confirmed:if ".test." in base or base.startswith("test_") or base.endswith("_test.py"): return True), and the transform takes a pre-resolvedunknownSpeakerLabelstring — so neither the transform nor its test introduces hardcoded CJK; (q) docs-drift (scripts/check-docs-drift.py, gated daily + the PR-gatingscripts/validate-install-docs.pyatci.yml:75) keys off the coarsefeatures[]list indocs/features.yaml(Video Dubbing, etc. — no "Stories" entry and no sub-feature entries) and the TTS/ASR engine registries; a Dub→Stories sub-feature button touches none of those, so the docs-drift checks are a structural no-op for this PR (any README mention is optional polish, not a gate). Data-shape verifications (round 4): (r)DubSegmentis literallyRecord<string, unknown> & { id: string; text: string }(dubSlice.ts:46) — onlyidandtextare statically typed; every other field the transform reads (start,end,speaker_id,profile_id,text_original) isunknown-typed and must be coerced defensively (this is why the transform's defensive coercion is mandatory, not optional polish); (s)speakerClonesis exactlyRecord<string, { ref_audio: string; ref_text: string; duration: number; source_count: number }>(dubSlice.ts:120-125); (t)setDubSegmentsis(v: DubSegment[] | ((prev: DubSegment[]) => DubSegment[])) => void(dubSlice.ts:148,resolveat:50-52) — the transform consumesdubSegments, neversetDubSegments; (u) the persistedstoryTracksshape inpartializeis destructured to exactly{ id, character, text, profileId, emotion, speed }(store/index.ts:109-110), which is bit-identical to the transform'sStoryTrackoutput, andcast/storyProjects/currentProjectIdare persisted verbatim (index.ts:111-113) — confirming zero persisted-shape drift. Test-infra verifications (round 5): (v) the slice harness pattern is already in-repo atstoriesSlice.test.ts:4-10—createStoriesSlice(set, get, {})is constructed with a hand-rolledset/getover a plain object, importing nomain, no torch, no GPU, no store provider (this is the canonical pure/handler-direct strategy to copy); (w) pure-util tests use the flatimport { describe, it, expect } from 'vitest'form with no setup file (storyCast.test.js,importStory.test.js,parseScript.test.js) —dubToStory.test.jsfollows this exactly; (x)nextCastColor([])returnsCAST_COLORS[0]andnextCastColor([{color: CAST_COLORS[0]}])returnsCAST_COLORS[1](asserted instoryCast.test.js), confirming the golden-fixture colors#fabd2f/#d3869b/#83a598; (y)CAST_COLORS = ['#fabd2f','#d3869b','#83a598','#b8bb26','#fe8019','#8ec07c','#fb4934','#458588'](storyCast.js:9-12) — exactly 8 entries, so the 9th speaker reuses index8 % 8 === 0→#fabd2f; (z) there is no automated i18n key-parity test gating CI —scripts/translate_all.pyis a manual GoogleTranslator propagation script (get_flat_keys/set_nested_value), not a CI gate; key parity is enforced by running that script + a manual diff, and a missing key surfaces at runtime as the raw key string (react-i18next defaultreturnNull:falsefallback), which is the localization regression to avoid; (aa) component tests that render JSX (e.g.ExportModal.test.jsx) pull in the real store +import '../i18n'via@testing-library/react— this is heavier than the pure path and is why the handler/confirm path is deliberately not unit-tested here (see Test plan rationale).
Problem¶
The Dub pipeline produces a fully attributed, multi-speaker transcript (dubSegments, each with speaker_id, text, start, end, and an assigned profile_id), but that work is locked inside the dubbing workflow. Users who want to re-voice the same conversation as a multi-track audiobook/story have to re-type every line and re-assign every voice in the Stories Editor by hand. The Stories Editor already models exactly this shape (cast = characters→voices, tracks = ordered lines — see StoryTrack/CastMember at storiesSlice.ts:12-26), and already renders through the shared /longform/render pipeline. The bridge between the two is missing.
Goal / Non-goals¶
Goals
- One-click "Create Story from Dub" from the Dub editor when segments exist.
- Group
dubSegmentsbyspeaker_id→ distinct Stories cast members, deterministic order (first-appearance). - Map each segment to a
StoryTrackpreserving order (bystart), text, and resolved voice. - Resolve each speaker's voice from the per-speaker
profile_idassigned in the Dub Cast UI (DubTab.jsx:831-867); carry auto-clone selections (auto:<speaker>) into a realprofileIdwhen a saved clone profile exists, else fall back to cast default. - Navigate to the Stories tab with the new cast + tracks loaded; do not silently clobber unsaved Stories work (confirm-or-merge — see Design).
- Pure, unit-tested transform; i18n for all user-facing strings.
- Total robustness against malformed/partial segment data — never throw, never produce a Story that crashes the StoriesEditor renderer (see Edge cases & failure paths). This robustness is load-bearing, not defensive polish:
DubSegmenttypes every field exceptid/textasunknown(dubSlice.ts:46), so the transform receives genuinely untyped data and the coercion rules in §"API / data shapes" are the contract.
Non-goals
- No translation choice surfaced here — we export the target/edited
text(i.e. what the user sees in the table). (A future toggle for original vs translated is noted under Risk.) - No timing/
start/endcarry-over into Stories (Stories has no timeline;startis used only for ordering).pause_mssynthesis from gaps is out of scope. - No backend endpoint, no HTTP/SSE shape, no schema change, no persistence migration beyond what
storiesSlicealready does. - No reverse direction (Story → Dub) — that is TASK #24/#31.
Design¶
Data flow¶
dubSegments (store) ──┐
profiles (prop) ──────┼──► dubToStory(segments, { profiles, unknownSpeakerLabel })
┘ │
├─► { cast: CastMember[], tracks: StoryTrack[], stats }
▼
storiesSlice.loadStoryFromDub(cast, tracks)
▼
setMode('stories') ──► StoriesEditor renders
speakerClones is intentionally dropped from the v1 transform signature — see E24 / §"API / data shapes". The call site still has it available as a prop should a future version re-add it.)
1. Pure transform — frontend/src/utils/dubToStory.js (NEW)¶
Exact exported signature (see §"API / data shapes" for the full JSDoc/TS typedef block to copy into the file):
export function dubToStory(segments, { profiles = [], unknownSpeakerLabel = 'Speaker' } = {})
// → { cast: CastMember[], tracks: StoryTrack[], stats: { lines: number, speakers: number, skippedEmpty: number } }
- Input guard. If
segmentsis not an array (Array.isArray(segments) === falsefor null/undefined/object/string/number), treat it as[]and return the empty result ({ cast: [], tracks: [], stats: { lines: 0, speakers: 0, skippedEmpty: 0 } }). Likewise coerceprofilesto[]if a caller passes a non-array (Array.isArray(profiles) ? profiles : []). The function must never throw on bad input. - Filter to renderable segments. For each segment compute
txt = (typeof seg?.text === 'string' || typeof seg?.text === 'number') ? String(seg.text).trim() : ''. Keep a segment only whentxt !== ''. Count every dropped segment (empty, whitespace-only, missingtext, non-string/non-numbertext, orsegitself non-object) asskippedEmpty. (Note: numerictextlike0survives becauseString(0).trim() === "0";null/undefined/objects/arrays →''→ dropped.) - Sort defensively. Sort a shallow copy by numeric
startascending, stable. Normalize viaconst n = Number(seg?.start); const start = Number.isFinite(n) ? n : 0soundefined/null/NaN/non-numeric-stringstartall collapse to0and keep their relative input order (stable sort preserves first-appearance as the tiebreak). The Dub table is already start-ordered, but SRT-imported or externally-merged segments may not be. Negativestartis valid and sorts before0— acceptable, ordering is purely relative. - Collect distinct speakers in first-appearance order. Walk the sorted, filtered segments; derive a normalized speaker key per segment:
rawSpk = seg?.speaker_id; coerce to string:spk = (rawSpk == null) ? '' : String(rawSpk).- If
spk.trim() === ''(falsy, whitespace-only, or a non-string that stringifies empty) → map to the single synthetic speaker with key' __unknown__'(an internal sentinel containing a NUL byte that can never collide with a realspeaker_id), display nameunknownSpeakerLabel. All unattributed segments share this one synthetic speaker (do not create one synthetic speaker per unattributed segment). - Otherwise the speaker key is the trimmed
spk. Collect keys in first-appearance order into an ordered list (Array<string>). - Build cast. For each distinct speaker key, in first-appearance order, create:
where
{ id: castIdFor(displayName, assignedIds), name: displayName, color: nextCastColor(accum), profileId: resolveVoice(firstSegmentForSpeaker, displayName, profiles) }displayNameis the trimmed realspeaker_id, orunknownSpeakerLabelfor the synthetic speaker,accumis the cast array built so far (sonextCastColorsees prior picks), andassignedIdsis aSet<string>of ids already minted in this cast pass. castIdFor(name, assignedIds)— exact signature/return in §"API / data shapes". Slugify with the same rule asStoriesEditor.slug()(StoriesEditor.jsx:173) —name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '') || 'char'. This regex is intentionally identical to the liveslug()and is linear-time ([^a-z0-9]+is one bounded negated char class with no nested/overlapping quantifiers — no(a+)+, no\s*…\s*overlap), so it stays clear of CodeQLjs/polynomial-redoseven though it runs on user/model-derivedspeaker_idinput (see Constraints). Special case (mirrorautoCastatStoriesEditor.jsx:184): ifname.toLowerCase() === 'narrator', the base id is exactlynarrator(so it aligns with the always-present default narrator anddeleteCharacter'sid === 'narrator'guard atStoriesEditor.jsx:166). Then guarantee uniqueness within this cast: withbaseas the slug, ifassignedIds.has(base)append-2,-3, … until free; add the chosen id toassignedIds; return it. Collisions are real and must be handled:- two different
speaker_ids that slug identically ("Speaker 1"vs"Speaker 1","a/b"vs"a-b"); - a
speaker_idthat slugs to empty (e.g. all-punctuation"???") → base'char', then'char-2'for a second such speaker; - a real speaker literally named "Narrator" alongside the synthetic-unknown speaker whose
unknownSpeakerLabellocalizes to "Narrator" — the second one takesnarrator-2(thenarratorspecial-case only claims the id for the first narrator-slug speaker).
- two different
- Maintain a
speakerKey → castIdmap (ObjectorMap<string, string>) for the track pass. nextCastColor(accum)fromfrontend/src/utils/storyCast.js:15-18— exact semantics: returnsCAST_COLORS.find(c => !usedSet.has(c))(first palette color not already inaccum), and only when all 8 are used falls back toCAST_COLORS[(accum.length || 0) % 8]. So for ≤8 speakers each gets a distinct color; the 9th+ deterministically reuse by index. (Not "wrap by count" for the first 8 — it's "first unused".) The palette is['#fabd2f','#d3869b','#83a598','#b8bb26','#fe8019','#8ec07c','#fb4934','#458588'](storyCast.js:9-12).resolveVoice(seg, displayName, profiles)— see step 6 — is called on the first segment of each speaker (first by the sorted order). Within a clean dub job every segment of a speaker shares oneprofile_id(the Cast UI writes it to all of them,DubTab.jsx:845), but reading first-seen is defensive against SRT-imported segments where they could differ; the first segment wins and the divergence is silently ignored (documented under Edge cases).resolveVoice(seg, displayName, profiles)— exact signature/return in §"API / data shapes". Resolve a segment'sprofile_idto a StoriesprofileId | null:pidRaw = seg?.profile_id; coerce:const pid = (pidRaw == null) ? '' : String(pidRaw).pid === ''(empty / missing / falsy after coercion) →null(Stories "Default").pid.startsWith('auto:')(viastartsWith, not regex) → it's an ephemeral auto-clone selection. Attempt to bind it to a saved profile so it survives the job:- The match key is the speaker's
displayName(the originalspeaker_id), not theauto:payload. (The payload is the underscored slugauto:${spk.toLowerCase().replace(/\s+/g,'_')}fromDubTab.jsx:836— lossy, so don't reverse it.) - Find the first
profiles[i]whereString(profiles[i]?.name ?? '').trim().toLowerCase() === displayName.trim().toLowerCase(). If found →profiles[i].id. Multiple profiles with the same name: first match wins (deterministic byprofilesarray order). If none →null(degrade to cast Default — never dangle). - Edge:
unknownSpeakerLabelspeaker carrying anauto:value matches a profile literally named "Speaker" only if one exists; otherwisenull.
- The match key is the speaker's
pid.startsWith('preset:')(viastartsWith) →nullfor v1. Design presets (DubTab.jsx:856-858) are notprofileIds the Stories voice dropdown lists; mapping tonullavoids a permanently-blank dropdown. Noted in Risk.- any other non-empty string → treat as a real saved
profileId, pass through verbatim (return pid). (From thedub.clone_profilesoptgroup,value={p.id},DubTab.jsx:851-855.) Do not validate againstprofiles— passing an unknown id through is safe (the Stories voice<select>renders blank/Default for an unknown id;effectiveProfilereturns it and/longform/renderfalls back to the default voice). Tolerates a profile deleted between dub-assignment and story-creation. - Regex-safety note: all four discriminations (
'',auto:,preset:, pass-through) use equality +startsWith, no regex on the user-controlledprofile_id. The only regex in the whole transform is the bounded linearslug()in step 5. - Build tracks. For each filtered+sorted segment, in order:
txtis the same trimmed string computed in step 2 (String(seg.text).trim()), already known non-empty.localIdis a1..Nsequence local to this call (let localId = 0;start, pre-increment → first id is1).profileIdisnull(per-line override unset) — voice inherited from the cast member, matchingeffectiveProfile(track, cast)(storyCast.js:21-25: line override → cast voice → default).emotion/speedarenull.characteris the cast id resolved from the segment's speaker key viacastIdMap— guaranteed present because the cast pass walked the same filtered+sorted segments first, so every track'scharacterreferences an existing cast member (no orphan tracks). An orphancharacterwould make StoriesEditor render the line under a missing cast member and fall to default voice with no way to fix it.- Return
{ cast, tracks, stats: { lines: tracks.length, speakers: cast.length, skippedEmpty } }.
Keep the util free of React/i18n: the caller passes a resolved unknownSpeakerLabel string. (This is also what keeps the transform out of the hardcoded-CJK gate — no t() call, no literal CJK; the localized label is injected by the caller — see Constraints.) The StoriesEditor mount effect re-seeds its module-level _trackId from max(track.id) on load (StoriesEditor.jsx:136-140), so lines added after load won't collide with the 1..N ids minted here.
2. Store action — frontend/src/store/storiesSlice.ts (loadStoryFromDub)¶
Add to the StoriesSlice interface (storiesSlice.ts:36-51, alongside loadProject/newProject):
createStoriesSlice (storiesSlice.ts:66-114, next to loadProject at 101-105 and newProject at 106):
loadStoryFromDub: (cast, tracks) =>
set({
storyTracks: (tracks || []).map((t) => ({ ...t })),
cast: (cast || []).map((c) => ({ ...c })),
currentProjectId: null,
}),
- Mirrors
newProject()semantics (fresh, unsaved project) + load —currentProjectId: nullso a subsequentsaveProject(name)(storiesSlice.ts:84-100) creates a new named project rather than overwriting (it only updates in place whencurrentProjectIdis set;id = s.currentProjectId || genProjectId()at line 86). - Defensive
|| []so the action never throws if called withundefined. Defensive shallow copy on both arrays (consistent withloadProjectat line 104) so the caller's arrays can't mutate store state afterward. Note the copy is one level deep ({ ...t }/{ ...c }) — matchingloadProject; track/cast member fields are all primitives ornull, so a shallow copy fully isolates them. - Backward-compat (no migration). The action sets exactly the persisted shapes
storyTracks({id,character,text,profileId,emotion,speed}, destructured identically inpartializeatstore/index.ts:109-110) andcast(index.ts:111). It introduces no new persisted key and no shape change, so the persistversionstays4and the existing pass-throughmigrate(index.ts:120-130) is untouched. Oldomnivoice_data/omnivoice.applocalStorage rehydrates exactly as before. - Empty-cast edge: if
castis[](only if a caller ignores the empty-tracks guard), the store ends withcast: [],storyTracks: []. StoriesEditor's narrator-default (DEFAULT_CAST) is not re-injected by this action —loadStoryFromDub([], [])leaves an empty cast, unlikenewProject()which seedsDEFAULT_CAST. The UI guard (step 3) prevents this in practice. (Decision: do not seedDEFAULT_CASThere — a dub-derived story always has ≥1 real cast member by the time it's called.) - Re-entrancy note: the StoriesEditor
_trackIdreseed effect runs only on mount (empty dep array, line 139). If a Story is already open when the user runs Create-Story-from-Dub a second time, the effect does not re-run, so_trackIdretains its prior (higher) value — new lines added after the second load still get ids above the loaded1..N, so no collision. The module counter being larger thanmax(loaded id)is harmless (ids only need uniqueness, not contiguity).
3. UI entry point — frontend/src/pages/DubTab.jsx¶
Add a "Create Story from Dub" button in the active-editor header actions — the dub-head__actions block at DubTab.jsx:715-750 (inside the post-transcription dub-col rendered when dubJobId && (dubStep === 'editing' || dubStep === 'generating' || dubStep === 'done'), line 704). Do not use the idle skeleton header at line 357 (its buttons are disabled).
Visibility & disabled states (enumerated):
- Hidden entirely when
!dubSegments.length. Because the wholedub-colonly renders fordubStep ∈ {editing,generating,done}, the button can never appear before transcription. - Visible + enabled when
dubSegments.length > 0anddubStep ∈ {editing, done}. - Visible + disabled when
dubStep === 'generating'ordubStep === 'stopping'— generation is mutating segments/progress; block the hand-off until it settles. (Mirror the existing primary-row gating atDubTab.jsx:724-728.) - The transform reads
dubSegmentsas they currently are; if the user edited text in the table, the latest edits are exported (store is the single source of truth).
Handler (new onCreateStory, defined with useCallback like the existing onGenerateClick at DubTab.jsx:197):
import { dubToStory } from '../utils/dubToStory';
// store reads (add near the existing reads at DubTab.jsx:101-110):
const setMode = useAppStore(s => s.setMode); // from uiSlice (uiSlice.ts:94)
const loadStoryFromDub = useAppStore(s => s.loadStoryFromDub); // from storiesSlice (new)
const storyTracks = useAppStore(s => s.storyTracks); // for the clobber guard
const onCreateStory = useCallback(() => {
if (!dubSegments.length) return; // guard A: nothing to export
const { cast, tracks, stats } = dubToStory(dubSegments, {
profiles,
unknownSpeakerLabel: t('stories.fromDub.unknownSpeaker'),
});
if (!tracks.length) { toast.error(t('dub.create_story_empty')); return; } // guard B: every line empty/whitespace
// guard C — clobber: an existing unsaved/loaded story would be replaced.
if (storyTracks.length && !window.confirm(t('dub.create_story_confirm'))) return;
loadStoryFromDub(cast, tracks);
setMode('stories');
toast.success(t('dub.create_story_done', { lines: stats.lines, speakers: stats.speakers }));
}, [dubSegments, profiles, storyTracks, loadStoryFromDub, setMode, t]);
- Guard A —
dubSegments.length === 0: return silently (button is hidden here, but the guard protects against a stale-closure click during a tab transition). - Guard B — transform returned
tracks.length === 0(all segments empty/whitespace/malformed): showdub.create_story_emptytoast, do not navigate, do not callloadStoryFromDub(preserves an existing Story). Distinct from guard A: there were segments, but none produced a line. - Guard C — clobber: only prompt when
storyTracks.length > 0. Cancel → return without changing anything (no navigation, no store write, no toast). Confirm → proceed. WhenstoryTracks.length === 0(fresh Stories, ornewProjectstate), skip the confirm and proceed directly. A loaded saved project being edited still hasstoryTracks.length > 0, so the confirm fires — replacing it does not delete the saved project fromstoryProjects(it only swaps the live working tracks/cast and setscurrentProjectId: null); the user can reopen the saved one. The confirm copy makes clear it replaces the current working project, not delete saved ones. - Success path:
loadStoryFromDub→setMode('stories')→ success toast with{lines, speakers}.skippedEmptyis not surfaced in v1; it's instatsfor a future "N empty lines skipped" note. - Double-click / rapid re-click: idempotent per-click; a second click after navigation re-runs the same transform on the unchanged
dubSegmentsand prompts the clobber confirm (sincestoryTracksis now populated). Acceptable; no debounce.
UI wiring details:
- Reuse the
Buttoncomponent already imported (import { Button, Segmented, Badge, Progress } from '../ui';—DubTab.jsx:25) or the localFooterBtnfamily (DubTab.jsx:1524). Follow the existing icon-only-with-tooltip pattern of Save/Reset atDubTab.jsx:718-721; place it among the secondary actions (next to Save/Reset) so it reads as a "send elsewhere" action. - Icon: the lucide import block at
DubTab.jsx:4-9already includesUsers(good semantic for a cast/story hand-off) andFileText.BookOpenandDramaare not imported — if preferred, add to that block; otherwise reuseUsers(zero new import). - No new props from
App.jsx:setModefrom the store (DubTab already doesuseAppStore.getState().setMode?.('enterprise')atDubTab.jsx:1328),loadStoryFromDubis the new store action,profiles(App.jsx:1124) is already destructured (DubTab.jsx:83). (speakerClonesatApp.jsx:1135/DubTab.jsx:89with default{}remains available but is not passed to the transform in v1.) t/i18nfromconst { t, i18n } = useTranslation();(DubTab.jsx:75);toastfromimport toast from 'react-hot-toast';(DubTab.jsx:23). All four user-facing strings (dub.create_storytooltip/label,dub.create_story_confirm,dub.create_story_done,dub.create_story_empty) and the injectedstories.fromDub.unknownSpeakergo throught()— no string literal in JSX (see Constraints, Localization).
window.confirm is acceptable for the clobber guard (synchronous, webview-native, identical across all three Tauri platforms — see Constraints). If the project later prefers a non-blocking pattern, swap for an inline confirm toast. Confirm dialogs are awkward in the Tauri webview test harness, so the unit tests target dubToStory + loadStoryFromDub, not the confirm path.
API / data shapes¶
HTTP / SSE / DB surface: none. No request body, no response body, no SSE event type, no route, no Python handler, no DB table, no alembic revision, no persist
versionchange. The only durable store islocalStorage["omnivoice.app"](persistversion: 4), written through pre-existingpartializekeys (cast,storyTracks,currentProjectId). All shapes below are in-memory TypeScript/JS function contracts.
Function signatures (copy verbatim into the file as JSDoc/typedefs)¶
Public transform — frontend/src/utils/dubToStory.js:
/**
* @typedef {Object} DubToStoryOptions
* @property {Array<{ id: string, name: string }>} [profiles] // default []
* @property {string} [unknownSpeakerLabel] // default 'Speaker'
*/
/**
* @typedef {Object} DubToStoryResult
* @property {CastMember[]} cast
* @property {StoryTrack[]} tracks
* @property {{ lines: number, speakers: number, skippedEmpty: number }} stats
*/
export function dubToStory(
segments /*: DubSegment[] | unknown */,
options /*: DubToStoryOptions */ = {}
) /*: DubToStoryResult */ { … }
Internal helpers (module-private; not exported, but pinned so the implementer doesn't guess):
// Slug a speaker name to a unique cast id within this transform call.
// - base: name.toLowerCase().replace(/[^a-z0-9]+/g,'-').replace(/^-|-$/g,'') || 'char'
// - 'narrator' (case-insensitive name) → base 'narrator'
// - mutate `assignedIds`: on collision append -2,-3,…; add the chosen id; return it.
function castIdFor(name /*: string */, assignedIds /*: Set<string> */) /*: string */
// Resolve a segment's profile_id (unknown-typed) to a Stories profileId | null.
// '' | null | undefined → null
// 'auto:…' → first profiles[i].name===displayName (ci) ? id : null
// 'preset:…' → null (v1)
// any other non-empty string → that string (verbatim pass-through)
function resolveVoice(
seg /*: unknown */,
displayName /*: string */,
profiles /*: Array<{ id: string, name: string }> */
) /*: string | null */
Decision: do not export
castIdFor/resolveVoice. They stay module-private; the test plan exercises them only throughdubToStory's public output (cast ids, castprofileIds). This keeps the public surface to one function and avoids the test coupling to internal helper shapes. (See the per-helper assertions in the Test plan — every helper behavior is reachable from adubToStory(...)input.)
Store action — frontend/src/store/storiesSlice.ts (interface + impl):
Input — DubSegment (the actual type, dubSlice.ts:46)¶
DubSegment = Record<string, unknown> & { id: string; text: string }. Only id/text are statically typed; every other field below is unknown at the type level and must be coerced (this is the contract — the algorithm's coercion rules above are mandatory because of this type, not optional hardening):
{
id: string; // present per type; NOT read by the transform (track ids are reassigned 1..N)
text: string; // STATICALLY string per type, but coerce: only string|number kept, else dropped
start?: unknown; // ordering only; Number(start) finite ? value : 0
end?: unknown; // unused (no Stories timeline)
speaker_id?: unknown; // grouping key; (==null ? '' : String(x)).trim(); empty → synthetic "unknown"
profile_id?: unknown; // (==null ? '' : String(x)); '' | '<id>' | 'auto:<slug>' | 'preset:<id>'
text_original?: unknown; // unused in v1
}
profiles (prop; minimal shape this transform reads)¶
(The Stories voice <select> lists these by p.id/p.name — StoriesEditor.jsx:557,715.)
speakerClones (store/prop — NOT consumed by the v1 transform, kept here for the call-site reader; dubSlice.ts:120-125)¶
Record<string /*speaker_id*/, {
ref_audio: string;
ref_text: string;
duration: number; // confirmed numeric — Dub UI does clone.duration.toFixed(1), DubTab.jsx:848
source_count: number;
}>
dubToStory signature — see E24. auto: binding uses profiles only.)
Output — Stories shapes (from storiesSlice.ts:12-26, unchanged)¶
CastMember = { id: string; name: string; color: string; profileId: string | null }
StoryTrack = { id: number; character: string; text: string; profileId: string|null; emotion: string|null; speed: number|null }
{ cast: CastMember[]; tracks: StoryTrack[]; stats: { lines: number; speakers: number; skippedEmpty: number } }
Persisted localStorage contract (the only durable store — unchanged shape)¶
Key omnivoice.app, persist version: 4 (store/index.ts:115). The action writes exactly:
{
"state": {
// … all other persisted keys unchanged …
"cast": [ { "id": "alice", "name": "Alice", "color": "#fabd2f", "profileId": "p_xyz" } ],
"storyTracks": [ { "id": 1, "character": "alice", "text": "Hello.", "profileId": null, "emotion": null, "speed": null } ],
"currentProjectId": null
// storyProjects untouched by loadStoryFromDub
},
"version": 4
}
partialize (index.ts:109-110) re-destructures storyTracks to exactly {id,character,text,profileId,emotion,speed} — bit-identical to the transform's StoryTrack, so there is nothing to migrate and the version stays 4.
Worked example (input → output — the canonical golden fixture)¶
Input segments:
[
{ "id": "s2", "start": 1.2, "text": "Hi there.", "speaker_id": "Alice", "profile_id": "p_alice" },
{ "id": "s1", "start": 0.0, "text": "Hello.", "speaker_id": "Alice", "profile_id": "p_alice" },
{ "id": "s3", "start": 2.5, "text": " ", "speaker_id": "Bob", "profile_id": "auto:bob" },
{ "id": "s4", "start": 3.1, "text": "Who's there?","speaker_id": "Bob", "profile_id": "auto:bob" },
{ "id": "s5", "start": 4.0, "text": "...", "speaker_id": "", "profile_id": "preset:warm" }
]
{ profiles: [{ id: "pb", name: "Bob" }], unknownSpeakerLabel: "Speaker" }.
Output dubToStory(segments, options):
{
"cast": [
{ "id": "alice", "name": "Alice", "color": "#fabd2f", "profileId": "p_alice" },
{ "id": "bob", "name": "Bob", "color": "#d3869b", "profileId": "pb" }, // auto:bob → name-match → pb
{ "id": "speaker", "name": "Speaker", "color": "#83a598", "profileId": null } // preset:* → null
],
"tracks": [
{ "id": 1, "character": "alice", "text": "Hello.", "profileId": null, "emotion": null, "speed": null },
{ "id": 2, "character": "alice", "text": "Hi there.", "profileId": null, "emotion": null, "speed": null },
{ "id": 3, "character": "bob", "text": "Who's there?","profileId": null, "emotion": null, "speed": null },
{ "id": 4, "character": "speaker", "text": "...", "profileId": null, "emotion": null, "speed": null }
],
"stats": { "lines": 4, "speakers": 3, "skippedEmpty": 1 }
}
s1/s2; the whitespace-only s3 is dropped and counted in skippedEmpty (its speaker Bob still appears, from surviving s4); auto:bob binds to the name-matched profile pb; preset:warm → null; the empty-speaker_id s5 collapses into the synthetic "Speaker" (id speaker); track ids are 1..4 regardless of input ids. The colors #fabd2f/#d3869b/#83a598 are CAST_COLORS[0..2] — confirmed by storyCast.test.js's "first unused" assertions. Make this exact object an expect(...).toEqual(...) assertion in the unit test.
Edge cases & failure paths (exhaustive)¶
This feature touches user-controlled, model-derived, and externally-imported (SRT) data, so the transform must survive every shape below. The transform never throws and always returns a valid {cast, tracks, stats}. The right-most column names the test (Test plan §Unit) that pins each row.
| # | Condition | Behavior | Pinned by test |
|---|---|---|---|
| E1 | segments is undefined/null/non-array |
Return empty result (lines:0, speakers:0, skippedEmpty:0). UI guard B catches tracks.length===0 → create_story_empty. |
returns the empty result for non-array input |
| E2 | segments is [] (empty) |
Same as E1. (UI guard A also blocks; button hidden.) | returns the empty result for non-array input |
| E3 | All segments empty/whitespace/missing text |
cast: [], tracks: [], skippedEmpty === segments.length. UI shows create_story_empty, no navigate, preserves open Story. |
drops empty/whitespace lines and counts skippedEmpty |
| E4 | Some empty, some not | Empty dropped + counted; non-empty produce tracks. Cast built only from speakers of surviving segments (a speaker whose only line was empty does not become a cast member). | golden fixture + a speaker with only empty lines never enters the cast |
| E5 | seg.text is a number (0, 42) |
Stringified; "0"/"42" non-empty → kept. (0 survives: String(0).trim()==="0".) |
keeps numeric text (0 survives) and drops object/array text |
| E6 | seg.text is null/undefined/object/array |
typeof not string/number → '' → dropped, counted. |
keeps numeric text (0 survives) and drops object/array text |
| E7 | seg.start missing / NaN / non-numeric string / null |
Normalized to 0; stable sort keeps input order. No throw. |
treats missing/NaN/string start as 0 and preserves input order |
| E8 | seg.start negative |
Valid; sorts before 0. Relative-only, harmless. |
sorts negative start before zero |
| E9 | All start equal (or all missing) |
Stable sort preserves input order → tracks in table order. | treats missing/NaN/string start as 0 and preserves input order |
| E10 | seg.speaker_id falsy/empty/whitespace |
All collapse into one synthetic speaker unknownSpeakerLabel, id via castIdFor (likely speaker). Not one-per-segment. |
collapses all unattributed segments into one synthetic speaker |
| E11 | seg.speaker_id is a number |
Coerced (String(5)→"5"); slug keeps digits → id "5". Distinct numerics → distinct ids. |
coerces numeric speaker_id to a distinct string id |
| E12 | Two distinct speaker_ids slug identically |
Uniqueness suffixing → id, id-2. Separate cast members; their tracks reference correct distinct ids. |
suffixes colliding slugs and keeps tracks pointing at the right id |
| E13 | A speaker_id slugs empty ("???", " - ") |
Base id 'char'; second → 'char-2'. |
empty-slug speaker_id becomes char then char-2 |
| E14 | A real speaker named "Narrator" | Cast id narrator (special case). A second narrator-slug speaker → narrator-2. |
a speaker named Narrator takes the narrator id |
| E15 | unknownSpeakerLabel localizes to "Narrator" and a real "Narrator" exists |
First-appearing claims narrator; the other gets narrator-2. No collision/throw. |
unknownSpeakerLabel "Narrator" collides with a real Narrator → narrator + narrator-2 |
| E16 | Same speaker, different profile_id across segments (SRT/external only) |
First segment (sorted order) wins; later divergence ignored. No per-line override. Documented limitation. | same speaker with diverging profile_id uses the first segment |
| E17 | profile_id === '' / missing / null |
→ null (cast Default). |
resolveVoice matrix (table-driven) |
| E18 | profile_id = auto:<x>, saved profile name-matches the speaker |
→ that profile's id. |
resolveVoice matrix |
| E19 | profile_id = auto:<x>, no name match |
→ null (degrade to Default; never dangle). |
resolveVoice matrix |
| E20 | profile_id = auto:<x>, multiple name-matches |
First match by profiles order wins (deterministic). |
auto: with two same-named profiles binds the first |
| E21 | profile_id = preset:<id> |
→ null for v1. |
resolveVoice matrix |
| E22 | profile_id = real id deleted since assignment |
Pass through verbatim; Stories <select> renders blank/Default; /longform/render uses default voice. No crash. |
resolveVoice matrix (deleted-but-passed id row) |
| E23 | profiles prop undefined/non-array |
Coerced to []; all auto: → null; pass-through ids still pass. |
non-array profiles coerces to [] and never throws |
| E24 | speakerClones param |
Dropped from the v1 signature. v1 resolves voices from profile_id + profiles only; the auto: match uses profiles. Re-add the param when a "bind auto-clone → saved profile" feature lands. (The call site no longer passes it — see §3 handler.) |
(no test — absence of a param; the signature test below asserts arity 2) |
| E25 | >8 distinct speakers |
First 8 get distinct palette colors (nextCastColor first-unused); 9th+ reuse via length % 8. Not an error; cast distinct by id/name. |
nine distinct speakers reuse CAST_COLORS[0] for the 9th |
| E26 | Duplicate segment ids |
Irrelevant — track ids reassigned 1..N; input id not read. |
reassigns track ids 1..N regardless of input ids |
| E27 | Single segment, single speaker | 1 cast member, 1 track. | single speaker, three lines → one cast member, ordered tracks |
| E28 | Story open (storyTracks.length>0), user cancels confirm |
No change: no navigation, no store write, no toast. Dub untouched. | (handler path — manual smoke; not unit-tested, see rationale) |
| E29 | Story open, user confirms | Live working tracks/cast replaced; currentProjectId: null. Saved storyProjects untouched (recoverable from Projects panel). |
loadStoryFromDub then saveProject creates a new project, saved projects survive (slice test) |
| E30 | No Story open (storyTracks.length===0) |
No confirm; proceed directly. | (handler path — manual smoke) |
| E31 | Click while dubStep ∈ {generating,stopping} |
Button disabled; even a stale click runs harmlessly on current segments. | (UI gating — manual smoke) |
| E32 | Create-Story twice | Second run prompts clobber confirm (Story now populated). _trackId already reseeded; new ids unique. |
(handler path — manual smoke) |
| E33 | Navigate to Stories where StoriesEditor never mounted before | On mount the reseed effect sets _trackId = max(loaded id); later lines unique. |
(StoriesEditor existing behavior — not re-tested here) |
| E34 | loadStoryFromDub(undefined, undefined) (future caller) |
\|\| [] guards → empty arrays set; no throw. |
loadStoryFromDub(undefined, undefined) sets empty arrays (slice test) |
| E35 | Orphan track risk | Impossible by construction — cast built from the same filtered+sorted segments before tracks, so every track's character is in castIdMap. |
every track character references an existing cast member |
| E36 | Pathological speaker_id (very long, all-punctuation, mixed unicode) into castIdFor regex |
[^a-z0-9]+ slug is linear-time on any length (no catastrophic backtracking); a 10⁵-char adversarial speaker_id slugs O(n). No ReDoS (see Constraints). |
adversarial-length speaker_id slugs in linear time |
Integration points (file:line)¶
frontend/src/pages/DubTab.jsx:704— post-transcription editor gate (dubJobId && (dubStep === 'editing'|'generating'|'done')); the button must live inside this branch.frontend/src/pages/DubTab.jsx:715-750— active-editor headerdub-head__actions(Save/ResetButtons at 718-721,FooterBtnprimary row at 723-749); add "Create Story" here.frontend/src/pages/DubTab.jsx:724-728— existingstopping/generatinggating to mirror for the disabled state.frontend/src/pages/DubTab.jsx:101-110— existing store reads (dubStepat 102,dubSegmentsat 109,setDubSegmentsat 110); addsetMode,loadStoryFromDub,storyTracksalongside.frontend/src/pages/DubTab.jsx:831-867— Dub Cast UI;profile_idper speaker assigned (setDubSegments(dubSegments.map(s => s.speaker_id === spk ? { ...s, profile_id: val } : s))at 845 — writes to every segment of that speaker, confirming "shared profile_id within a job").auto:prefix built at 836;preset:at 858; saved-profile ids from thedub.clone_profilesoptgroup at 851-855.frontend/src/pages/DubTab.jsx:1048-1049— "Restore original" overwritesseg.textwithseg.text_original, confirmingseg.textis always "what the user sees".frontend/src/store/dubSlice.ts:46—DubSegment = Record<string, unknown> & { id: string; text: string }(the input type; everything exceptid/textisunknown→ coerce).frontend/src/store/dubSlice.ts:120-125—speakerClonesvalue shape ({ref_audio, ref_text, duration:number, source_count}); not consumed by v1.frontend/src/store/dubSlice.ts:148,50-52—setDubSegmentsupdater signature (T | (prev=>T)); transform readsdubSegments, never the setter.frontend/src/store/storiesSlice.ts:36-51—StoriesSliceinterface; addloadStoryFromDubsignature.frontend/src/store/storiesSlice.ts:101-106—loadProject(101-105) /newProject(106); addloadStoryFromDubimpl adjacent.frontend/src/store/storiesSlice.ts:84-100—saveProject(id = s.currentProjectId || genProjectId()at 86); confirmscurrentProjectId: null→ a new saved project on first save.frontend/src/store/storiesSlice.ts:12-26—StoryTrack(12-19) /CastMember(21-26) output types (do not change).frontend/src/store/storiesSlice.ts:53-55—DEFAULT_CAST(narrator default) — noteloadStoryFromDubdoes not re-seed it (unlikenewProjectat 106).frontend/src/store/storiesSlice.test.ts:4-10,12-14— the harness to extend.harness()builds the slice viacreateStoriesSlice(set, get, {})with a hand-rolledset/get(no provider, nomain, no torch);track(id, character, text)helper at 12-14 mints a validStoryTrack.loadStoryFromDubtests reuse both.frontend/src/utils/storyCast.js:9-12,15-18—CAST_COLORS(8-color palette at 9-12) + reusenextCastColor; exact semantics: first unusedCAST_COLORSentry, elseCAST_COLORS[length % 8].effectiveProfile(21-25) documents the line-override→cast→default resolution thenulltrackprofileIdrelies on.frontend/src/utils/storyCast.test.js— existing assertionsnextCastColor([]) === CAST_COLORS[0]andnextCastColor([{color: CAST_COLORS[0]}]) === CAST_COLORS[1]; the golden-fixture colors follow directly. The newdubToStory.test.jsimportsCAST_COLORSfromstoryCast.jsfor the >8-speaker assertion rather than hard-coding hex.frontend/src/utils/importStory.test.js,frontend/src/utils/parseScript.test.js— the pure-util test template (import { describe, it, expect } from 'vitest', no setup file, table-drivenexpect(fn(input)).toBe/.toEqual(output));dubToStory.test.jsfollows this form exactly.frontend/src/components/StoriesEditor.jsx:91-92—makeTrackshape (target track shape, minus transientgenerating/audioUrl);:95-98—genCastId(random helper, not used here);:136-140—_trackIdreseed effect (mount-only; why util-assigned1..Nids are safe and re-entrant);:166—deleteCharacternarrator guard (why thenarratorspecial-case id matters);:173—slug()rule to mirror forcastIdFor(linear-time, ReDoS-safe — copy verbatim, do not "improve" into backtracking);:175-199—autoCast(closest existing precedent: builds cast+tracks from parsed speakers, special-casesnarrator, assignsnextCastColor, dedupes existing cast by id).frontend/src/components/StoriesEditor.jsx:557,715— Stories voice<select>listsprofilesbyp.id/p.name; confirms unknown ids render blank (no crash) and profile objects expose{id, name}.frontend/src/App.jsx:1115-1135—<DubTab>render;profiles={profiles}(1124),speakerClones={speakerClones}(1135) already passed.frontend/src/store/index.ts:72-114—partialize;storyTracks(109-110, destructured to{id,character,text,profileId,emotion,speed}),cast(111),storyProjects(112),currentProjectId(113) persisted. New tracks/cast flow through unchanged — no partialize edit, noversionbump (version: 4at:115).frontend/src/store/index.ts:120-130—migrate(pass-through forversion < 4, identity for4); untouched.frontend/src/i18n/locales/en.json—"stories"block at line 26 (closes ~95),"dub"block at line 524; add new keys (below). Mirror into all 21i18n/locales/*.json. (Propagation is viascripts/translate_all.py, a manual script — no CI parity gate exists, see Test plan.)frontend/src/i18n/index.ts:4,13-…—en.jsonis statically imported (import en from './locales/en.json') and the other 20 locales are lazyimport()s; the new keys must exist inen.json(the always-loaded base) and in each lazy locale..github/workflows/security.yml:74,95-96— CodeQL matrix (python,javascript-typescript) +security-and-qualitysuite (includesjs/polynomial-redos); the new.jsregex is in scope.tests/test_no_hardcoded_cjk.py:89— CJK gate;_is_allowedauto-allows files whose basename contains.test.(alsotest_*/*_test.py), and prefix/tests/paths; scans git-tracked source. Transform (no literal CJK; localized label injected) and itsdubToStory.test.jsdon't trip it.docs/features.yaml:11-23(coarsefeatures[]:Video Dubbing, …) +scripts/check-docs-drift.py/scripts/validate-install-docs.py(ci.yml:75) — docs-drift gates; key off this coarse list + engine registries; a sub-feature button touches none → structural no-op.
API / data shapes — new i18n keys¶
New i18n keys (en.json; mirror into all 21 locale files under frontend/src/i18n/locales/ per the Localization rule — values may stay English until translated, but keys must exist so t() doesn't render raw keys). Add the dub.* keys to the "dub" block (en.json:524) and stories.fromDub.unknownSpeaker to the "stories" block (en.json:26):
dub.create_story= "Create Story from Dub"dub.create_story_confirm= "Replace the current Stories project with this dub's cast and lines? (Saved Stories projects are not affected.)"dub.create_story_done= "Created a story: {{lines}} lines, {{speakers}} voice(s)" (interpolation vars:lines:number,speakers:number— passed fromstats)dub.create_story_empty= "No transcript lines to turn into a story yet."stories.fromDub.unknownSpeaker= "Speaker"
(Note: dub.cast = "CAST" and dub.default = "Default" already exist at en.json:543,545 — do not redefine; the new keys are all unused names.)
i18n-mirroring mechanics (so the PR doesn't ship 20 missing keys): add the keys to
en.jsonfirst, then propagate the same key paths to the other 20 files with English placeholder values (runscripts/translate_all.py, or hand-add placeholders). Keys must exist in every file — there is no automated key-parity test in CI, so a missing key silently falls back to the raw key string at runtime (react-i18next default), surfacingdub.create_story_doneliterally to non-English users — a localization regression that no gate will catch for you.stories.fromDub.unknownSpeakeris the value displayed as a cast name, so once translated it localizes the synthetic speaker's name (and, per E15, can change its slug) — expected.
Test plan¶
Strategy — pure / handler-direct, no main+torch/GPU import (the load-bearing rule)¶
Per MEMORY (local-pytest-segfault-torch-triton), any test path that imports main + torch can segfault locally and is slow/GPU-coupled. Every test in this PR avoids that path entirely by testing at two pure seams the repo already uses:
- The transform is a pure function.
dubToStory.test.jsimports onlydubToStory(andCAST_COLORSfromstoryCast.jsfor color assertions) — no store, no React, nomain, no torch, noimport '../i18n', no setup file. It follows the exact form ofimportStory.test.js/parseScript.test.js/storyCast.test.js:import { describe, it, expect } from 'vitest', then table-drivenexpect(dubToStory(input, opts)).toEqual(output). This is fully deterministic and runs in milliseconds with zero native deps. - The store action is tested via the slice harness, not a mounted store. Extend
storiesSlice.test.tsusing its existingharness()(createStoriesSlice(set, get, {})over a hand-rolledset/getobject —storiesSlice.test.ts:4-10) and thetrack()helper (:12-14). This constructs the slice in isolation — no Zustandcreate, no persist middleware, no provider, nomain, no torch. Callingget().loadStoryFromDub(...)and asserting onget().storyTracks/get().cast/get().currentProjectIdis the same pattern the slice's other actions are already tested with. - The UI handler +
window.confirm+setModenavigation are NOT unit-tested. RenderingDubTab.jsxpulls in the real store andimport '../i18n'(the heavier path component tests likeExportModal.test.jsxtake), andwindow.confirmis awkward/unreliable in the webview test harness. The branch logic (guards A/B/C) is trivial glue over the two pure seams above, which are exhaustively tested; the navigation + confirm are covered by the manual smoke (E28–E32). This split is deliberate: maximize coverage of the data-correctness logic in fast pure tests, accept manual verification for the thin imperative shell.
Run with bunx vitest run (the test script, package.json:"test": "vitest run"). No new test deps; no setup file.
Unit — frontend/src/utils/dubToStory.test.js (NEW)¶
Vitest, flat import { describe, it, expect } from 'vitest'; also import { CAST_COLORS } from './storyCast' for the color/palette assertions (don't hard-code hex). Concrete it(...) blocks and what each asserts:
the worked-example golden fixture deep-equals the pinned output—expect(dubToStory(segments, options)).toEqual({…})against the exact object in §"API / data shapes". One assertion locks cast order/colors/ids, track ids/order/text, andstatstogether (covers E4, E10, E18, E21, E35 simultaneously).returns the empty result for non-array input— loops[undefined, null, {}, 'x', 42, []], each→ { cast: [], tracks: [], stats: {lines:0,speakers:0,skippedEmpty:0} }, wrapped inexpect(() => …).not.toThrow()(E1, E2).single speaker, three lines → one cast member, ordered tracks— 1 cast member; 3 tracks in order; each track'scharacterequalscast[0].id; every track'sprofileId/emotion/speedisnull; track ids[1,2,3](E27, E35, E26).every track character references an existing cast member— for a multi-speaker input, asserttracks.every(t => cast.some(c => c.id === t.character))istrue(E35 — the orphan-impossibility invariant, asserted directly).two interleaved speakers get first-appearance order and first-unused colors— 2 distinct cast members in first-appearance order; tracks keep interleaved order;cast[0].color === CAST_COLORS[0],cast[1].color === CAST_COLORS[1].treats missing/NaN/string start as 0 and preserves input order— segments withstartmissing /NaN/"abc"/nulland equal/absent values; assert resulting track-text order equals input order (stable sort; E7, E9).sorts negative start before zero— astart: -1segment lands before astart: 0one (E8).out-of-order start is start-sorted—start[2,0,1]→ track text order matches ascending-start order (E7).resolveVoice matrix— table-driven; one speaker per row, single line each, assert the resulting cast member'sprofileId:- real id
"p1"→"p1"(passthrough); ""→null; missingprofile_id→null;profile_id: null→null(E17);auto:fox+profiles:[{id:'pf',name:'fox'}]→"pf"(E18);auto:fox+ no match →null(E19);auto:FOX+{name:'fox'}(case-insensitive) →"pf";preset:warm→null(E21);"deleted_id"not inprofiles→"deleted_id"verbatim (E22).auto: with two same-named profiles binds the first—profiles:[{id:'a',name:'fox'},{id:'b',name:'fox'}],auto:fox→"a"(E20, deterministic by array order).non-array profiles coerces to [] and never throws—dubToStory([{id:'1',text:'hi',speaker_id:'A',profile_id:'auto:a'}], { profiles: undefined })→ no throw, that cast member'sprofileId === null(E23).drops empty/whitespace lines and counts skippedEmpty— mix of""," ", missing-text, and one real line; asserttracks.length === 1,stats.skippedEmpty === 3,stats.lines === 1(E3 partial; full-empty E3 also: an all-empty input →cast:[], tracks:[], skippedEmpty===N).a speaker with only empty lines never enters the cast— speaker "Ghost" with only a" "line + speaker "Real" with text →casthas exactly one member named "Real" (E4).keeps numeric text (0 survives) and drops object/array text—text: 0→ track text"0"kept;text: 42→"42";text: {}/text: []/text: null→ dropped+counted (E5, E6).collapses all unattributed segments into one synthetic speaker— three segments withspeaker_idof""," ",null(each non-empty text) → exactly one cast member namedunknownSpeakerLabel, all 3 tracks share its id (E10).coerces numeric speaker_id to a distinct string id—speaker_id: 5andspeaker_id: 7→ cast ids"5"and"7"(E11).a speaker named Narrator takes the narrator id—speaker_id: "Narrator"→ cast id"narrator"; a second narrator-slug speaker (e.g."NARRATOR!") →"narrator-2"(E14).unknownSpeakerLabel "Narrator" collides with a real Narrator → narrator + narrator-2— call withunknownSpeakerLabel: 'Narrator', input has a real "Narrator" first then an unattributed segment; assert the first-appearing claims"narrator", the other"narrator-2", no throw (E15).suffixes colliding slugs and keeps tracks pointing at the right id—"Speaker 1"and"Speaker 1"(double space) → cast ids"speaker-1"and"speaker-1-2"; assert each speaker's track references its own distinct id (E12).empty-slug speaker_id becomes char then char-2—"???"→"char"; a second all-punctuation speaker"!!!"→"char-2"(E13).same speaker with diverging profile_id uses the first segment— samespeaker_id, two segments with differentprofile_id(sorted bystart); cast member'sprofileIdis from the first sorted segment (E16).reassigns track ids 1..N regardless of input ids— input with duplicate/stringids → track ids exactly[1..N]ascending (E26).adversarial-length speaker_id slugs in linear time—speaker_id: 'x.'.repeat(50000)mixed with punctuation; assert the call returns a valid result (cast length ≥ 1) and the test completes well under vitest's default timeout. This is the local proxy for the CodeQLjs/polynomial-redosgate — if a future edit replaces the linearslug()with a backtracking regex, this test blows its timeout (E36). (Optional belt-and-braces: wrap in ait(..., { timeout: 1000 })so a regression is an explicit timeout failure, not a hang.)nine distinct speakers reuse CAST_COLORS[0] for the 9th— 9 distinct speakers; assertcast.slice(0,8).map(c=>c.color)equalsCAST_COLORS(all 8 distinct, first-unused), andcast[8].color === CAST_COLORS[8 % CAST_COLORS.length] === CAST_COLORS[0]; cast ids all distinct; no throw (E25).stats reflect tracks, cast, and dropped counts— for a mixed fixture assertstats.lines === tracks.length,stats.speakers === cast.length, andstats.skippedEmptyequals the number of dropped segments.
Unit — extend frontend/src/store/storiesSlice.test.ts¶
Add a describe('storiesSlice — loadStoryFromDub', ...) using the existing harness() (:4-10) and track() (:12-14). Concrete it(...) blocks:
loadStoryFromDub sets tracks, cast, and clears currentProjectId— callget().loadStoryFromDub([{id:'a',name:'A',color:'#fabd2f',profileId:'p1'}], [track(1,'a','hi')]); assertget().storyTrackshas length 1 withcharacter: 'a',get().casthas the one member,get().currentProjectId === null.defensive deep-enough copy: mutating the passed arrays/objects after the call does not change store state— pass arrays, then mutate both the arrays (.push(...)) and a member object (cast[0].name = 'X'); assertget().cast[0].nameis unchanged and lengths are unchanged (validates the{ ...c }/{ ...t }shallow copy is sufficient because members are flat).loadStoryFromDub(undefined, undefined) sets empty arrays and does not throw—expect(() => get().loadStoryFromDub(undefined, undefined)).not.toThrow();get().storyTracks=== [],get().cast=== [](E34).empty cast is not re-seeded with DEFAULT_CAST—get().loadStoryFromDub([], []); assertget().casthas length 0 (contrastnewProject()which seedsDEFAULT_CAST— assert that contrast explicitly by also callingnewProject()on a fresh harness and showing it yields a 1-member narrator cast).after loadStoryFromDub, saveProject creates a NEW project and leaves saved ones intact— on a harness:saveProject('Existing')(snapshot exists), thenloadStoryFromDub(dubCast, dubTracks), thensaveProject('FromDub'); assertget().storyProjectshas length 2, the original "Existing" still present, andget().currentProjectIdnow points at the new one (E29 — confirmscurrentProjectId: null→genProjectId()branch atstoriesSlice.ts:86).
Constraint gates (must be green before merge)¶
- CodeQL (
javascript-typescript,security-and-quality) — no newjs/polynomial-redosalert. The only regex is the verbatim linearslug()fromStoriesEditor.jsx:173;profile_iddiscrimination usesstartsWith/equality. Theadversarial-lengthvitest test is the local proxy that catches a regression before CodeQL does. (Per MEMORY merge-discipline: watch this check explicitly ongh pr checks— a new.jsregex file is exactly what this query targets.) - CJK gate —
pytest tests/test_no_hardcoded_cjk.py— passes. The transform has no literal CJK (the only displayable string,unknownSpeakerLabel, is injected by the i18n caller), anddubToStory.test.jsis auto-allowed (_is_allowed,.test.basename,tests/test_no_hardcoded_cjk.py:89). (Run locally — pure Python, no torch import, no segfault risk.) - i18n completeness (no automated gate — manual checklist) — there is no CI key-parity test, so this is a reviewer checklist, not a green check: all 5 new keys (
dub.create_story,dub.create_story_confirm,dub.create_story_done,dub.create_story_empty,stories.fromDub.unknownSpeaker) present in all 21i18n/locales/*.json. (Optional hardening, out of scope but worth a follow-up: a tiny vitest that loadsen.jsonflat keys and asserts each lazy locale contains every key path — would turn this manual checklist into a real gate. Not required for this PR.)
Local gates (per MEMORY merge-discipline: CI gates)¶
bunx vitest rungreen (includes the two new/extended test files);bun run lintclean (eslint);bun run typecheck:ciclean.pytest tests/test_no_hardcoded_cjk.pygreen (safe locally — no torch).- Do NOT run the full
pytestsuite locally if it importsmain+torch (segfaults per MEMORY); let CI validate those. This PR adds no Python, so no backend test is implicated anyway. - After push (not docs-only), Monitor
gh pr checksuntil green — wait specifically for the CodeQL (javascript-typescript) job and the CJK job, since this PR adds a new JS regex file and five new user-facing strings.
Manual smoke (covers the un-unit-tested handler path — E28–E32)¶
On /run (Dub tab): load a 2-speaker clip → transcribe → assign voices in the Dub Cast UI (DubTab.jsx:831) → click "Create Story from Dub" → lands on Stories with 2 cast members carrying the assigned voices and all lines present in start order. Then:
- (a) clobber confirm — with a Story already open (
storyTracks.length>0), click again; confirm thedub.create_story_confirmprompt appears, Cancel preserves the open Story (no navigation/no toast — E28), Confirm replaces the working tracks and a previously saved project is still in the Projects panel (E29). - (b) empty path — clear every line's text, click; confirm the
dub.create_story_emptytoast and no navigation (E3/guard B). - (c) disabled during generation — start generation; confirm the button is disabled while
dubStep ∈ {generating,stopping}(E31). - (d) twice — after landing on Stories, return to Dub and click again; confirm it re-prompts the clobber (Story now populated) and new line ids don't collide (E32, E33).
Cross-platform note: default feature → the same smoke must hold on macOS/Windows/Linux. The only platform-touching call is window.confirm (webview-native, identical across WKWebView/WebView2/WebKitGTK), and the success toast/setMode are pure JS, so a single-platform manual smoke is sufficient evidence of the default behavior — no per-OS divergence is possible.
Constraints¶
This feature ships in default mode (a visible button, no toggle, no opt-in), so by the project's strict P0 default-behavior rule it must behave identically across platforms. Each relevant hard rule and how it's met:
- Cross-platform parity (P0 default-behavior rule). Everything here is pure JS in the Tauri webview: the
dubToStorytransform, the Zustand actionloadStoryFromDub,setMode('stories')(uiSlice.ts:94). The single OS-surface element iswindow.confirmfor the clobber guard, webview-native and identical on macOS (WKWebView), Windows (WebView2), Linux (WebKitGTK). No filesystem, shell, or native-dialog API. Nothing to "miss", nothing to move behind opt-in. The user-visible default behavior (button → transform → navigate → toast) is bit-identical across all three. - Local-first guarantee. Zero network calls added — no new HTTP request/response, no SSE event, no API route. The transform reads in-memory store state + a prop; the action mutates client-side persisted state only. No GitHub Issues URL, telemetry, account, API key, or backend round-trip — the whole feature runs offline. (The pre-existing
API//longform/render//dub/audio/:jobIdsymbols in the touched file are not called by this PR.) - Backward-compatible project data (no migration). No DB/alembic touch (front-end-only).
loadStoryFromDubwrites only already-persistedcastandstoryTracksshapes (store/index.ts:109-111), introducing no new persisted key and no shape change, so persistversionstays4(index.ts:115) and the pass-throughmigrate(index.ts:120-130) is untouched. Existing localStorage rehydrates exactly as before — nothing to migrate. (partializealready strips transientgenerating/audioUrl; the transform never sets those.) - CodeQL
js/polynomial-redos. CodeQL gates on bothpythonandjavascript-typescriptwithsecurity-and-quality(security.yml:74,95-96), which includes the polynomial-ReDoS query. The new.jstransform runs a regex on user/model-derivedspeaker_id, so it is in scope. The only regex iscastIdFor's slug, copied verbatim fromStoriesEditor.slug()(StoriesEditor.jsx:173):name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, ''). Both patterns are linear-time — a single bounded negated char class ([^a-z0-9]+) and a simple anchored alternation (^-|-$), no nested quantifiers, no overlapping\s*/.+, no(a+)+— so they cannot exhibit polynomial backtracking (matches the MEMORY ReDoS rule). Theauto:/preset:/pass-through discrimination onprofile_idusesstartsWith+ equality, no regex. Theadversarial-length speaker_idvitest test is the local regression guard that fires before CodeQL would. Net: no new alert. - Localization (hard rule). All five new user-facing strings (
dub.create_story,dub.create_story_confirm,dub.create_story_done,dub.create_story_empty,stories.fromDub.unknownSpeaker) aret()keys added toen.jsonand mirrored into all 21i18n/locales/*.json(placeholder English until translated, but the key must exist everywhere — no CI parity gate catches a missing one; it silently renders the raw key). No string literal in JSX. The transform util is i18n-free by design — receives the already-resolvedunknownSpeakerLabel, carries no hardcoded text.speaker_idvalues (e.g. "Speaker 1") are model/engine-derived data, not UI literals — passing them through as cast names is not a CJK violation; the CJK gate confirms it (scans git-tracked source for literal CJK, transform has none,dubToStory.test.jsauto-allowed at_is_allowed,tests/test_no_hardcoded_cjk.py:89). (The localizedunknownSpeakerLabelcorrectly varies the synthetic speaker's display name and slug per locale — E15 — absorbed by uniqueness suffixing.) - Versioning (continuous-to-main, no RCs). Feature PR; main already carries the next-patch version. No version-file edit (
tauri.conf.json/Cargo.toml/pyproject.tomlunchanged) and no persist-versionbump. No RC tag, nov0.4deferral, no codename — merges continuous-to-main into the open v0.3.x line. - Docs-sync (hard rule) — verified structural no-op. The docs-drift checker (
scripts/check-docs-drift.py, daily + the PR-gatingscripts/validate-install-docs.pyatci.yml:75) keys off the coarsefeatures[]list indocs/features.yaml(Voice Cloning,Video Dubbing,Dictation Widget, … — no "Stories" entry, no sub-feature granularity) plus the TTS/ASR engine registries. A Dub→Stories sub-feature button adds no top-level feature, engine, ordoc:/readme:reference, so neither check fires and nofeatures.yaml/README change is required. A README mention is optional polish — and if the maintainer adds afeatures.yamlentry, that entry's verbatimreadme:string must land in README in the same PR; default expectation is "no doc change needed."
Dependencies¶
- None new. Reuses:
nextCastColor/CAST_COLORS(utils/storyCast.js:9-18),useAppStore(store/index.ts),react-hot-toast(DubTab.jsx:23),react-i18next(DubTab.jsx:3,75),Buttonfrom'../ui'(DubTab.jsx:25), lucide icons already imported (DubTab.jsx:4-9;Users/FileTextavailable,BookOpen/Dramawould need adding). The transform depends only on store-provideddubSegmentsand theprofilesprop (App.jsx:1124). (speakerClonesis not consumed in v1 — E24.) - Test deps: none new.
vitest+@testing-library/reactalready indevDependencies; the new tests need onlyvitest(no Testing Library, noimport '../i18n', no setup file) because they target pure functions and the slice harness. - Conceptually adjacent to TASK #22 (shared
<VoiceSelector>) and #31 (unified LongformProject store) but does not block on them; standalone transform + wiring.
Risk¶
- Auto-clone voices don't survive the hand-off.
auto:<speaker>profiles are per-dub-job artifacts (speakerClones,DubTab.jsx:837), not saved voice profiles. A Story referencing them would dangle once the dub job is gone, so v1 degrades them tonull(cast Default) unless a saved profile with the same name exists inprofiles. Mitigation: note in a success toast or a follow-up "save clone as profile" (ties into TASK #25). Low risk; dangling refs are worse. (E18–E20.) - Original vs translated text. v1 exports the current
text(translated/edited). A user who translated the dub gets a target-language story; one who didn't gets source language. Mitigation: acceptable default ("what you see is what you get"); a future toggle could offertext_original. Documented as Non-goal. preset:voices dropped. Design-preset assignments (DubTab.jsx:858) map tonull(E21). Low impact — presets aren't first-class Stories voices.- Diverging
profile_idacross one speaker's segments (SRT-import path). First-seen wins; later assignments silently ignored (E16). Low risk — the in-app Cast UI writes uniformly. - Clobbering an in-progress Story. Guarded by
window.confirmwhenstoryTracks.length > 0(E28–E30). The new project is unsaved (currentProjectId: null) so saved projects instoryProjectsare never overwritten silently (E29). Test coverage gap (accepted): the confirm/navigation path is manual-smoke only, not unit-tested (the data effect it triggers —loadStoryFromDub+ new-project-on-save — is slice-tested). If this path regresses, the smoke catches it, not CI. - Empty-cast defensive state.
loadStoryFromDub([], [])leaves an empty cast (no narrator re-seed). Not user-reachable (UI guard B blockstracks.length===0), but unit-tested (empty cast is not re-seeded with DEFAULT_CAST) so a future refactor can't regress it into a silent empty-Stories crash. - Slug collisions across speakers — handled by uniqueness suffixing in
castIdFor; covered by tests E12–E15 (suffixes colliding slugs…,empty-slug…,a speaker named Narrator…,unknownSpeakerLabel "Narrator"…). - i18n missing-key regression (no automated catch). There is no CI key-parity gate, so a forgotten key in one of the 20 non-English locales ships a raw
dub.create_story_doneto those users. Mitigation: the mirroring checklist +scripts/translate_all.py; a future tiny vitest parity test would close this (noted as optional hardening). - Constraint-regression risk (low, test-guarded). A future edit to the slug regex (e.g. adding
\s*overlap) could introduce ReDoS and trip CodeQL; theadversarial-length speaker_idtimeout test + "copy slug verbatim" note guard against it. A future contributor adding a hardcoded confirm/toast string instead of at()key would trip the CJK gate; the all-strings-via-t()rule and 21-file mirroring checklist guard against it.
PR slices¶
Single small PR (cohesive, easy to review):
feat(dub): Create Story from Dub — cast+lines exportfrontend/src/utils/dubToStory.js(transform) +frontend/src/utils/dubToStory.test.js.frontend/src/store/storiesSlice.tsloadStoryFromDub(interface + impl) + test additions infrontend/src/store/storiesSlice.test.ts.frontend/src/pages/DubTab.jsxbutton +onCreateStoryhandler + store reads (and, if a non-imported icon is chosen, the lucide import line atDubTab.jsx:4-9).- i18n keys in all 21
i18n/locales/*.json(en.json with real copy; the other 20 with English placeholders). - No
docs/features.yaml/README change required (docs-sync verified no-op); optional one-line README mention only if the maintainer wants discoverability — and if so, it lands in this same PR.
If the reviewer prefers smaller diffs, split as: (1a) transform + store + tests (no UI), (1b) DubTab wiring + i18n. 1a carries both test files and the CodeQL-relevant regex (so CodeQL + the full vitest data-correctness suite run on it first, with no UI to mount); 1b carries all the i18n keys (so the CJK gate runs on it) and only the thin handler glue (smoke-verified). Both slices continuous-to-main, no version bump.
Acceptance criteria¶
- [ ] A "Create Story from Dub" control appears in the active-editor Dub header (
DubTab.jsx:715-750, inside thedubStep ∈ {editing,generating,done}branch) whendubSegments.length > 0, disabled duringgenerating/stopping. - [ ] Clicking it builds a Stories project: one cast member per distinct
speaker_id(first-appearance order, distinct colors fromnextCastColor), oneStoryTrackper non-empty segment instartorder, then navigates viasetMode('stories'). - [ ] Each cast member's
profileIdequals the speaker's assigned saved-profileprofile_id;""/missing/preset:/unmatched-auto:→null(Default); matched-auto:→ the saved profile id. - [ ] Track
textequals the segment's current (edited/target)text; trackprofileId/emotion/speedarenull; every track'scharacterreferences an existing cast member (no orphans). - [ ]
dubToStory(worked-example input)deep-equals the worked-example output object in §"API / data shapes" (golden-fixture assertion —the worked-example golden fixture deep-equals the pinned output). - [ ] The transform never throws on malformed input (null/non-array segments, missing/NaN
start, non-stringtext/speaker_id, undefinedprofiles) and returns a valid{cast, tracks, stats}for every shape in the Edge cases table (each row pinned by a named test in the Test plan; E28–E33 by manual smoke). - [ ] Empty/whitespace/missing-
textsegments are dropped and counted instats.skippedEmpty; when every segment is dropped the UI showsdub.create_story_empty, does not navigate, and preserves any open Story. - [ ] The loaded Stories project is unsaved (
currentProjectId === null);saveProjectcreates a new named project without overwriting existing saved projects (slice testafter loadStoryFromDub, saveProject creates a NEW project…). - [ ] If a non-empty Stories project is already open (
storyTracks.length > 0), the user is asked to confirm before replacement; Cancel leaves everything unchanged; Confirm does not delete saved projects (manual smoke; the data effect slice-tested). - [ ]
dubToStoryandloadStoryFromDubare unit-tested via the pure transform + sliceharness()seams (nomain/torch/GPU import), incl. all edge/guard cases + the adversarial-length slug test;bunx vitest runpasses. - [ ] Cross-platform parity: default-mode, only platform-agnostic webview APIs (lone OS-surface call is
window.confirm); identical on macOS/Windows/Linux — no platform-only path, nothing behind opt-in. - [ ] Local-first: no network call, no HTTP/SSE shape, no telemetry, no account/key added.
- [ ] Backward-compatible data: no persist-
versionbump, nomigratechange, no new persisted key; existing localStorage rehydrates unchanged. - [ ] CodeQL: no new
js/polynomial-redosalert — the only regex is the verbatim linearslug();profile_iddiscrimination usesstartsWith, not regex; the adversarial-length vitest test is the local proxy. - [ ] Localization: all five new strings are i18n keys present in every one of the 21
i18n/locales/*.json(manual checklist — no CI parity gate); no hardcoded UI text in JSX; the CJK gate (pytest tests/test_no_hardcoded_cjk.py) passes. - [ ] Versioning / docs-sync: no version-file change, no RC; no
docs/features.yaml/README change required; any optional README mention lands in the same PR. - [ ] No backend, schema, SSE, or new-dependency changes.
- [ ] CI checks green before merge (explicit watch on CodeQL (
javascript-typescript) and CJK, since this PR adds a new JS regex file and new user-facing strings).
本页原文 Markdown:在 AtomGit 查看·内容源自开源项目 om/VoiceStudio