feat(slint): per-visual-line karaoke highlight for lyrics

A wrapped lyric line (one logical line over two visual rows) lit BOTH rows at
once because the active line was a single word-wrap Text masked by one clip
Rectangle of full block height + scalar width. Now it progresses per visual
line — row 1 fills to 100% before row 2 starts — matching the final Tauri
behavior. NOT a Slint limitation (proven in the investigation doc).

- lyrics_measure.rs (new): swash advance pass over the active line at the same
  font (by font-index) + ppem + wght=700 + 0.2px letter-spacing the view
  renders with, greedy word-wrap (CJK per-char) -> per-visual-line segments
  with pixel widths. Embeds the same TTFs the view registers. swash is the
  shaper femtovg uses, so widths match the render.
- lyrics_sync.rs: on active-index / content-width / font / size / uppercase
  change, segment the active line and push [LyricsSegment{text, start-weight,
  width-weight}] (Tauri's width-weighted partition); pushed atomically with the
  active index. Per-surface cache keys (main + mini). line-progress unchanged.
- state.slint: LyricsSegment struct + active-segments + content-width on
  LyricsState.
- LyricsLinesView.slint: active row renders a VerticalLayout of per-segment
  rows (no-wrap), each the existing dim-base + clipped-bright-overlay with clip
  width = segwidth * clamp((line-progress - start-weight)/width-weight, 0, 1).
  Root publishes content-width. Inactive rows unchanged.
- miniplayer.rs: mini lyrics surface segmentation fan-out.

Investigation + plan: qbz-nix-docs/lyrics/2026-06-18-karaoke-highlight-investigation/.
Owner-verified working. Known tuning point: wrap-boundary precision per font tier.

cc
This commit is contained in:
vicrodh committed 2026-06-18 22:49:11 -06:00
1 parent 41973a1ea1
commit ae550b76a5
8 files changed
+584 -33

No files matched your search

+291
View File
@@ -0,0 +1,291 @@
//! Per-visual-line wrap measurement for the karaoke highlight.
//!
//! # Why this exists
//!
//! A logical lyric line that word-wraps to two (or more) visual lines used to
//! light up ALL visual rows at once, because the active line was rendered as a
//! single `wrap: word-wrap` Text masked by ONE clip rectangle of full block
//! height and a single-scalar width (`LyricsLinesView.slint`). A
//! single-scalar-width rectangle of full height reveals the same left fraction
//! of *every* visual row simultaneously — Tauri instead splits the active line
//! into one element PER VISUAL LINE and fills them sequentially (row 1 to
//! 100%, then row 2). This module is the Rust analogue of Tauri's Pretext
//! layout pass: it reproduces Slint's wrap decision and reports, per visual
//! line, the segment text and its pixel ink width so the engine can partition
//! the single global progress fraction across the segments by width share.
//!
//! # How it measures
//!
//! femtovg (the Linux/Windows renderer) shapes glyphs with `swash`, so a swash
//! advance pass at the same font + ppem + weight axis + letter-spacing matches
//! what is actually rendered. The four bundled variable lyric fonts plus the
//! Inter "System"/default bold are embedded here via `include_bytes!` (the
//! same TTFs `LyricsLinesView.slint` registers via `import`) and parsed once
//! into process-global `swash::FontRef`s (held as owned byte buffers in a
//! `OnceLock`).
//!
//! The active lyric line is rendered BOLD; for the variable fonts we set the
//! `wght` variation axis to 700 so the advances match the bold raster. (Inter
//! "System"/default is measured from `Inter_18pt-Bold.ttf` directly, which is
//! already a bold static instance — no axis needed. LINE Seed JP ships only a
//! Regular file with no weight axis, so it is measured at its single instance;
//! the split is still per-visual-line and self-consistent because we render
//! the segments ourselves.)
//!
//! Letter-spacing is added as a flat `LETTER_SPACING_PX` per glyph cluster, to
//! mirror the `letter-spacing: 0.2px` on the rendered Text items.
//!
//! # Wrapping algorithm
//!
//! Greedy word wrap, matching `wrap: word-wrap`: split the text on ASCII/Unicode
//! whitespace into words; keep appending words (with their separating space) to
//! the current visual line while they fit `max_width_px`; otherwise start a new
//! visual line. A single word longer than the budget is broken per grapheme
//! cluster (CJK / no-space runs also break per character this way). Each
//! emitted segment carries its measured pixel width.
use std::sync::OnceLock;
use swash::shape::ShapeContext;
use swash::FontRef;
/// Per-glyph-cluster letter spacing, mirroring `letter-spacing: 0.2px` on the
/// rendered Text items in `LyricsLinesView.slint`.
const LETTER_SPACING_PX: f32 = 0.2;
/// The weight axis value for the bold active line (`Typography.bold` ~= 700).
const BOLD_WGHT: f32 = 700.0;
/// One bundled lyric font, parsed lazily and kept alive for the process.
struct LoadedFont {
/// Owned font bytes — `FontRef` borrows from these, so they must outlive it.
data: &'static [u8],
/// Font collection index (0 for all single-face TTFs here).
index: usize,
/// Whether this font exposes a `wght` variation axis (set 700 when true).
variable: bool,
}
/// Embedded copies of the EXACT TTFs `LyricsLinesView.slint` registers, plus
/// the Inter bold used for the "System"/default family. `include_bytes!` paths
/// are relative to THIS source file (`crates/qbz-slint/src/`).
static FONT_SYSTEM_INTER_BOLD: &[u8] =
include_bytes!("../ui/assets/fonts/Inter_18pt-Bold.ttf");
static FONT_LINE_SEED_JP: &[u8] =
include_bytes!("../../../static/fonts/LINESeedJP-Regular.ttf");
static FONT_MONTSERRAT: &[u8] =
include_bytes!("../../../static/fonts/Montserrat-VariableFont_wght.ttf");
static FONT_NOTO_SANS: &[u8] =
include_bytes!("../../../static/fonts/NotoSans-VariableFont_wdth,wght.ttf");
static FONT_SOURCE_SANS_3: &[u8] =
include_bytes!("../../../static/fonts/SourceSans3-VariableFont_wght.ttf");
/// Map a `font-index` (the same enum as `LyricsState.font-index` /
/// `LyricsSidebar.slint`: 0=System→Inter, 1=LINE Seed JP, 2=Montserrat,
/// 3=Noto Sans, 4=Source Sans 3) to its embedded bytes. Unknown indices fall
/// back to the System/Inter default, matching the `.slint` `font-name` default.
fn loaded_font(font_index: i32) -> &'static LoadedFont {
static FONTS: OnceLock<[LoadedFont; 5]> = OnceLock::new();
let fonts = FONTS.get_or_init(|| {
[
// 0 System (window default = Inter bold static instance).
LoadedFont { data: FONT_SYSTEM_INTER_BOLD, index: 0, variable: false },
// 1 LINE Seed JP (Regular only — no weight axis).
LoadedFont { data: FONT_LINE_SEED_JP, index: 0, variable: false },
// 2 Montserrat (variable wght).
LoadedFont { data: FONT_MONTSERRAT, index: 0, variable: true },
// 3 Noto Sans (variable wdth,wght).
LoadedFont { data: FONT_NOTO_SANS, index: 0, variable: true },
// 4 Source Sans 3 (variable wght).
LoadedFont { data: FONT_SOURCE_SANS_3, index: 0, variable: true },
]
});
let idx = match font_index {
1..=4 => font_index as usize,
_ => 0,
};
&fonts[idx]
}
/// One measured visual line of a wrapped logical lyric line.
#[derive(Debug, Clone)]
pub struct Segment {
/// The text of this visual line (no leading/trailing wrap whitespace).
pub text: String,
/// The rendered pixel ink+advance width of this visual line (includes the
/// per-cluster letter spacing, so it matches the drawn Text width).
pub width_px: f32,
}
/// Measure the advance width (px) of `text` at `size_px` for `font`, including
/// the flat per-cluster letter spacing — the same metric Slint uses to size a
/// `no-wrap` Text. Returns the total advance.
fn measure_width(ctx: &mut ShapeContext, font: &LoadedFont, text: &str, size_px: f32) -> f32 {
if text.is_empty() {
return 0.0;
}
let Some(font_ref) = FontRef::from_index(font.data, font.index) else {
// Parse failure: fall back to a crude estimate so callers still split.
return text.chars().count() as f32 * size_px * 0.5;
};
let mut builder = ctx.builder(font_ref).size(size_px);
if font.variable {
builder = builder.variations(&[("wght", BOLD_WGHT)]);
}
let mut shaper = builder.build();
shaper.add_str(text);
let mut total = 0.0_f32;
let mut clusters = 0_u32;
shaper.shape_with(|cluster| {
for glyph in cluster.glyphs {
total += glyph.advance;
}
clusters += 1;
});
// Letter spacing is applied per cluster on the rendered side.
total + clusters as f32 * LETTER_SPACING_PX
}
/// Split a single over-long word into per-grapheme-cluster pieces that each fit
/// `max_width_px` (CJK and other no-space runs land here too). Appends the
/// resulting segments to `out`.
fn break_long_word(
ctx: &mut ShapeContext,
font: &LoadedFont,
word: &str,
size_px: f32,
max_width_px: f32,
out: &mut Vec<Segment>,
) {
let mut current = String::new();
let mut current_w = 0.0_f32;
// Break per Unicode scalar (good enough for CJK; avoids pulling in a
// grapheme-segmentation dependency — lyric runs are short).
for ch in word.chars() {
let piece: String = ch.to_string();
let piece_w = measure_width(ctx, font, &piece, size_px);
if !current.is_empty() && current_w + piece_w > max_width_px {
out.push(Segment { text: std::mem::take(&mut current), width_px: current_w });
current_w = 0.0;
}
current.push(ch);
current_w += piece_w;
}
if !current.is_empty() {
out.push(Segment { text: current, width_px: current_w });
}
}
/// Compute the per-visual-line segmentation of `text` as it would word-wrap
/// inside `max_width_px`, at the given `font_index` + `size_px` (bold weight,
/// 0.2px letter spacing — matching the active lyric line render).
///
/// Returns one [`Segment`] per visual line, each with the segment text and its
/// rendered pixel width. An empty / whitespace-only input yields a single empty
/// segment so the caller always has at least one row to render.
pub fn wrap_segments(text: &str, font_index: i32, size_px: f32, max_width_px: f32) -> Vec<Segment> {
let font = loaded_font(font_index);
let mut ctx = ShapeContext::new();
let budget = max_width_px.max(1.0);
// Tokenize into words, preserving nothing of the original whitespace except
// that each inter-word gap becomes a single space when re-joined.
let words: Vec<&str> = text.split_whitespace().collect();
if words.is_empty() {
return vec![Segment { text: String::new(), width_px: 0.0 }];
}
let space_w = measure_width(&mut ctx, font, " ", size_px);
let mut out: Vec<Segment> = Vec::new();
let mut line = String::new();
let mut line_w = 0.0_f32;
for word in words {
let word_w = measure_width(&mut ctx, font, word, size_px);
if line.is_empty() {
// First word on a fresh line.
if word_w > budget {
// Word alone overflows — hard-break it into pieces.
break_long_word(&mut ctx, font, word, size_px, budget, &mut out);
// The tail piece (if any) becomes the start of the current line
// so following words can still pack onto it.
if let Some(last) = out.pop() {
line = last.text;
line_w = last.width_px;
}
} else {
line.push_str(word);
line_w = word_w;
}
continue;
}
// Subsequent word: does it fit with a leading space?
let added = space_w + word_w;
if line_w + added <= budget {
line.push(' ');
line.push_str(word);
line_w += added;
} else {
// Flush the current line and start anew with this word.
out.push(Segment { text: std::mem::take(&mut line), width_px: line_w });
line_w = 0.0;
if word_w > budget {
break_long_word(&mut ctx, font, word, size_px, budget, &mut out);
if let Some(last) = out.pop() {
line = last.text;
line_w = last.width_px;
}
} else {
line.push_str(word);
line_w = word_w;
}
}
}
if !line.is_empty() || out.is_empty() {
out.push(Segment { text: line, width_px: line_w });
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn single_short_line_one_segment() {
let segs = wrap_segments("hello world", 0, 15.0, 10_000.0);
assert_eq!(segs.len(), 1);
assert_eq!(segs[0].text, "hello world");
assert!(segs[0].width_px > 0.0);
}
#[test]
fn empty_input_yields_one_empty_segment() {
let segs = wrap_segments(" ", 0, 15.0, 100.0);
assert_eq!(segs.len(), 1);
assert_eq!(segs[0].text, "");
assert_eq!(segs[0].width_px, 0.0);
}
#[test]
fn narrow_budget_wraps_to_multiple_segments() {
// A budget too small for the whole phrase must split it.
let segs = wrap_segments("alpha beta gamma delta", 0, 15.0, 40.0);
assert!(segs.len() >= 2, "expected a wrap, got {} segs", segs.len());
// Every emitted segment must be non-empty (no dangling blank rows).
for seg in &segs {
assert!(!seg.text.is_empty());
}
}
#[test]
fn over_long_word_is_hard_broken() {
// One unbreakable token wider than the budget breaks per character.
let segs = wrap_segments("aaaaaaaaaaaaaaaaaaaa", 0, 15.0, 20.0);
assert!(segs.len() >= 2, "expected hard break, got {}", segs.len());
}
}
+167 -3
View File
@@ -47,12 +47,13 @@ use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::Arc;
use std::time::Duration;
use slint::{ComponentHandle, Timer, TimerMode};
use slint::{ComponentHandle, Model, ModelRc, SharedString, Timer, TimerMode, VecModel};
use qbz_app::shell::AppRuntime;
use crate::adapter::SlintAdapter;
use crate::{AppWindow, LyricsState, ShellState};
use crate::lyrics_measure;
use crate::{AppWindow, LyricsSegment, LyricsState, ShellState};
type Runtime = Arc<AppRuntime<SlintAdapter>>;
@@ -96,6 +97,33 @@ thread_local! {
static FAST: Cell<bool> = const { Cell::new(false) };
static CTX: RefCell<Option<(Runtime, slint::Weak<AppWindow>)>> =
const { RefCell::new(None) };
/// Last inputs the active-line segmentation was computed for, so the
/// (potentially font-shaping) recompute only runs on a real change —
/// active-line index, content-width budget, font family, size tier, or the
/// uppercase transform — never every 30Hz tick. `None` forces a recompute.
/// Separate cells per surface (the main window vs the miniplayer have
/// different content widths, hence different segmentations).
static SEG_KEY_MAIN: RefCell<Option<SegKey>> = const { RefCell::new(None) };
static SEG_KEY_MINI: RefCell<Option<SegKey>> = const { RefCell::new(None) };
}
/// Which surface's segmentation cache to consult.
#[derive(Clone, Copy)]
pub(crate) enum SegSurface {
Main,
Mini,
}
/// Cache key for the active-line segmentation (see `SEG_KEY`).
#[derive(PartialEq, Clone)]
struct SegKey {
index: i32,
/// Content width in px, quantized to whole px so sub-pixel jitter from the
/// layout doesn't thrash the recompute.
width_px: i32,
font_index: i32,
size_index: i32,
uppercase: bool,
}
/// Start the engine timer. UI-thread only (slint::Timer requirement);
@@ -178,7 +206,16 @@ fn compute_and_push(window: &AppWindow, runtime: &Runtime, require_playing: bool
let (position_ms, playing) = resolve_position_ms(runtime);
if require_playing && !playing {
// Pause freezes the ladder + fill in place (Tauri parity: the rAF
// gate stops, getCurrentTime holds — review §8 "Pause").
// gate stops, getCurrentTime holds — review §8 "Pause"). Still refresh
// the segmentation from the last-pushed active index so a resize or a
// font/size pref change WHILE PAUSED re-flows the wrapped highlight
// (cache-guarded — a no-op when nothing relevant changed).
refresh_active_segments(
&lyrics,
lyrics.get_active_index(),
main_render_prefs(&lyrics),
SegSurface::Main,
);
return false;
}
@@ -217,6 +254,10 @@ fn compute_and_push(window: &AppWindow, runtime: &Runtime, require_playing: bool
lyrics.set_line_progress(fraction);
}
// Per-visual-line karaoke segmentation: recompute (cache-guarded) when the
// active line, content-width, font, size tier, or uppercase pref changes.
refresh_active_segments(&lyrics, index, main_render_prefs(&lyrics), SegSurface::Main);
// REQ-1 fan-out: mirror the karaoke scalars to the miniplayer (30Hz; no-op
// when the mini is closed) so the mini lyrics highlight stays smooth.
crate::miniplayer::mirror_lyrics_scalars(window);
@@ -233,3 +274,126 @@ fn now_ms() -> u64 {
.map(|d| d.as_millis() as u64)
.unwrap_or(0)
}
/// Render prefs for the MAIN sidebar surface — driven by the persisted prefs
/// on `LyricsState`, exactly as `LyricsSidebar.slint` binds them into
/// `LyricsLinesView`.
fn main_render_prefs(lyrics: &LyricsState) -> RenderPrefs {
RenderPrefs {
font_index: lyrics.get_font_index(),
size_index: lyrics.get_size_index(),
uppercase: lyrics.get_uppercase(),
}
}
/// Active-line font size in px (`size-active`), replicating the size-tier ->
/// px mapping in `LyricsSidebar.slint` (0 S = 13 · 1 M = 15 · 2 L = 19 ·
/// 3 XL = 24). Used as the ppem for the segmentation measure pass so segment
/// widths match the drawn active line.
fn size_active_px(size_index: i32) -> f32 {
match size_index {
0 => 13.0,
2 => 19.0,
3 => 24.0,
_ => 15.0,
}
}
/// The font + size tier + uppercase the active line is RENDERED with — the
/// segmentation must measure against these so segment widths match the draw.
/// The main sidebar drives them from the persisted prefs on `LyricsState`; the
/// miniplayer renders with FIXED values (Inter / 15px / no uppercase — it
/// binds none of the prefs, see `MiniLyricsSurface.slint`), so it passes those.
#[derive(Clone, Copy)]
pub(crate) struct RenderPrefs {
pub font_index: i32,
pub size_index: i32,
pub uppercase: bool,
}
/// Recompute (cache-guarded) the per-visual-line segmentation of the ACTIVE
/// line and push it onto `lyrics.active-segments`. Cheap no-op unless the
/// active index, content-width, or render prefs changed since the last push.
/// When there is no active line, clears the model. `surface` selects the
/// per-window cache cell (main window vs miniplayer).
pub(crate) fn refresh_active_segments(
lyrics: &LyricsState,
index: i32,
prefs: RenderPrefs,
surface: SegSurface,
) {
let content_width = lyrics.get_content_width();
let font_index = prefs.font_index;
let size_index = prefs.size_index;
let uppercase = prefs.uppercase;
let key = SegKey {
index,
// Quantize to whole px so layout sub-pixel jitter doesn't re-segment.
width_px: content_width.round() as i32,
font_index,
size_index,
uppercase,
};
let set_key = |k: Option<SegKey>| match surface {
SegSurface::Main => SEG_KEY_MAIN.with(|cell| *cell.borrow_mut() = k),
SegSurface::Mini => SEG_KEY_MINI.with(|cell| *cell.borrow_mut() = k),
};
let unchanged = match surface {
SegSurface::Main => SEG_KEY_MAIN.with(|cell| cell.borrow().as_ref() == Some(&key)),
SegSurface::Mini => SEG_KEY_MINI.with(|cell| cell.borrow().as_ref() == Some(&key)),
};
if unchanged {
return;
}
// No active line, no measurable width, or no doc -> clear the model.
if index < 0 || content_width <= 0.0 {
if lyrics.get_active_segments().row_count() > 0 {
lyrics.set_active_segments(ModelRc::new(VecModel::<LyricsSegment>::default()));
}
set_key(Some(key));
return;
}
// Pull the raw active-line text from the parsed doc (the UI model carries
// the same text, but the doc read is already the engine's source of truth).
let raw = crate::lyrics::with_current_doc(|doc| match doc {
Some(doc) if doc.synced => doc
.lines
.get(index as usize)
.map(|line| line.text.clone()),
_ => None,
});
let Some(raw) = raw else {
lyrics.set_active_segments(ModelRc::new(VecModel::<LyricsSegment>::default()));
set_key(Some(key));
return;
};
// Match the rendered transform: the active line is uppercased when the pref
// is on, so segment widths must be measured on the uppercased string.
let text = if uppercase { raw.to_uppercase() } else { raw };
let size_px = size_active_px(size_index);
let segments = lyrics_measure::wrap_segments(&text, font_index, size_px, content_width);
// Convert measured widths into cumulative start-weight + per-segment
// width-weight (Tauri's getSegmentProgress partition) so the row can
// re-map the single global `line-progress` to a per-segment local fill.
let total: f32 = segments.iter().map(|seg| seg.width_px).sum();
let mut cumulative = 0.0_f32;
let mut out: Vec<LyricsSegment> = Vec::with_capacity(segments.len());
for seg in &segments {
let width_weight = if total > 0.0 { seg.width_px / total } else { 0.0 };
out.push(LyricsSegment {
text: SharedString::from(seg.text.as_str()),
start_weight: cumulative,
width_weight,
});
cumulative += width_weight;
}
lyrics.set_active_segments(ModelRc::new(VecModel::from(out)));
set_key(Some(key));
}
+1
View File
@@ -71,6 +71,7 @@ mod local_library;
mod local_playlist;
mod local_library_settings;
mod lyrics;
mod lyrics_measure;
mod lyrics_prefs;
mod lyrics_sync;
mod media_controls;
+20
View File
@@ -464,6 +464,25 @@ pub fn mirror_lyrics_scalars(main: &AppWindow) {
d.set_active_index(s.get_active_index());
d.set_line_progress(s.get_line_progress());
d.set_fill_anim_ms(s.get_fill_anim_ms());
refresh_mini_segments(&d);
}
/// Recompute the mini's per-visual-line karaoke segmentation against ITS OWN
/// content-width (the mini surface has a different width + padding than the
/// sidebar). The mini renders with FIXED render prefs — Inter / 15px / no
/// uppercase (`MiniLyricsSurface.slint` binds none of the display prefs) — so
/// it passes those rather than the persisted sidebar prefs. Cache-guarded.
fn refresh_mini_segments(d: &LyricsState) {
crate::lyrics_sync::refresh_active_segments(
d,
d.get_active_index(),
crate::lyrics_sync::RenderPrefs {
font_index: 0, // System / Inter (mini default font-name "")
size_index: 1, // 15px (mini's fixed size-active)
uppercase: false,
},
crate::lyrics_sync::SegSurface::Mini,
);
}
fn force_full_mirror(main: &AppWindow, mini: &MiniPlayerWindow) {
@@ -533,6 +552,7 @@ fn mirror_lyrics_gated(main: &AppWindow, mini: &MiniPlayerWindow) {
d.set_active_index(s.get_active_index());
d.set_line_progress(s.get_line_progress());
d.set_fill_anim_ms(s.get_fill_anim_ms());
refresh_mini_segments(&d);
// Gate the lines-model copy on (status, line-count) so we don't thrash the
// lyrics view (which would reset its scroll) every tick.