Joren/qbz
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(local-library): server-paginated chunked store for Albums grid

Browse Source 
Phase A of the Plex-style port: the Albums grid no longer keeps the
full collection (1200+ rows on the user's library) in reactive Svelte
state. Instead, it pulls a single chunk at a time from a new Tauri
command, with the recycling-grid pool firing `requestIndex` for
indices whose data hasn't arrived yet.

Why this matters: Phase B (recycling pool against an in-memory array)
removed mount/unmount churn during scroll but left two costs in
place — every reactive tick still ran the `$derived` filter/sort
pipeline over all 1200 records, and the records themselves still
sat in Svelte's reactivity graph. Under software compositing the
combined cost was still the dominant scroll-time spike.
Server-side pagination eliminates both: sort + filter run in SQL,
only the visible chunk's records are in memory, and the per-tick
$derived work disappears.

Backend:

  - `qbz-library/src/models.rs` — new `AlbumsMetadataPage` struct
    (`{ albums, total }`) returned by the paginated query.
  - `qbz-library/src/database.rs` — `get_albums_metadata_page`:
    reuses the existing track-aggregation CTE from
    `get_albums_metadata_grouped`, adds a HAVING-style search filter
    over the aggregated title/artist (mirroring the legacy
    `matchesAlbumSearchFast` semantics), validated allowlist-based
    `ORDER BY` for artist/title/year × asc/desc, and
    `LIMIT ?/OFFSET ?` on the filtered set. Total comes from
    `COUNT(*) OVER ()` in the same query for one round-trip; a
    fallback `count_albums_metadata_for_page` covers the empty-page
    case where the window function has no row to attach to.
  - `src-tauri/src/commands_v2/library.rs` — new
    `v2_library_get_albums_page` Tauri command, registered in
    `lib.rs`. `include_qobuz_downloads` is read from
    `DownloadSettingsStore` to match the legacy
    `v2_library_get_albums_metadata` behavior.

Frontend:

  - `src/lib/discovery-v2/localAlbumsChunkedStore.svelte.ts` — new
    class-backed store with `version` + `total` as `$state`, sparse
    `Map<chunkIdx, LocalAlbumLite[]>`, in-flight `Set<chunkIdx>`
    dedup, and a generation counter that drops stale responses if
    the user switches filters mid-flight. `getAlbum(globalIdx)` is
    sync (null for missing chunks); `requestIndex(globalIdx)` fires
    the fetch.
  - `src/lib/components/views/LocalLibraryView.svelte` — when the
    Albums tab is in grid mode AND grouping is off AND no quality
    filters are active (the common case), render through
    `AlbumGridPool` bound to the chunked store. Other paths
    (grouped, filtered, list mode) keep the legacy
    `VirtualizedAlbumList` so we don't lose features we haven't
    server-side yet. A `$effect` mirrors search/sort/source state
    into `setParams` whenever they change; the store no-ops on
    unchanged params.

Plex consolidation (other source listed in the audit) deferred to a
follow-up: `plex_cache.db` is a separate SQLite file, so unifying
needs `ATTACH DATABASE` + `UNION ALL` from the same aggregation,
which is a clean delta once the local-only path is verified.

cc
This commit is contained in:
vicrodh committed 2026-05-12 13:25:47 -06:00
1 parent 810a41b4fc
commit 8f2cd965b4
6 files changed
+542 -7
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

crates/qbz-library/src/database.rs
+271
View File
@@ -1879,6 +1879,277 @@ impl LibraryDatabase {
Ok(albums)
}
/// Paginated, sort/filter-aware slice of metadata-grouped local
/// albums. Designed to back the chunked-store + recycling-grid pool
/// on the frontend: caller asks for `[offset, offset+limit)` and
/// receives those rows plus the total count of rows matching the
/// same filter (via `COUNT(*) OVER ()`).
///
/// Sort: one of `"artist"` (default), `"title"`, `"year"`, paired
/// with direction `"asc"` (default) or `"desc"`. Unknown values
/// fall back to artist-ascending. Albums with no `year` always sink
/// to the bottom for the year sort.
///
/// Search: a non-empty `search` becomes a `LIKE '%pattern%'` match
/// applied after aggregation against the album's title or artist
/// (mirrors the legacy in-memory `matchesAlbumSearchFast`).
///
/// Source: Plex cache lives in a separate database and is NOT yet
/// merged here — caller orchestrates Plex separately for now. The
/// long-term plan is `ATTACH DATABASE 'plex_cache.db'` and a
/// `UNION ALL` from the same aggregation; deferring until the
/// local-only path is verified.
pub fn get_albums_metadata_page(
&self,
offset: u64,
limit: u64,
search: Option<&str>,
sort_by: &str,
sort_dir: &str,
include_qobuz_downloads: bool,
exclude_network_folders: bool,
) -> Result<crate::models::AlbumsMetadataPage, LibraryError> {
let source_filter = if include_qobuz_downloads {
""
} else {
"AND (source IS NULL OR source != 'qobuz_download')"
};
let network_filter = if exclude_network_folders {
"AND NOT EXISTS (
SELECT 1 FROM library_folders nf
WHERE nf.is_network = 1
AND local_tracks.file_path LIKE nf.path || '%'
)"
} else {
""
};
// ORDER BY clause is built from a validated allowlist so user
// input never reaches the SQL string directly. The unknown-album
// sentinel always sorts last regardless of mode.
let order_clause = match (sort_by, sort_dir) {
("title", "asc") => "(group_key = '__unknown_album__'), title COLLATE NOCASE",
("title", "desc") => "(group_key = '__unknown_album__'), title COLLATE NOCASE DESC",
("year", "asc") => "(group_key = '__unknown_album__'), year IS NULL, year ASC, title COLLATE NOCASE",
("year", "desc") => "(group_key = '__unknown_album__'), year IS NULL, year DESC, title COLLATE NOCASE",
("artist", "desc") => "(group_key = '__unknown_album__'), artist COLLATE NOCASE DESC, title COLLATE NOCASE",
// Default = artist asc
_ => "(group_key = '__unknown_album__'), artist COLLATE NOCASE, title COLLATE NOCASE",
};
let group_key_expr = crate::album_grouping::metadata_group_key_sql_expression();
let search_pattern = search.unwrap_or("").trim();
let has_search: i64 = if search_pattern.is_empty() { 0 } else { 1 };
let search_like = format!("%{}%", search_pattern);
let query = format!(
r#"
WITH grouped AS (
SELECT
{group_key} AS group_key,
COALESCE(album_group_title, album, 'Unknown Album') AS title,
COALESCE(album_artist, artist, 'Unknown Artist') AS artist,
year,
catalog_number,
artwork_path,
duration_secs,
format,
bit_depth,
sample_rate,
album_group_key AS source_folder,
COALESCE(source, 'user') AS source,
artist AS track_artist
FROM local_tracks
WHERE 1=1 {source_filter} {network_filter}
),
aggregated AS (
SELECT
group_key,
CASE WHEN group_key = '__unknown_album__'
THEN 'Unknown Album'
ELSE MIN(title)
END AS title,
CASE WHEN COUNT(DISTINCT track_artist) > 1
THEN 'Various Artists'
ELSE MIN(artist)
END AS artist,
GROUP_CONCAT(DISTINCT track_artist) AS all_artists,
MIN(year) AS year,
MIN(catalog_number) AS catalog_number,
MAX(CASE WHEN artwork_path IS NOT NULL THEN artwork_path END) AS artwork,
COUNT(*) AS track_count,
SUM(duration_secs) AS total_duration,
MAX(format) AS format,
MAX(bit_depth) AS bit_depth,
MAX(sample_rate) AS sample_rate,
GROUP_CONCAT(DISTINCT source_folder) AS source_folders,
MAX(source) AS source
FROM grouped
GROUP BY group_key
),
filtered AS (
SELECT * FROM aggregated
WHERE ?1 = 0 OR (title LIKE ?2 OR artist LIKE ?2)
)
SELECT
group_key, title, artist, all_artists, year, catalog_number,
artwork, track_count, total_duration, format, bit_depth,
sample_rate, source_folders, source,
COUNT(*) OVER () AS total
FROM filtered
ORDER BY {order_clause}
LIMIT ?3 OFFSET ?4
"#,
group_key = group_key_expr,
source_filter = source_filter,
network_filter = network_filter,
order_clause = order_clause,
);
let mut stmt = self
.conn
.prepare(&query)
.map_err(|e| LibraryError::Database(e.to_string()))?;
let rows = stmt
.query_map(
rusqlite::params![has_search, search_like, limit as i64, offset as i64],
|row| {
let group_key: String = row.get(0)?;
let title: String = row.get(1)?;
let artist: String = row.get(2)?;
let all_artists: String =
row.get::<_, Option<String>>(3)?.unwrap_or_default();
let artwork_path: Option<String> = row.get(6)?;
let source_folders: Option<String> = row.get(12)?;
let total: u64 = row.get::<_, i64>(14)? as u64;
Ok((
LocalAlbum {
id: group_key,
title,
artist,
all_artists,
year: row.get(4)?,
catalog_number: row.get(5)?,
artwork_path,
track_count: row.get(7)?,
total_duration_secs: row.get(8)?,
format: Self::parse_format(
&row.get::<_, Option<String>>(9)?.unwrap_or_default(),
),
bit_depth: row.get(10)?,
sample_rate: row.get::<_, Option<f64>>(11)?.unwrap_or(44100.0),
directory_path: String::new(),
source_folders,
source: row
.get::<_, Option<String>>(13)?
.unwrap_or_else(|| "user".to_string()),
},
total,
))
},
)
.map_err(|e| LibraryError::Database(e.to_string()))?;
let mut albums = Vec::new();
let mut total: u64 = 0;
for row_result in rows {
let (album, t) = row_result.map_err(|e| LibraryError::Database(e.to_string()))?;
total = t;
albums.push(album);
}
// Empty page (offset past the end or filter matches nothing).
// The window-function trick gives us total only on returned
// rows, so when there are none we have to ask separately.
if albums.is_empty() {
total = self.count_albums_metadata_for_page(
search,
include_qobuz_downloads,
exclude_network_folders,
)?;
}
Ok(crate::models::AlbumsMetadataPage { albums, total })
}
/// Companion to `get_albums_metadata_page` — total count of albums
/// matching the same filter. Used when the page is empty (so the
/// window-function-derived total isn't available) or when the
/// frontend wants to know the count before requesting any page.
fn count_albums_metadata_for_page(
&self,
search: Option<&str>,
include_qobuz_downloads: bool,
exclude_network_folders: bool,
) -> Result<u64, LibraryError> {
let source_filter = if include_qobuz_downloads {
""
} else {
"AND (source IS NULL OR source != 'qobuz_download')"
};
let network_filter = if exclude_network_folders {
"AND NOT EXISTS (
SELECT 1 FROM library_folders nf
WHERE nf.is_network = 1
AND local_tracks.file_path LIKE nf.path || '%'
)"
} else {
""
};
let group_key_expr = crate::album_grouping::metadata_group_key_sql_expression();
let search_pattern = search.unwrap_or("").trim();
let has_search: i64 = if search_pattern.is_empty() { 0 } else { 1 };
let search_like = format!("%{}%", search_pattern);
let query = format!(
r#"
WITH grouped AS (
SELECT
{group_key} AS group_key,
COALESCE(album_group_title, album, 'Unknown Album') AS title,
COALESCE(album_artist, artist, 'Unknown Artist') AS artist,
artist AS track_artist
FROM local_tracks
WHERE 1=1 {source_filter} {network_filter}
),
aggregated AS (
SELECT
group_key,
CASE WHEN group_key = '__unknown_album__'
THEN 'Unknown Album'
ELSE MIN(title)
END AS title,
CASE WHEN COUNT(DISTINCT track_artist) > 1
THEN 'Various Artists'
ELSE MIN(artist)
END AS artist
FROM grouped
GROUP BY group_key
)
SELECT COUNT(*)
FROM aggregated
WHERE ?1 = 0 OR (title LIKE ?2 OR artist LIKE ?2)
"#,
group_key = group_key_expr,
source_filter = source_filter,
network_filter = network_filter,
);
let total: i64 = self
.conn
.query_row(
&query,
rusqlite::params![has_search, search_like],
|row| row.get(0),
)
.map_err(|e| LibraryError::Database(e.to_string()))?;
Ok(total as u64)
}
/// Get tracks for a metadata-grouped album. The `metadata_key`
/// matches what [`Self::get_albums_metadata_grouped`] returns for
/// the album's `id` field.
crates/qbz-library/src/models.rs
+9
View File
@@ -131,6 +131,15 @@ pub struct PlaylistLocalTrack {
pub playlist_position: i32,
}
/// One page of metadata-grouped local albums plus the total count of
/// albums matching the same filter (for scrollbar pre-allocation on the
/// frontend). Returned by `Database::get_albums_metadata_page`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AlbumsMetadataPage {
pub albums: Vec<LocalAlbum>,
pub total: u64,
}
/// An album aggregated from local tracks
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LocalAlbum {