Two related changes: a 68-track Qobuz album was gated by the API throttle
rather than the network, and Qobuz lyrics are now supported, which is what the
faster album path had to be designed around.
--- perf ---
ratelimit: token bucket instead of fixed spacing
The limiter spaced every request exactly interval apart with no burst, so at
the default 60 requests/minute it allowed exactly one request per second no
matter how many workers ran. Measured: 8 requests took 7s with 1 worker and
7s with 6, i.e. max_connections had no effect on anything touching the API.
Since a track needs its metadata and stream URL before downloading, workers
could only start a track every two seconds, which is why an album looked
serial even at 57 MiB/s per track.
New() now takes a burst (providers pass max_connections), idle time accrues
budget up to that burst, and burst is clamped to one minute's worth of
requests so a large max_connections cannot bypass a small configured rate.
app: reuse the track objects already in the album payload
ripAlbum fetched the album, kept only the track ids, then asked the provider
for every track again, doubling the request count. Providers opt in via
AlbumTracksAreComplete(); only Qobuz does, because it is only verified there
and because Tidal does extra work in its track lookup.
Verified against the live Qobuz API on a 68-track release: the embedded track
objects already carry title, version, track_number, media_number,
parental_warning, isrc, copyright, performer, performers, composer, duration,
audio_info, maximum_bit_depth, maximum_sampling_rate and
release_date_original. The only field a per-track lookup adds that this app
reads is the nested album object, so albumTrackMetadata attaches the parent
album with its own track list stripped, or a 68-track album would hold 68
copies of itself. trackMetaUsable refetches anyway when title or track number
is missing, so a thin payload cannot produce a wrongly named file.
Measured end-to-end through the real app and limiter at the shipped defaults,
68 tracks: 137 API calls / 2m11s before, 69 calls / 1m3s after.
--- qobuz lyrics ---
Qobuz's main API does not serve lyrics: has_lyrics is only a flag, extra=lyrics
adds nothing, and track/getLyrics does not exist. Its apps use a two-step flow
instead, confirmed live:
GET track/lyricsUrl?track_id=&app_id=&request_ts=&request_sig=
-> {"track_id","album_id","lyrics_url"} (signed CloudFront URL)
GET <lyrics_url>
-> {"original":{"type","lang","lines":[{"line","start","end"}]},
"translation_langs":[...],"writers","publishers":[...]}
type is plain, lsync or wsync; start/end are milliseconds; the line text is
under "line". Passing a language returns a translation instead of the original,
and only for languages that exist for the track (en 403s on an English song),
so the language parameter is deliberately omitted. A 403 on the signed URL is
how Qobuz represents "no lyrics", so it is not treated as an error.
Lyrics are exposed through a TrackLyrics method rather than inside GetMetadata,
because album rips skip the per-track metadata call. That keeps the fast path:
with lyrics on, an album costs the same two provider calls per track it always
did, and the lyrics document itself comes from a CDN that does not consume the
provider's request budget. With metadata.exclude = ["lyrics"] it drops to one
call per track and the full speedup applies.
Existing behaviour is preserved: Tidal and Deezer already attach lyrics in
their metadata, so attachLyrics leaves them alone, and a lyrics failure is
logged and ignored rather than failing a download.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The provider apps' own identifiers and protocol keys were scattered as
constants across six packages. They are not user secrets and stay baked in so
the tool still works out of the box, but they are now mirrored into the config
so that when a provider rotates one, a user can fix their install with a config
edit instead of waiting for a release.
Moved, each with the current value as the default in internal/config:
qobuz mobile_app_id, mobile_app_secret
tidal app_client_id, app_client_secret
deezer api_key, gateway_key, blowfish_secret
yandex sign_key, legacy_mp3_sign_salt
beatport app_client_id
applyVendorDefaults backfills any empty value, so configs written before these
keys existed keep working and the clients never have to nil-check them. Each
client reads through a small accessor; the package-level signing helpers
(mobileSignRequest, yandexDownloadSign, legacyDirectURL, deriveGatewayAuthToken
and friends, deriveDeezerBlowfishKey) now take the key as a parameter instead
of closing over a constant. The blowfish secret reaches internal/download
through a field on Downloader set by app.New.
The blowfish key derivation XORs 16 bytes of the secret, so a mistyped override
would have panicked; a wrong-length value is now rejected with a warning and
the built-in value is used. Tests pin the length and that the qobuz mobile
secret is valid hex, since the HKDF step requires it.
Also drops soundcloud.client_id and soundcloud.app_version: soundcloud goes
through yt-dlp, which supplies its own, so nothing read them.
config.toml.example documents each new key as "leave empty unless the vendor
rotates theirs".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dead code
Removed, each verified to have no non-test caller: promptSearchSelection
(77 lines, superseded by promptSearchSelectionMenu), AddPlaylistByTrackIDs,
download.NewWithVerifySSL, qobuz.Client.GetTrackMetadata,
tidal.Client.trackSupportsAtmos (test-only, and the comment above
GetDownloadable already documented its removal from the live path) and its
now-orphaned test, yandex.canonicalTrackRequestID (a no-op returning its
trimmed input; the call site uses strings.TrimSpace directly) and the unused
defaultEstimatedKbps constant.
Config keys that did nothing
qobuz.download_booklets and misc.check_for_updates were declared, defaulted
and documented in config.toml.example but read by no code. The whole
[youtube] block went too: there is no YouTube provider. go-toml ignores
unknown keys, verified, so existing user configs still load.
Duplicated helpers
tidal.stringify and yandex.firstNonEmpty were byte-identical copies of
jsonutil.StringFromAny and jsonutil.FirstNonEmpty. displayArtistNames
existed three times, byte-identical, in internal/app, provider/yandex and
provider/beatport; it now lives in jsonutil. The CLI's normalizeCodec
delegates to config.NormalizeCodec so the flag and the config file are
checked by the same rules.
Tests for packages that had none
internal/jsonutil (0% -> 100%) and internal/netutil (0% -> 78%), the latter
covering redactURL, which is a security control, and the zero-timeout
contract that internal/download and the qobuz body copy depend on.
internal/config rose to 82% with coverage of the new Validate(), the
0600 permissions, CloneConfigData not sharing slices, and a concurrent
UpdateTokens test that exercises the mutex under -race.
Test suite is ~85x faster
Provider tests built clients with the default 60 requests/minute, so the
rate limiter slept 1s per HTTP call: deezer took 31s, tidal 11s, qobuz 8s.
Each provider test package now builds its config through a testConfigData
helper that disables throttling. Whole suite: ~60s -> 0.7s, and 2.8s
under -race.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JorenandClaude Opus 5