JorenandClaude Opus 5
da4add57fe
Two related changes: a 68-track Qobuz album was gated by the API throttle
rather than the network, and Qobuz lyrics are now supported, which is what the
faster album path had to be designed around.
--- perf ---
ratelimit: token bucket instead of fixed spacing
The limiter spaced every request exactly interval apart with no burst, so at
the default 60 requests/minute it allowed exactly one request per second no
matter how many workers ran. Measured: 8 requests took 7s with 1 worker and
7s with 6, i.e. max_connections had no effect on anything touching the API.
Since a track needs its metadata and stream URL before downloading, workers
could only start a track every two seconds, which is why an album looked
serial even at 57 MiB/s per track.
New() now takes a burst (providers pass max_connections), idle time accrues
budget up to that burst, and burst is clamped to one minute's worth of
requests so a large max_connections cannot bypass a small configured rate.
app: reuse the track objects already in the album payload
ripAlbum fetched the album, kept only the track ids, then asked the provider
for every track again, doubling the request count. Providers opt in via
AlbumTracksAreComplete(); only Qobuz does, because it is only verified there
and because Tidal does extra work in its track lookup.
Verified against the live Qobuz API on a 68-track release: the embedded track
objects already carry title, version, track_number, media_number,
parental_warning, isrc, copyright, performer, performers, composer, duration,
audio_info, maximum_bit_depth, maximum_sampling_rate and
release_date_original. The only field a per-track lookup adds that this app
reads is the nested album object, so albumTrackMetadata attaches the parent
album with its own track list stripped, or a 68-track album would hold 68
copies of itself. trackMetaUsable refetches anyway when title or track number
is missing, so a thin payload cannot produce a wrongly named file.
Measured end-to-end through the real app and limiter at the shipped defaults,
68 tracks: 137 API calls / 2m11s before, 69 calls / 1m3s after.
--- qobuz lyrics ---
Qobuz's main API does not serve lyrics: has_lyrics is only a flag, extra=lyrics
adds nothing, and track/getLyrics does not exist. Its apps use a two-step flow
instead, confirmed live:
GET track/lyricsUrl?track_id=&app_id=&request_ts=&request_sig=
-> {"track_id","album_id","lyrics_url"} (signed CloudFront URL)
GET <lyrics_url>
-> {"original":{"type","lang","lines":[{"line","start","end"}]},
"translation_langs":[...],"writers","publishers":[...]}
type is plain, lsync or wsync; start/end are milliseconds; the line text is
under "line". Passing a language returns a translation instead of the original,
and only for languages that exist for the track (en 403s on an English song),
so the language parameter is deliberately omitted. A 403 on the signed URL is
how Qobuz represents "no lyrics", so it is not treated as an error.
Lyrics are exposed through a TrackLyrics method rather than inside GetMetadata,
because album rips skip the per-track metadata call. That keeps the fast path:
with lyrics on, an album costs the same two provider calls per track it always
did, and the lyrics document itself comes from a CDN that does not consume the
provider's request budget. With metadata.exclude = ["lyrics"] it drops to one
call per track and the full speedup applies.
Existing behaviour is preserved: Tidal and Deezer already attach lyrics in
their metadata, so attachLyrics leaves them alone, and a lyrics failure is
logged and ignored rather than failing a download.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5.9 KiB
streamrip-go
A Go port of streamrip: a command-line music downloader for Qobuz, Tidal, Deezer, SoundCloud, Yandex Music and Beatport. Downloads tracks, albums, playlists, artist discographies and labels, tags them, optionally embeds cover art and converts the result.
You need your own paid account for any source you use. This tool does not provide or circumvent access to anything.
Requirements
- Go 1.25 or newer (to build)
ffmpegandffprobe— tagging and format conversionyt-dlp— required for SoundCloud only
# Arch
sudo pacman -S ffmpeg yt-dlp
# Debian/Ubuntu
sudo apt install ffmpeg && pipx install yt-dlp
# macOS
brew install ffmpeg yt-dlp
Build
make build # -> ./rip, with version info stamped in
make install # into $(go env GOBIN)
./rip --version
Or without make: go build ./cmd/rip.
Configuration
The first run writes a config file and tells you where. It lives at
$XDG_CONFIG_HOME/streamrip/config.toml (~/Library/Application Support/streamrip/config.toml on macOS).
rip config path # print the path
rip config open # open in $EDITOR
rip config reset # restore defaults
config.toml.example documents every key. The file holds credentials in
plaintext — provider passwords, the Deezer ARL cookie and OAuth tokens — so it
is created 0600. Keep it that way.
Credentials you need to set, per source:
| Source | Required config |
|---|---|
| Qobuz | email_or_userid + password_or_token (md5 of your password), or use_auth_token = true with an auth token |
| Tidal | access_token / refresh_token |
| Deezer | arl (recommended), or email + password |
| Yandex | access_token |
| Beatport | username + password |
| SoundCloud | none; handled via yt-dlp |
Lyrics (plain and synced .lrc-style) are fetched for Qobuz, Tidal and Deezer.
Each costs one extra API request per track, so metadata.exclude = ["lyrics"]
turns them off for a faster rip.
Each provider section also has vendor app_client_id-style keys. Leave them
empty: they default to values built into the binary. They exist so that if a
provider rotates one, you can patch your install with a config edit instead of
waiting for a release.
Usage
rip url https://play.qobuz.com/album/abc123
rip url https://www.beatport.com/release/example/123 --force
rip id qobuz album abc123
rip search qobuz album "kind of blue"
rip search tidal track "teardrop" --first
rip file urls.txt # one URL per line, or a JSON list of IDs
rip lastfm --source qobuz <lastfm-playlist-url>
rip database downloads # what has already been downloaded
rip database failed
Global options (these must come before the command):
-f, --folder <path> Override downloads folder
-q, --quality <0-4> Override provider quality
-c, --codec <codec> Convert after download: ALAC, FLAC, OGG, MP3, AAC
-ndb, --no-db Ignore the download database
--no-progress Disable progress bars
--no-ssl-verify Disable TLS verification
--config-path <path> Use a custom config file
-v, -vv Verbose logging (-vv also logs HTTP requests)
-V, --version Show version
-h, --help Show help
Run rip help <command> for per-command help.
Quality
-q is a 0–4 scale mapped onto each provider's own ladder. Qobuz and Beatport
have no tier 0, so -q 0 is raised to 1 for those and reported on stderr.
| Source | Range | 0 | Top tier |
|---|---|---|---|
| Qobuz | 1–4 | — | FLAC 24-bit up to 192 kHz |
| Tidal | 0–4 | LOW (HE-AAC) | HI_RES_LOSSLESS |
| Deezer | 0–4 | MP3 128 | FLAC (anything ≥ 2) |
| Yandex | 0–4 | LOW | lossless |
| Beatport | 1–4 | — | FLAC 16/44.1 |
| SoundCloud | 0–4 | ignored | provider-defined |
Each provider saturates at the top of its own ladder, so values above its highest real tier are accepted and behave as that tier.
Download database
Completed downloads are recorded in a SQLite database and skipped next time.
--force / --ignore-db overrides this for one run; -ndb disables it
entirely. Paths are configurable under [database].
If you are upgrading from a build that predates source tracking, the database
is migrated on first open and a copy is kept alongside it as
downloads.db.bak-<timestamp>.
Development
make check # gofmt + vet + tests, what CI runs
make test
make race
make cover
make lint # needs golangci-lint
The test suite runs in about a second and makes no network requests; provider
tests drive httptest servers.
Layout:
cmd/rip CLI entry point, one file per command group
internal/app orchestration: queue, resolve, rip
internal/provider/* one client per source, behind provider.Client
internal/download HTTP downloads, progress bars, Deezer/Yandex decryption
internal/audio/{tag,convert} ffmpeg tagging and conversion
internal/{naming,artwork,store,config,netutil,ratelimit,jsonutil,verbose}
Known rough edges
Two deliberate pieces of technical debt, both documented rather than hidden:
- No shared HTTP layer. The request/decode/retry sequence is reimplemented
in each provider client, and retry on 429/5xx exists in only two of six. A
provider/transportpackage would remove roughly 700 lines. - Metadata is
map[string]any.provider.Clientreturns untyped maps, so each client hand-builds an undeclared "Qobuz-shaped" schema thatcmd/rip/search.gothen unpicks in a long switch. A typed media model is the natural next step;internal/provider/qobuz/model.gois a started-and-parked attempt at it.
License
No license file is present, so this is "all rights reserved" by default. Add one before distributing. Note that upstream streamrip is GPL-3.0.