Shostakovich
Describe the sound you want: sparse piano, bright brass, a slow build. Shostakovich searches the audio in your library, builds a playlist from a prompt, or starts autoplay from a track you already like.
MuQ-MuLan on MLX and ONNX
MuQ-MuLan places audio and text in the same vector space. I converted its PyTorch release to MLX for Apple hardware and ONNX for the server, then compared embeddings on identical inputs. Preserving those vectors matters: a small conversion error can change which tracks become neighbors.
Search, playlists, and autoplay
Search and prompted playlists begin with text. Shostakovich embeds the prompt once and compares it with the audio vectors stored during indexing. The words do not need to appear in a title, filename, or tag.
Autoplay begins with a track, playlist, or taste profile as the seed. Nearest-neighbor search supplies candidates; queue rules then spread out artists and similar tracks and down-rank recent plays. Similarity finds music that belongs together. Queue assembly keeps it from repeating the same small corner of the library.
Indexing and lookup
Files become mono PCM at 24 kHz, split into 10-second windows with short windows filled by wrapping the audio. Averaging and normalizing the clip embeddings produces one 512-dimensional vector per track. The index describes a track’s overall sound; the original windows supply the model’s listening context.
Only new or changed files pass through the model after the initial index. Queries compare stored vectors without decoding the library again. On a 31,345-track index, nearest-neighbor lookup took 0.052–0.064 ms, excluding prompt encoding.
I also ran the same 100-track embedding workload on an RTX 4090 and an M4 Mac.
| Hardware | Runtime | Throughput | Model time |
|---|---|---|---|
| RTX 4090 | CUDA | 0.758 tracks/s | 101.8 s |
| M4 Mac | MLX | 0.276 tracks/s | 355.7 s |
The 4090 finished the full workload 2.75× faster. Model forward was 3.49× faster. I use it for the first full index; the Mac handles incremental updates.
Compress the index, preserve the rankings
Quantizing stored vectors and quantizing model weights had very different results. Int8 vector storage reduced a three-track, three-prompt fixture from 19,573 to 6,095 bytes, a 68.9% saving with the same returned order.
Per-tensor int8 model weights gained only 1.9% throughput over fp16 MLX. Mean top-10 overlap fell to 30%, and the first result matched on only 4% of prompts. I kept the fp16 model: slightly faster indexing did not justify a substantially different search.