Post
6599
πΆ Released mule-torch β an unofficial PyTorch port of MULE (SF-NFNet-F0), SiriusXM/Pandora's music-audio embedding model (McCallum et al., ISMIR 2022).
No retraining: I re-implemented the architecture in pure PyTorch and transferred the original TensorFlow weights, then checked it layer by layer against the genuine TF pipeline.
β End-to-end clip-embedding cosine 0.9999999 vs the original
β ONNX backbone parity < 1e-6
β 62.35M params (paper: ~62.4M)
β Batched, GPU-native, ONNX-exportable β none of which the original
π€ Weights: matteospanio/mule
π» Code: https://github.com/matteospanio/mule-torch
π¦ PyPI: https://pypi.org/project/mule-torch/
The fun bug: parity was perfect through every conv but the block output was anti-correlated (cos = β1). Cause: the learnable skip-init gains couldn't be mapped by layer name (Keras scrambles the order) β they had to be recovered from the graph.
β οΈ Unofficial, community port β not affiliated with or endorsed by the original authors. All credit to them; please cite the paper. Weights inherit CC-BY-NC-4.0.
No retraining: I re-implemented the architecture in pure PyTorch and transferred the original TensorFlow weights, then checked it layer by layer against the genuine TF pipeline.
β End-to-end clip-embedding cosine 0.9999999 vs the original
β ONNX backbone parity < 1e-6
β 62.35M params (paper: ~62.4M)
β Batched, GPU-native, ONNX-exportable β none of which the original
Analysis pipeline doespip install mule-torchfrom mule_torch import MuleModel
emb = MuleModel.from_pretrained()(waveform) # (B, T)@16kHz -> (B, 1728)π€ Weights: matteospanio/mule
π» Code: https://github.com/matteospanio/mule-torch
π¦ PyPI: https://pypi.org/project/mule-torch/
The fun bug: parity was perfect through every conv but the block output was anti-correlated (cos = β1). Cause: the learnable skip-init gains couldn't be mapped by layer name (Keras scrambles the order) β they had to be recovered from the graph.
β οΈ Unofficial, community port β not affiliated with or endorsed by the original authors. All credit to them; please cite the paper. Weights inherit CC-BY-NC-4.0.