HFStack

Infrastructure and integrations for the Hugging Face ecosystem.

HFStack is an open-source organization focused on building reproducible ML infrastructure around the Hugging Face stack.

work on:

• dataset orchestration
• experiment tracking
• artifact pipelines
• benchmarking systems
• runtime tooling
• ecosystem integrations

---

Focus Areas

Datasets & Storage

Building reproducible workflows around Hugging Face Datasets and HF Buckets.

Trackio & Observability

Experiment tracking, artifact lineage, and reproducible evaluation pipelines using Trackio.

Benchmarking & Runtime Systems

Inference benchmarking, optimization workflows, and runtime evaluation tooling.

Orchestration & Integrations

Composable integrations with tools like Dagster and ecosystem-native ML workflows.

---

Philosophy

HFStack focuses on the systems surrounding modern ML:

• reproducibility
• interoperability
• observability
• infrastructure simplicity

The goal is to make Hugging Face workflows easier to build and operationalize.

Projects & Integrations

• dagster-hf-datasets: Dagster-HF-Datasets integrates Hugging Face datasets with Dagster for building reproducible, observable data pipelines. Load datasets directly as Dagster assets, apply transformations, and publish results back to the Hub.

---

Contributions and ecosystem collaborations are welcome.