Qwopus3.5-4B-Coder-Fable5-v1-GGUF

Instructions to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF",
	filename="Qwopus3.5-4B-Coder-Fable5-v1-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Ollama
How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Ollama:
```
ollama run hf.co/AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
```

Unsloth Studio

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF to start chatting

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Docker Model Runner:
```
docker model run hf.co/AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M
```

Lemonade

How to use AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AkshayCoder48/Qwopus3.5-4B-Coder-Fable5-v1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwopus3.5-4B-Coder-Fable5-v1-GGUF-Q4_K_M

List all available models

lemonade list

AkshayCoder48

shuhulx commited on Jun 18

Commit

ce2fb03

0 Parent(s):

Duplicate from shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-GGUF

Browse files

Co-authored-by: Shuhul Razdan <shuhulx@users.noreply.huggingface.co>

Files changed (5) hide show

.gitattributes +38 -0
Qwopus3.5-4B-Coder-Fable5-v1-Q4_K_M.gguf +3 -0
Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf +3 -0
Qwopus3.5-4B-Coder-Fable5-v1-mmproj-BF16.gguf +3 -0
README.md +201 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,38 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwopus3.5-4B-Coder-Fable5-v1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwopus3.5-4B-Coder-Fable5-v1-mmproj-BF16.gguf filter=lfs diff=lfs merge=lfs -text

Qwopus3.5-4B-Coder-Fable5-v1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3d3621cf4d7745b196d7bd6795eb452d777e6ebb910d049ab151339f32d29a00
+size 2783446432

Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f48d43eca68c80ec2455bd482ca4c0efab411402edc4dcee48383585b8fedc8
+size 3161425312

Qwopus3.5-4B-Coder-Fable5-v1-mmproj-BF16.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:263043e99496ce01a6a050defcd3ded5a6d64e4638e5291a1b48926587c1fe50
+size 675568672

README.md ADDED Viewed

	@@ -0,0 +1,201 @@

+---
+license: apache-2.0
+base_model: shuhulx/Qwopus3.5-4B-Coder-Fable5-v1
+datasets:
+- Glint-Research/Fable-5-traces
+language:
+- en
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+- gguf
+- llama-cpp
+- lm-studio
+- qwen3_5
+- fable5
+- reasoning
+- agent
+- tool-use
+- function-calling
+- coder
+- coding
+- debugging
+- local-inference
+- quantized
+- conversational
+---
+<div align="center">
+# 💻 Qwopus3.5-4B-Coder-Fable5-v1 GGUF
+### GGUF builds for llama.cpp, LM Studio, and local inference
+<p><b>Fable-5 traces</b> · <b>agentic coding</b> · <b>tool use</b> · <b>debugging</b></p>
+</div>
+---
+## Overview
+**Qwopus3.5-4B-Coder-Fable5-v1** is a Fable-5 trace continuation of [`Jackrong/Qwopus3.5-4B-Coder`](https://huggingface.co/Jackrong/Qwopus3.5-4B-Coder).
+The base model, Qwopus3.5-4B-Coder, is a compact Qwen3.5-based coding model trained for reasoning, tool use, function calling, coding workflows, and agent-style behavior.
+This release continues that model on [`Glint-Research/Fable-5-traces`](https://huggingface.co/datasets/Glint-Research/Fable-5-traces), a dataset of Claude Fable 5 local coding-agent traces. The dataset is heavily oriented around tool-use trajectories, repository work, local command context, code editing, debugging loops, and `<think>`-style reasoning completions.
+The result is a small local coding-agent model intended for:
+| Area | Description |
+|---|---|
+| Tool-use workflows | Bash, Read, Write, Edit, repo inspection, and action traces. |
+| Debugging | Failing tests, stack traces, root-cause analysis, and patch planning. |
+| Trace-style reasoning | Long-form planning and `<think>` style reasoning traces. |
+| Local agents | Hermes-style, Claude-Code-style, OpenCode-style, and LM Studio workflows. |
+## Files
+Typical GGUF files:
+- `Qwopus3.5-4B-Coder-Fable5-v1-Q4_K_M.gguf`
+- `Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf`
+- `Qwopus3.5-4B-Coder-Fable5-v1-mmproj-BF16.gguf`
+## Which file should I use?
+| File | Use case |
+|---|---|
+| `Q4_K_M` | Best default. Small, fast, good quality. |
+| `Q5_K_M` | Better quality while still compact. |
+| `Q8_0` | Higher quality, larger memory use, if included. |
+| `mmproj-BF16` | Multimodal projector for compatible runtimes. |
+## llama.cpp
+```bash
+llama-cli \
+  -m Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf \
+  -p "Write a Bash/Read/Edit style plan for debugging a failing Python repo." \
+  -n 768 \
+  --temp 0.7 \
+  --top-p 0.95
+```
+## llama.cpp Server
+```bash
+llama-server \
+  -m Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf \
+  --host 0.0.0.0 \
+  --port 8080 \
+  --ctx-size 8192
+```
+Then call it with an OpenAI-compatible client:
+```bash
+curl -X POST "http://localhost:8080/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  --data '{
+    "model": "Qwopus3.5-4B-Coder-Fable5-v1-Q5_K_M.gguf",
+    "messages": [
+      {"role": "user", "content": "Write a tool-use plan for debugging a Python repo."}
+    ],
+    "temperature": 0.7,
+    "top_p": 0.95
+  }'
+```
+## About the Fable-5 Traces
+[`Glint-Research/Fable-5-traces`](https://huggingface.co/datasets/Glint-Research/Fable-5-traces) contains Claude Fable 5 coding traces.
+The dataset includes fields such as:
+```text
+uid
+source_file
+session
+model
+context
+cot
+output_type
+output
+completion
+origin
+```
+The examples are not simple chat pairs. They are multi-step agent trajectories with local development context, reasoning traces, and tool-use outputs.
+Common patterns in the dataset include:
+- user coding requests
+- local-command caveats
+- repository inspection
+- Bash command usage
+- file reads
+- file writes
+- edits
+- debugging passes
+- playtesting / validation loops
+- `<think>...</think>` reasoning traces
+- tool-use completions
+A large portion of the dataset is `tool_use` style data, which makes it especially relevant for local coding agents and developer automation.
+## Capabilities
+### Agentic coding
+Designed for coding-agent loops where the model must inspect a repo, plan work, call tools, edit files, and validate changes.
+### Tool-use style outputs
+Works well with prompts that expose structured tools such as:
+```text
+Bash
+Read
+Write
+Edit
+Search
+Grep
+```
+### Debugging and repair
+Useful for:
+- finding likely failing files
+- explaining stack traces
+- planning test commands
+- proposing minimal patches
+- iterating after errors
+### Local-first deployment
+The release includes Transformers, GGUF, MLX, and MLX 4-bit formats so it can run in Python, llama.cpp, LM Studio, and Apple Silicon workflows.
+## Available Releases
+| Release | Repo | Best for |
+|---|---|---|
+| Transformers / Safetensors | [`shuhulx/Qwopus3.5-4B-Coder-Fable5-v1`](https://huggingface.co/shuhulx/Qwopus3.5-4B-Coder-Fable5-v1) | Python, Transformers, custom inference. |
+| GGUF | [`shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-GGUF`](https://huggingface.co/shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-GGUF) | llama.cpp, LM Studio, local CPU/GPU inference. |
+| MLX | [`shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-MLX`](https://huggingface.co/shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-MLX) | Apple Silicon full MLX inference. |
+| MLX 4-bit | [`shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-MLX-4bit`](https://huggingface.co/shuhulx/Qwopus3.5-4B-Coder-Fable5-v1-MLX-4bit) | Apple Silicon low-memory inference. |
+## Credits
+Built on:
+- [`Jackrong/Qwopus3.5-4B-Coder`](https://huggingface.co/Jackrong/Qwopus3.5-4B-Coder) by Jackrong
+- [`Glint-Research/Fable-5-traces`](https://huggingface.co/datasets/Glint-Research/Fable-5-traces) by Glint-Research
+- Qwen / Qwen3.5 model family
+- Unsloth
+- Hugging Face
+- llama.cpp
+- mlx-lm