Instructions to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF",
	filename="GLM-5.1-Abliterated-Q5_K_M-00001-of-00033.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Use Docker

docker model run hf.co/arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

LM Studio
Jan

vLLM

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Ollama
How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Ollama:
```
ollama run hf.co/arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
```

Unsloth Studio

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF to start chatting

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Docker Model Runner:
```
docker model run hf.co/arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M
```

Lemonade

How to use arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF:Q5_K_M

Run and chat with the model

lemonade run user.GLM-5.1-Abliterated-Q5_K_M-GGUF-Q5_K_M

List all available models

lemonade list

GLM-5.1-Abliterated Q5_K_M GGUF

TL;DR

For agentic / long-horizon / coding workloads where you don't want refusals on technical questions
754B MoE (40B active), abliterated, Q5_K_M, 534 GB / 33 shards
~37 t/s decode for ~$8-10/hr on 8× RTX PRO 6000 Blackwell (vast.ai)
~23 t/s decode for ~$8-10/hr on 8× A100-SXM4 (vast.ai)
Full 200K context tested, with q8_0 KV cache
AIME 2026: 6/6 correct on the first six problems (I did partial run with thinking-on)
8× 80GB or 8× 96GB GPU for the recommended config

GGUF Q5_K_M quantization of helixdouble/GLM-5.1-Abliterated. Runs in mainline llama.cpp, LM Studio, Ollama.

Abliterated means the model was modified to reduce refusal behavior. It removes the directions in the model's activation space that produce refusals, without retraining. The result is a model that declines less often on prompts it would otherwise refuse, while keeping general capability intact. This quant inherits that behavior from upstream — no additional abliteration was applied here.

The model will engage with technical questions that mainstream chat models often over-refuse (security research, defensive tooling, dual-use). However, this model still declines to provide methods on self-harm queries and similar.

Performance

--parallel 1, q8_0 KV cache, flash attention on.

8× RTX PRO 6000 Blackwell 96GB

ctx=202752

Test	Prefill t/s	Decode t/s
256-token gen, short prompt	215	36.80
8k prompt + 128 gen	659	34.05
32k prompt + 128 gen	502	28.57
128k prompt + 128 gen	240	17.91
short prompt + 2k gen	295	36.14

~$8-10/hr on vast.ai. huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated or similar model also fits in VRAM simultaneously - you can use it for subagent tasks!

8× A100-SXM4 80GB

ctx=202752

Test	Prefill t/s	Decode t/s
short prompt, 256 gen	55	23.27
short prompt, 512 gen	59	23.00
short prompt, 1024 gen	57	22.83
4.5k prompt + 128 gen	209	22.43

~$8-10/hr on vast.ai.

2× RTX PRO 6000 Blackwell 96GB

ctx=8192, mostly offloaded to RAM

Run	Output tokens	Prefill t/s	Decode t/s
prompt 60 tokens, 256 gen	256	18.56	5.43
prompt 67 tokens, 512 gen	512	19.65	5.33
prompt 2222 tokens, 128 gen	66	81.48	5.37
prompt 45 tokens, no max_tokens	1976	14.98	5.47

~$3-5/hr on vast.ai.

Full-config numbers above are the recommended hardware target. Two-GPU operation works but the model does not fully fit in VRAM at this configuration.

Quality — AIME 2026

Just to quickly check nothing went wrong, I ran partial evaluation of AIME 2026.

1 attempt per problem, sampling temperature=1.0 top_p=0.95, enable_thinking=true, default matharena evaluator.

Problem	Wall time	Reasoning tokens	Result
1	0m 48s	1,089	✓
2	6m 08s	8,117	✓
3	5m 16s	6,987	✓
4	9m 05s	11,869	✓
5	2m 19s	3,131	✓
6	1m 29s	2,005	✓

6/6 correct on the first 6 problems consecutively.

Quick start

Start llama-server and let it download the GGUF from Hugging Face:

llama-server \
  --hf-repo arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF \
  --hf-file GLM-5.1-Abliterated-Q5_K_M-00001-of-00033.gguf \
  --ctx-size 202752 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0

Open the llama.cpp web UI or connect with an OpenAI-compatible client.
For manual downloads, keep all 33 shards in the same directory and load the first shard.

Prompt Format

GLM-5.1 uses its own chat template with explicit <think> / </think> blocks for the reasoning trace. The full jinja template is embedded in the GGUF.

Token markers:

[gMASK]<sop><|system|>system prompt<|user|>user message<|assistant|><think>reasoning</think>visible answer

Thinking mode is on by default. To disable, pass enable_thinking=false in chat_template_kwargs:

{
  "messages": [{"role": "user", "content": "Hello"}],
  "chat_template_kwargs": {"enable_thinking": false}
}

OpenAI-compatible clients work directly with the /v1/chat/completions endpoint.

Source

This was quantized from helixdouble/GLM-5.1-Abliterated, which is based on zai-org/GLM-5.1-FP8 / zai-org/GLM-5.1. This release changes the storage/runtime format and quantization only.

Quantization Recipe

Direct Q5_K_M from BF16 via llama-quantize. No imatrix calibration, no per-tensor overrides. Reproducible end-to-end from helixdouble/GLM-5.1-Abliterated FP8 source with a single command.

Conversion timings (CPU-only, 128 threads, ~3 TB scratch disk):

HF safetensors -> BF16 GGUF: 55m 14s
BF16 GGUF -> Q5_K_M GGUF: 31m 37s

Notes

Benchmarks above are single-client. I haven't run concurrent-client throughput.
For production deployment add your own guardrails - abliterated != aligned.

Feedback

Open an issue in the Community tab. This is my first quant, feedback is genuinely useful😀

Support

If this quant saved you some vast.ai bucks, a tip helps fund the next one.

Disclaimer

Provided AS IS for research and educational purposes. This model has reduced refusal behavior inherited from upstream - outputs may be inaccurate, biased, unsafe, or that you find may offensive. You are responsible for compliance with applicable laws in your jurisdiction and for any guardrails you add when deploying.

No warranty is given, no liability accepted for downstream use.

License

This GGUF follows the source model licensing. The source model is listed as AGPL-3.0 and also refers users to the upstream GLM-5.1-FP8 license. For redistribution, modification, or hosted use, check the upstream model cards and license files.

Downloads last month: 99

GGUF

Model size

754B params

Architecture

glm-dsa

Hardware compatibility

5-bit

Model tree for arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF

Base model

zai-org/GLM-5.1-FP8

Quantized

helixdouble/GLM-5.1-Abliterated

Quantized

(3)

this model