almernzh/Ornith-1.0-35B-BNB-NF4

Ornith-1.0-35B BNB NF4

This model card provides a bitsandbytes NF4 checkpoint of deepreinforce-ai/Ornith-1.0-35B.

The goal is practical Transformers usage with lower memory requirements. It is suited for local experiments, notebooks, adapter workflows, and generation tests on high-memory NVIDIA hardware.

Format

• Quantization: bitsandbytes 4-bit
• Quant type: NF4
• Compute dtype: bfloat16
• Nested quantization: enabled
• Base model: deepreinforce-ai/Ornith-1.0-35B

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "almernzh/Ornith-1.0-35B-BNB-NF4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "user",
        "content": "Give a concise plan to debug a failing Python unit test.",
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=200,
        temperature=0.6,
        top_p=0.95,
    )

print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Reasoning Behavior

Ornith-1.0-35B may emit a <think>...</think> block before the final answer. This follows the behavior of the base model.

Notes

This model targets Transformers and bitsandbytes. Runtime support may vary across serving engines. For vLLM, SGLang, or TensorRT-LLM, use a format that the target runtime explicitly supports.