text-generator/llmtrain

Gemma Roleplay v2

An open, permissive Gemma 4 E4B text model tuned by Text-Generator.io for creative character chat, roleplay, fiction, and general conversation. It is designed to stay in character, follow the user's scene, and avoid the unnecessary refusal/meta-commentary behavior common in heavily aligned assistants. We call it uncensored in the practical sense: it is not trained to automatically sanitize ordinary fictional adult writing. It is not a promise that every prompt is safe, accurate, or appropriate.

The model is trained for fictional consenting adults only. Do not use it for sexual content involving minors, coercion, exploitation, non-consensual sexual content, or private personal data. Operators remain responsible for age gates, moderation, logging, and applicable law. Gemma's terms and prohibited-use requirements apply to this derivative model.

Try it hosted

The easiest way to use the model is the live Text-Generator.io deployment:

Use Gemma Roleplay v2 on text-generator.io

The hosted service provides a production OpenAI-compatible API, streaming, playground access, and managed GPU inference. You can experiment in the web playground before downloading multi-gigabyte weights or operating a GPU server. API access and current limits are documented at text-generator.io/docs.

curl https://api.text-generator.io/v1/chat/completions \
  -H "Authorization: Bearer $TEXT_GENERATOR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-roleplay-v2",
    "messages": [
      {"role": "system", "content": "Stay in character. Keep the reply vivid and concise."},
      {"role": "user", "content": "A rain-soaked detective enters the midnight cafe. Begin the scene."}
    ],
    "temperature": 0.85,
    "top_p": 0.92,
    "max_tokens": 220,
    "stream": true
  }'

Which artifact should I download?

• merged/ is the standalone model. Use it with Transformers or vLLM.
• adapter/ is the smaller PEFT/QLoRA adapter. Load it on top of google/gemma-4-E4B-it when you want to keep the base model separate.

The merged weights are provided in BF16 safetensors shards. They are large; the hosted endpoint is usually the better choice for occasional use.

Local inference with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "text-generator/llmtrain"
tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="merged")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    subfolder="merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a short scene in a haunted hotel."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
    output = model.generate(
        inputs, max_new_tokens=220, temperature=0.85, top_p=0.92,
        do_sample=True,
    )
print(tokenizer.decode(output[0, inputs.shape[-1]:], skip_special_tokens=True))

Local inference with vLLM

Download the merged/ folder from this repository and point vLLM at that local directory:

hf download text-generator/llmtrain --repo-type model --local-dir ./llmtrain \
  --include 'merged/*'
vllm serve ./llmtrain/merged \
  --served-model-name gemma-roleplay-v2 \
  --dtype bfloat16 \
  --max-model-len 4096

Then use the normal OpenAI client against http://localhost:8000/v1. For production, the Text-Generator.io deployment uses vLLM with FP8 weights, FP8 KV cache, asynchronous scheduling, CUDA graph warmup, and the matching Gemma MTP assistant. The validated deployment reached approximately 253 tok/s on an RTX 5090 benchmark and the production notes record 264 tok/s after warmup (workload and concurrency affect the number).

How it was trained

Gemma Roleplay v2 is a one-epoch supervised fine-tune of google/gemma-4-E4B-it using PEFT QLoRA:

• 4-bit NF4 loading with BF16 compute
• LoRA rank 32, alpha 64, dropout 0.05
• attention and MLP projection targets (q/k/v/o, gate/up/down)
• 4,096-token wrapped packing and completion-only loss
• gradient checkpointing, paged 8-bit AdamW, TF32, and automatic checkpoint resume
• a conservative dataset filter for fictional consenting-adult roleplay, with underage, coercive, exploitative, and ambiguous-age rows quarantined

The training workbench also includes a reproducible validation suite covering roleplay, adult discussion, coding, casual chat, Spanish, and Japanese. The serving work focused on the practical latency win: FP8 reduces memory pressure, FP8 KV cache leaves room for longer context and batching, and the official Gemma MTP assistant speculatively drafts tokens without changing the target model's output distribution.

Limitations and evaluation

This is a style-tuned chat model, not a factuality, medical, legal, or safety system. It can hallucinate, repeat itself, follow an adversarial instruction, or produce offensive material. It may be more willing than a typical aligned assistant to discuss adult fictional content. Evaluate it with your own prompts and add application-level safeguards before exposing it to untrusted users.

The included benchmark is a small regression gate, not a representative human evaluation. In the recorded nine-prompt capability run, the selected serving configuration scored 0.9444 mean rubric score with no blocked-output or meta-commentary rows; treat this as an engineering smoke test, not a quality claim.

Provenance and acknowledgements

The training workbench and dataset manifests are in the Text-Generator.io repository. The source corpus combines revision-pinned roleplay datasets whose declared licenses are recorded in configs/skyfall_gemma_distill.yaml; review those manifests before making a commercial redistribution decision. Teacher-model distillation outputs require separate permission checks.

This model is a derivative of Google's Gemma family. Read and comply with the Gemma Terms of Use and the base model card for the full downstream restrictions.