thealper2/lfm2-700m-linux-command

LiquidAI/LFM2-700M fine-tuned to map a natural-language Linux task to a single shell command. The target output is the command only; no explanation is produced.

Model details

Field	Value
Base model	LiquidAI/LFM2-700M
Architecture	Lfm2ForCausalLM — hybrid, 16 layers: full attention at [2, 5, 8, 10, 12, 14], gated short convolution elsewhere
Parameters	742,489,344 total (641,826,048 non-embedding)
Hidden size / heads / KV heads	1536 / 24 / 8
Vocabulary	65,536
Context length (base)	128,000
Training precision	bfloat16
Fine-tuning method	full
Task	NL instruction → shell command

Prompt format

The model uses the LFM2 ChatML-style chat template and was trained with no system prompt. Apply the template rather than constructing the string manually.

<|startoftext|><|im_start|>user
Find which process is using port 8080.<|im_end|>
<|im_start|>assistant
lsof -i :8080<|im_end|>

Two LFM2 tokenizer details matter:

• The chat template emits bos_token itself and add_bos_token is true in tokenizer_config.json. Tokenise templated text with add_special_tokens=False, or the sequence gets a duplicated BOS.
• Decode with clean_up_tokenization_spaces=False; the BPE cleanup step strips spaces around punctuation and can corrupt shell commands.

Special tokens: BOS <|startoftext|> (1), EOS <|im_end|> (7), PAD <|pad|> (0).

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "thealper2/lfm2-700m-linux-command"
tokenizer = AutoTokenizer.from_pretrained(model_id, clean_up_tokenization_spaces=False)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to("cuda").eval()

messages = [{"role": "user", "content": "Find which process is using port 8080."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=128,
    do_sample=False,                      # deterministic decoding
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
)
command = tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()
print(command)  # lsof -i :8080

Greedy decoding (do_sample=False) is the intended configuration: the task has a single intended answer and sampling only adds variance.

Training data

Source	Raw rows	Schema
jiacheng-ye/nl2bash	9,305	nl, bash
mecha-org/linux-command-dataset	8,669	input, output

Both were normalised to {instruction, command, source} and then:

1. Cleaned — markdown fences and $/# prompt prefixes stripped, whitespace runs collapsed outside quoted strings, records with unbalanced quotes, prose instead of a command, or no parseable utility dropped. Commands themselves were never rewritten.
2. Deduplicated — exact (instruction, command) duplicates removed. Rows sharing a command with a different instruction, or an instruction with a different valid command, were kept deliberately.
3. Balanced — find was 35.2% of the raw corpus. It was capped per-utility using a diversity-aware ordering that retains every distinct flag signature before retaining any repeat, lowering its share to ~19%.
4. Split — 90/5/5, grouped by instruction template (filenames, paths, numbers and quoted literals abstracted) so templated paraphrases cannot straddle the train/test boundary.

Split sizes: 11756 train / 652 validation / 653 test. Leakage checks report zero overlap across splits at the exact-pair, instruction and instruction-template level.

Training configuration

Hyper-parameter	Value
Method	full
Epochs	3
Learning rate	3e-05
Scheduler	cosine
Warmup ratio	0.03
Weight decay	0.01
Optimizer	adamw_torch_fused
Per-device batch size	16
Gradient accumulation	2
Max sequence length	128
Precision	bfloat16
Seed	42

Loss is computed on the assistant turn only; prompt tokens are masked with -100.

max_length was chosen from the tokenised length distribution of the corpus (mean 37.7, median 34, p90 57, p95 66, p99 87, max 403) — a 128-token budget covers 99.94% of examples.

Run record

Field	Value
Final training loss	0.516
Validation loss	0.6445
Training time	425.0 s
Peak VRAM	8.95 GB
GPU	NVIDIA GeForce RTX 5060 Ti
torch / transformers	2.11.0+cu128 / 5.17.0

Evaluation

Measured on the held-out test set with greedy decoding.

Metric	Base LFM2-700M	Fine-tuned	Delta
Exact match	0.0061	0.2910	+0.2849
Normalised exact match	0.0061	0.2910	+0.2849
Structural match	0.0107	0.3032	+0.2925
Command validity	0.4763	0.9939	+0.5176
Token F1	0.1195	0.6470	+0.5275
Primary-utility accuracy	0.1807	0.8377	+0.6570
Prose-output rate	0.3783	0.0000	-0.3783

Metric definitions:

• Exact match — string equality after stripping surrounding whitespace.
• Normalised exact match — equality after collapsing whitespace runs outside quotes and removing a trailing ;.
• Structural match — utility, flag multiset (short-flag bundles expanded for utilities that use them) and operand sequence compared per pipeline segment. Recognises ls -la == ls -al.
• Command validity — the output parses under a bash grammar parser (bashlex), not membership in a list of known utilities.
• Token F1 — token-level overlap, as partial credit.
• Primary-utility accuracy — the first utility matches the reference.
• Prose-output rate — fraction of outputs that read as an explanation rather than a command.

Limitations

• Structural match is not a semantic oracle. It compares command shape. It cannot tell that find . -name '*.py' and a shell glob achieve the same result, and it does not reason about flag semantics. It is an upper bound on exact match, not semantic accuracy.
• Exact match understates correctness. Many Linux tasks have several valid answers; the test set carries one reference each.
• Source-distribution bias. nl2bash is find-heavy and composition-heavy; mecha-org/linux-command-dataset is templated and single-utility-heavy. Per-source metrics differ and are reported separately in the project reports.
• Distribution shift. Commands reference paths, hosts and variables that appear in the training corpora (/path/to/..., $source). Outputs may embed those placeholders instead of the user's real paths.
• Short outputs only. Trained at a 128-token budget; long multi-stage scripts are out of distribution.
• No verification of correctness or safety at generation time. The model can produce syntactically valid but wrong — or destructive — commands.

Intended use and safety

Intended for generating candidate shell commands for review, and as the command generator of a sandboxed terminal agent.

Do not execute generated commands directly on a host. The project this model comes from executes commands only inside a disposable Docker container started with --network none, --read-only, --cap-drop ALL, --security-opt no-new-privileges, a non-root user, no host bind mounts, bounded CPU/memory/PIDs and a wall-clock timeout, and screens commands against a destructive-pattern list and a read-only allowlist before running them.

License

Inherits the LFM Open License v1.0 of the base model, LiquidAI/LFM2-700M. Dataset licenses apply to the training data: mecha-org/linux-command-dataset is Apache-2.0; nl2bash derives from the TellinaTool/nl2bash corpus.

Citation

The NL2Bash corpus:

@inproceedings{LinWZE2018:NL2Bash,
  author    = {Xi Victoria Lin and Chenglong Wang and Luke Zettlemoyer and Michael D. Ernst},
  title     = {NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System},
  booktitle = {LREC 2018},
  year      = {2018}
}