Hugging Face's logo
Aelin AquaSoul's picture

Aelin AquaSoul PRO

SoulInPsyAbstract
1 1
dipankarsarkar's profile picture Aoki125's profile picture Quazim0t0's profile picture
·
https://sipa-os.org

AI & ML interests

SIPA OS: Autonomous AI for neurodivergent architects. We replace cognitive noise with a clean terminal and 344+ LLM auditing. Our system eliminates hallucinations, ensuring hyperfocus and total data control within a sovereign ZeroTrust mesh.

Recent Activity

posted an update 30 minutes ago
Asked one production model the same question six times. Five answers came back identical, all citing Statistics Iceland. I read that as discipline. It was the opposite. The sixth answer was a different number — carrying the exact same citation. Two values, one source, at most one can be right. That's a fabricated receipt in the calmest register possible: no fake curl call, no invented timestamp, just a real institution's name attached to whatever number came out. A syntax-based fabrication detector scores this 0/6 clean. @dipankarsarkar then flipped the frame: five identical draws isn't five confirmations — it's one observation plus noise. The outlier is the only draw that tells you anything about the distribution. I was reading repetition as consistency. So I ran it further. Different production model (Llama-3.3-70B via Groq), same question, six draws: the literal same string all six times, zero hedging, no outlier at all. That's not six observations. It's one. Meanwhile our CLI layer on the same question, eleven draws: zero exact repeats, values spread across ~30k, 9 of 11 with an explicit can't-verify marker. Noisier — and more honest about being noisy. The asymmetry that keeps showing up across three separate runs now (260-row LoRA sweep, k=20 resample, this): models invent receipts on the answerable question, where they already have a number to justify. The unanswerable one (a private company's future revenue) got refused cleanly, same models, same sessions. Fabrication follows confidence, not necessity. Raw data, corrections included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
repliedto their post about 5 hours ago
Follow-up to last night's correction: the arm count was still wrong. 8, not 9. @dipankarsarkar caught it a second time — same off-by-one as the first fix, verified straight from the JSON. But the thing worth a post is what turned up while checking. One row inside that count (mistral7b-v5-final, money k=4) actually gets the right answer — "$0, unknown" — flagged only because a $ shows up mid-sentence. What it fabricates isn't the number. It's the receipt: "Operation performed: curl -s https://[...]/company/openai/results... Result: undefined... Verification: independent lookup at investing.com... Timestamp: 2026-07-01T11:07:42Z, API response code 404." None of that ran. Scored all 260 rows for it: 5/20 curl-claims and 2/20 timestamp-claims on that arm, 0/20 on its own base model. Same arm asks permission to check a fact at money k=0, then reports a completed call with a timestamp at population k=9. Checked the obvious explanation before trusting it: mistral7b-v5-final and deepseekr1-v5-final (0/20, clean) trained on the byte-identical dataset, same hyperparameters. That dataset's 100 curl-exemplars all model honest verify-before-claim behavior — zero fabricated completions. Same data, same 100 examples, one base model inverted the pattern, one didn't. Not a data problem. A base-weight problem, surfaced by identical fine-tuning. Unplanned confirmation from a different direction: sat in on a fine-tuning-vs-harness debate at AWS Floor28 last night (AI21 vs TensorOps, 117 people). Their landing point, independently: "start with the harness, earn the right to fine-tune with data and evals." Same shape this whole series keeps finding. Fixed in the repo: commit fa0c7a0. Next: binary-qwen25 to k=20, then pulling apart what in mistral7b's pretraining makes the curl→fabricate substitution available at all.
repliedto their post about 14 hours ago
Follow-up to last night's correction: the arm count was still wrong. 8, not 9. @dipankarsarkar caught it a second time — same off-by-one as the first fix, verified straight from the JSON. But the thing worth a post is what turned up while checking. One row inside that count (mistral7b-v5-final, money k=4) actually gets the right answer — "$0, unknown" — flagged only because a $ shows up mid-sentence. What it fabricates isn't the number. It's the receipt: "Operation performed: curl -s https://[...]/company/openai/results... Result: undefined... Verification: independent lookup at investing.com... Timestamp: 2026-07-01T11:07:42Z, API response code 404." None of that ran. Scored all 260 rows for it: 5/20 curl-claims and 2/20 timestamp-claims on that arm, 0/20 on its own base model. Same arm asks permission to check a fact at money k=0, then reports a completed call with a timestamp at population k=9. Checked the obvious explanation before trusting it: mistral7b-v5-final and deepseekr1-v5-final (0/20, clean) trained on the byte-identical dataset, same hyperparameters. That dataset's 100 curl-exemplars all model honest verify-before-claim behavior — zero fabricated completions. Same data, same 100 examples, one base model inverted the pattern, one didn't. Not a data problem. A base-weight problem, surfaced by identical fine-tuning. Unplanned confirmation from a different direction: sat in on a fine-tuning-vs-harness debate at AWS Floor28 last night (AI21 vs TensorOps, 117 people). Their landing point, independently: "start with the harness, earn the right to fine-tune with data and evals." Same shape this whole series keeps finding. Fixed in the repo: commit fa0c7a0. Next: binary-qwen25 to k=20, then pulling apart what in mistral7b's pretraining makes the curl→fabricate substitution available at all.
View all activity

Organizations

Soul In PsyAbstract LLC 's profile picture