🧬 Carbon-VEPor: Efficient Variant Effect Prediction with Carbon

Automating Variant Effect Prediction (VEP)—the task of determining whether a specific genetic mutation is pathogenic (disease-causing) or benign—requires a careful marriage of deep biological sequence modeling and fast, deterministic classification.

By utilizing an autonomous ML-Intern agent with NVIDIA's Nemotron-3-Nano-4B to build our script foundation and pairing it with MiniCPM-V-4.6 and Carbon-3B in a multi-stage production pipeline, we built an end-to-end machine learning system that transitions from raw sequence tokens to a production-ready classification head.

---

Link to space: https://huggingface.co/spaces/build-small-hackathon/carbon-vepor

1. Autonomous Data Engineering & Ingestion Setup (ML-Intern)

The technical heavy lifting of setting up our data engineering pipelines was handled by an autonomous data-sourcing agent session (ML-Intern). Given a description of our task parameters, the agent operated within an isolated workspace sandbox to read the raw data and output our underlying script infrastructure:

+-------------------------------------------------------------------------+
|                          ML-INTERN AGENT WORKSPACE                      |
|                                                                         |
|  1. Parses User Description                                             |
|  2. Streams & Inspects Hugging Face Dataset (viveksil/clinvar-cls)      |
|  3. Resolves Schema Anomalies & Data Splits                             |
|                                                                         |
|         │                                                       │       |
|         ▼ (Code Generation)                                     ▼       |
|  ┌──────────────────────────────┐                ┌───────────────────┐  |
|  │          extract.py          │                │     train.py      │  |
|  │                              │                │                   │  |
|  │ - Carbon GGUF Logit Ingestion│                │ - PyTorch 3-Layer │  |
|  │ - LLR Feature Array Creation │                │   MLP Framework   │  |
|  └──────────────────────────────┘                └───────────────────┘  |
+-------------------------------------------------------------------------+

• Data Profiling: The ML-Intern automatically analyzed and streamed the viveksil/clinvar-cls genomic variant dataset directly from the Hugging Face Hub.
• Automatic Script Generation: Rather than requiring manual development loops, the agent generated the exact code for extract.py (to handle sequence extraction and token math) and train.py (to setup and optimize the PyTorch classification head).
• Self-Correction: If the dataset structures or columns shifted during runtime ingestion, the ML-Intern parsed the error traces and adjusted the target mapping constraints automatically to ensure the generated code compiled without intervention.

---

2. Quantitative Feature Extraction: Log-Likelihood Ratios (LLR)

The feature file written by the ML-Intern (extract.py) isolates a metric called the Log-Likelihood Ratio (LLR) using a local Carbon-3B model via native llama_cpp bindings. The LLR measures the specific statistical disruption caused by a nucleotide substitution variant.

Tokenizer Realignment

To extract token-level log-probabilities from the model, genomic sequences are wrapped in <dna>...</dna> sequence boundaries. This enforces that Carbon's 6-mer tokenizer matches its pre-training tokenization offsets. For a single nucleotide variation fixed at base-index $501501$, the exact mutation coordinate is mapped to its relative token index via a shifting window calculation:

Base Sequence Index:
[0] ... [500] [501] [502] ...
       └─┬───┘   │
      +5 Bases   │ (Target Mutation Site)
         │       ▼
   ┌─────▼──────────────┐
   │ (501 + 5) // 6 = 84│ ──► Maps directly to Token Index: [84]
   └────────────────────┘

$$\text{target\_token\_idx}=\frac{501+5}{6}=84\backslash text\{target\backslash \_token\backslash \_idx\}\; =\; \backslash frac\{501\; +\; 5\}\{6\}\; =\; 84$$

The LLR Formulation

Running a forward pass with the model parameters configured to track all sequence probabilities (logits_all=True) exposes the raw output logit matrices.

The LLR represents the difference between the likelihood of seeing the alternate mutation under the mutated context versus the reference nucleotide under the wild-type context:

$$\text{LLR}=\log p(\text{alt}\mid \text{mutated context})-\log p(\text{ref}\mid \text{wild-type context})\backslash text\{LLR\}\; =\; \backslash log\; p(\backslash text\{alt\}\; \backslash mid\; \backslash text\{mutated\; context\})\; -\; \backslash log\; p(\backslash text\{ref\}\; \backslash mid\; \backslash text\{wild-type\; context\})$$

• Interpretation: A highly negative LLR score signifies that the biological language model finds the mutation highly unnatural or unexpected given the surrounding sequence context, serving as a powerful predictive marker for pathogenicity.
• Feature Vector: The pipeline pairs this LLR_score with an engineered binary flag indicating whether the mutation falls in a functional protein-coding region (coding_flag). The resulting 2D matrix is saved to disk as a serialized PyTorch tensor (data/extracted_llr.pt).

---

3. Optimizing the Neural Decision Boundary (Classifier Head)

The model code written by the ML-Intern (train.py) sets up a downstream Classifier Head neural network to map the engineered LLR features directly to diagnostic labels.

           Input Vector: [ LLR_score , coding_flag ] (Shape: 1x2)
                                │
                                ▼
                   ┌─────────────────────────┐
                   │   Linear Layer (2→32)   │
                   ├─────────────────────────┤
                   │          ReLU           │
                   ├─────────────────────────┤
                   │      Dropout (0.2)      │
                   └────────────┬────────────┘
                                │
                                ▼
                   ┌─────────────────────────┐
                   │  Linear Layer (32→16)   │
                   ├─────────────────────────┤
                   │          ReLU           │
                   └────────────┬────────────┘
                                │
                                ▼
                   ┌─────────────────────────┐
                   │   Linear Layer (16→1)   │
                   └────────────┬────────────┘
                                │
                                ▼
                           Raw Logit

Model Architecture

The architecture consists of a compact 3-layer Multi-Layer Perceptron (MLP) optimized for binary classification:

1. Input Layer: Ingests the 2-dimensional feature pair ([LLR_score, coding_flag]).
2. Hidden Topology: Passes through a Linear(2 → 32) projection, a ReLU activation function, and a Dropout(p=0.2) layer to penalize co-adaptation of features. This is followed by a secondary Linear(32 → 16) projection and another ReLU layer.
3. Output Node: A final Linear(16 → 1) mapping layer yields a raw, unscaled pathogenicity logit.

Optimization Strategy

The training script loads the data engineered by the ML-Intern, enforces a reproducible $80/20$ train/validation split, and optimizes parameters across multiple epochs using the AdamW optimizer $\text{lr}={10}^{-3}\backslash text\{lr\}=10^\{-3\}$, $\text{weight\_decay}={10}^{-4}\backslash text\{weight\backslash \_decay\}=10^\{-4\}$

The model tracks validation loss and ROC-AUC scores to evaluate performance. To preserve mathematical precision during backpropagation, a BCEWithLogitsLoss function is applied—keeping the sigmoid function out of the training loop graph and utilizing it strictly during the inference runtime.

---

4. Multi-Stage Production Inference & NumPy Compilation

The runtime production environment is tied together by a central coordinator (orchestrator.py). This script does not handle data building or training; instead, it coordinates our active model endpoints to execute live, multi-stage pipeline inference on incoming patient reports.

The Inference Lifecycle

       ┌─────────────────────────────────────────────────────────┐
       │             User Uploads Clinical PDF Report            │
       └────────────────────────────┬────────────────────────────┘
                                    │
                                    ▼
┌───────────────────────────────────────────────────────────────────────┐
│ STAGE 1: Document Parsing & Extraction                                │
│ Model: MiniCPM-V (Vision) | Port: 8081                                │
│                                                                       │
│  Input : Raw PDF Document Visual Stream                               │
│  Action: Visual table parsing & structural pattern matching           │
│  Output: Structured JSON Data                                         │
│          {                                                            │
│            "wild_type_sequence": "ATCG...",                           │
│            "mutated_sequence":   "ATGG...",                           │
│            "is_coding":          true                                 │
│          }                                                            │
└────────────────────────────────────┬──────────────────────────────────┘
                                    │
                                    ▼
┌───────────────────────────────────────────────────────────────────────┐
│ STAGE 2: Genomic Language Scoring                                     │
│ Model: Carbon-3B (LLM via llama.cpp GGUF) | Port: 8082                │
│                                                                       │
│  Input : Sequences + Alignment Target (Token Index: 84)               │
│  Action: Extracts raw log-probabilities from token-level logits       │
│  Math  : LLR = log_p(alt | mutated) - log_p(ref | wild-type)          │
│  Output: Compressed Feature Vector -> [ LLR_score , coding_flag ]     │
└────────────────────────────────────┬──────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ STAGE 3: Bare-Metal Classification Boundary                             │
│ Engine: NumPy Head (carbon_backend.py) | Pure CPU Math                  │
│                                                                         │
│  Input : [ LLR_score , coding_flag ]                                    │
│  Action: Runs multi-layer perceptron forward pass using raw weight      │
│          tensors (W1, W2, W3) and biases loaded from PyTorch checkpoint.│
│  Math  : Sub-millisecond matrix dot products + explicit Sigmoid         │
│  Output: Pathogenicity Probability Score (Continuous scalar in [0, 1])  │
└────────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌───────────────────────────────────────────────────────────────────────┐
│ STAGE 4: Clinical Report Synthesis                                    │
│ Model: MiniCPM-V (Text-Gen) | Port: 8081                              │
│                                                                       │
│  Input : Extracted Variant Data + Pathogenicity Probability Score     │
│  Action: Autoregressive prompt engineering pass                       │
│  Output: Finished Corporate Markdown Diagnostic Report                │
└────────────────────────────────────┬──────────────────────────────────┘
                                    │
                                    ▼
       ┌─────────────────────────────────────────────────────────┐
       │       Polished Clinical Report Rendered on UI Dashboard │
       └─────────────────────────────────────────────────────────┘

• Stage 1: Multimodal Ingestion: A vision-language model acts as a structural parser, taking unstructured PDF inputs and translating layout matrices into explicit string segments (wild_type_sequence, mutated_sequence) alongside the is_coding parameter.
• Stage 2: Carbon-3B LLR Computation: The orchestrator dispatches these extracted sequences to a local Carbon-3B framework running via low-latency llama.cpp HTTP endpoints. The model isolates target token index $8484$ and computes the mathematical LLR score.
• Stage 3: Bare-Metal NumPy Classification: To bypass the latency, overhead, and compute footprint of importing heavy deep learning frameworks (like PyTorch) on production web servers, the trained classifier head is re-compiled into pure matrix mathematics (carbon_backend.py).

During initialization, the production engine loads the raw weight tensors $(W_1,W_2,W_3)(W\_1,\; W\_2,\; W\_3)$ and bias vectors $(b_1,b_2,b_3(b\_1,\; b\_2,\; b\_3$) directly from the serialized checkpoint file (classifier_head.pt). The forward pass is then executed entirely through pure NumPy matrix dot-products and manual element-wise mathematical functions:

$$z_1=W_1\cdot x+b_1⟹a_1=\max (0,z_1)z\_1\; =\; W\_1\; \backslash cdot\; x\; +\; b\_1\; \backslash implies\; a\_1\; =\; \backslash max(0,\; z\_1)$$

$$z_2=W_2\cdot a_1+b_2⟹a_2=\max (0,z_2)z\_2\; =\; W\_2\; \backslash cdot\; a\_1\; +\; b\_2\; \backslash implies\; a\_2\; =\; \backslash max(0,\; z\_2)$$

$$\text{logit}=W_3\cdot a_2+b_3⟹\text{pathogenicity\_probability}=\frac{1}{1+e^{-\text{logit}}}\backslash text\{logit\}\; =\; W\_3\; \backslash cdot\; a\_2\; +\; b\_3\; \backslash implies\; \backslash text\{pathogenicity\backslash \_probability\}\; =\; \backslash frac\{1\}\{1\; +\; e^\{-\backslash text\{logit\}\}\}$$

By shifting live production inference from heavy deep learning computation graphs to explicit CPU math operations, the final classification step runs faster, providing an instant binary probability mapping inside the dashboard.