Add project agent contract

AGENTS.md

@@ -0,0 +1,142 @@
+# Project: GLM Visual Variable Runtime
+
+## Mission
+
+Build an OpenAI-compatible multimodal gateway called:
+
+    glm-5.2-visual-runtime
+
+The gateway must make the text-only GLM-5.2 model appear to support persistent image input across chat turns, OCR and document understanding, object-level color and style extraction, chart-to-data extraction, visual questions, presentation generation based on visual references, and references such as "this image", "those colors", "that chart", and composed multi-image/property references.
+
+This project must not train, fine-tune, or modify GLM-5.2. The image is not converted into a permanent caption or inserted as visual tokens. Instead, every uploaded image becomes a persistent visual variable with lazily generated typed views.
+
+Example:
+
+    img_01.original
+    img_01.thumbnail
+    img_01.ocr
+    img_01.objects
+    img_01.palette
+    img_01.presentation_theme
+    img_01.chart
+    img_01.chart.data_table
+
+GLM-5.2 receives only the compact structured evidence needed for the current request.
+
+## Non-negotiable requirements
+
+1. No GPU training.
+2. No modification of GLM-5.2 weights.
+3. The public API must support OpenAI-compatible request formats.
+4. The original image must be stored and remain retrievable.
+5. Expensive visual analysis must run lazily, only when requested.
+6. Derived visual views must be cached by image hash and lens version.
+7. Later turns must work without uploading the image again.
+8. Every extracted value must retain provenance: asset ID, bounding box or source region, extraction method, confidence, and lens version.
+9. Do not send an entire OCR transcript to GLM unless the task requires it.
+10. Do not let GLM estimate chart numbers when they can be calculated from extracted structured data.
+11. Do not regenerate exact OCR strings when they can be copied from evidence.
+12. The gateway must support cloud mode with no local GPU.
+13. All model providers must be replaceable through interfaces.
+14. Build API and integration tests before adding a user interface.
+
+## Architecture
+
+Implement this processing flow:
+
+    OpenAI-compatible request
+              |
+              v
+    Request normalizer
+              |
+              v
+    Conversation and visual asset ledger
+              |
+              v
+    Visual reference resolver
+              |
+              v
+    Intent and visual-program compiler
+              |
+              v
+    Lazy lens executor
+      |       |       |       |
+      OCR   Object  Palette  Chart
+              |
+              v
+    Compact evidence compiler
+              |
+              v
+           GLM-5.2
+              |
+              v
+    Text, JSON, table, image, or PPTX artifact
+
+The public model ID is `glm-5.2-visual-runtime`. The internal reasoning model defaults to `glm-5.2`.
+
+## Repository structure
+
+Use the monorepo layout requested by the project brief:
+
+    glm-vvr/
+      AGENTS.md
+      README.md
+      docker-compose.yml
+      .env.example
+      apps/api/
+      services/slides/
+      tests/
+      docs/
+
+## Public API
+
+Implement:
+
+    POST /v1/responses
+    POST /v1/chat/completions
+    POST /v1/files
+    GET  /v1/files/{file_id}
+    GET  /v1/files/{file_id}/content
+    GET  /health
+    GET  /ready
+
+`/v1/responses` is canonical. Support `input_image.image_url`, Base64 data URLs, `input_image.file_id`, chat completion `image_url` content parts, multiple images in one user message, and `previous_response_id`.
+
+When a URL image is supplied, download and snapshot it immediately. Validate MIME type, file size, image dimensions, decoded file signature, redirect limits, and private-network or localhost URLs to prevent SSRF.
+
+## Runtime rules
+
+- On image ingest, store the original image, create an `img_NN` asset, calculate SHA-256, generate a thumbnail and cheap visual fingerprint, and set the asset as active. Do not run OCR, object detection, or chart extraction until needed.
+- Resolve references deterministically first. If confidence is below threshold, call GLM-5.2 with structured output and validate the result.
+- Compile user requests into validated visual programs before executing lenses.
+- Use compact task-specific visual evidence for GLM-5.2. Include exact values, units, uncertainty, and source region IDs. Never claim visual evidence contains absent information.
+
+## Lenses
+
+Implement replaceable lenses for OCR, object grounding, palette extraction, chart extraction, and presentation theme derivation. Cache views by:
+
+    sha256(asset bytes) + lens name + lens version + canonicalized parameters
+
+OCR should preserve exact Unicode, bounding boxes, reading order, tables, Markdown, and structured JSON. Chart extraction should produce reversible ChartIR and answer numeric chart questions from structured data, not model prose. Palette extraction should produce area, attention, and semantic palettes plus role-based design tokens with provenance and contrast checks.
+
+## Providers
+
+Use interfaces for GLM-5.2, GLM-5V-Turbo, GLM-OCR, and fake deterministic providers. The MVP must default to API providers in cloud mode and fake providers in test mode. Do not self-host GLM-5.2 in the MVP.
+
+## Presentation generation
+
+Use a separate TypeScript PptxGenJS service. GLM-5.2 produces a validated `PresentationSpec`; the renderer creates editable `.pptx` files with slide masters, extracted design tokens, editable charts/tables, source images where appropriate, speaker notes containing provenance when enabled, and low-confidence chart warnings.
+
+## Modes
+
+- `cloud`: GLM-5.2 API, GLM-5V-Turbo API, GLM-OCR API, CPU image processing, no local GPU.
+- `hybrid`: GLM-5.2 API, local or hosted OCR, optional local segmentation.
+- `test`: deterministic fake providers, no external API calls.
+
+## Acceptance tests
+
+The system must demonstrate persistent image references, palette-to-PPTX generation, object-region color extraction, exact OCR evidence, ChartIR-based chart answers, multi-image references, URL snapshot durability, duplicate hash view reuse, lazy lens execution, SSRF rejection, no-key test runs, and OpenAI Python client compatibility with only `base_url` changed.
+
+## Definition of done
+
+The project is complete when it runs with Docker Compose, OpenAI Python client can call `/v1/responses`, images can be supplied by URL/Base64/file ID, later text-only turns can reference prior images, OCR/palette/object/chart lenses work through replaceable providers, ChartIR is emitted, editable PPTX files are generated, provenance and confidence are included, tests pass, no training code exists, default cloud deployment needs no local GPU, and docs explain how to add a visual lens without modifying GLM-5.2.