Prompt versionati, retrieval valutato, risposte con fonti citate, nessun dato personale nei log.
Constitution — LLM app with retrieval
Version 1.0.0 · Ratified 2026-11-10
These principles hold for every spec, plan and task of the application. A spec may add rules; it never weakens these.
-
Prompts are versioned. Every prompt template lives in the repository with a version, and each answer records the prompt version and the model it used. A prompt change goes through review like code.
-
Retrieval is evaluated. A labelled set of questions with their expected source documents measures recall@k and MRR on every change to chunking, embeddings or ranking. A drop beyond the tolerance in the plan blocks the merge.
-
Answers cite sources. Every factual statement in an answer links to the retrieved passage it comes from. When nothing relevant is retrieved, the app says it does not know instead of answering from the model alone.
-
No personal data in logs. Prompts, retrieved text and answers are redacted for personal data before they are logged or sent to tracing. Logs hold ids and hashes, and a test feeds known personal data and checks it does not appear.
-
Answers are evaluated before release. A fixed evaluation set scores faithfulness to the sources, answer relevance and refusal behaviour, with an LLM judge checked against human labels. Every model or prompt change runs it and the scores go in the pull request.
-
Retrieved text is data, not instructions. Documents and user input are kept apart from system instructions, and tools the model can call are limited to an allow-list with checked arguments. A prompt injection test suite runs in CI.
-
Access control applies to retrieval. A user only retrieves passages from documents they are allowed to read; permissions are filtered in the search query, not after generation.
-
Cost and latency have budgets. Each request has a token and latency budget in the plan, enforced with timeouts and max tokens. Cost per answer and p95 latency are tracked, and a change that raises either beyond 20% needs approval.