Generative AI / Basic: generation and evidence foundations / Lesson 1
Models, generation, and system boundaries
Separate learned generation from application policy, factual verification, and permission to act.
On this page
Lesson goals
- Explain the assumptions and trust boundaries of models, generation, and system boundaries.
- Interpret both examples and distinguish offline fixture results from model or production evidence.
- Diagnose the listed failures and justify the worked decisions while preserving task, data, permission, and uncertainty contracts.
Define the task before the model
Generative AI produces candidate content under a learned model and supplied conditions. Text, image, audio, and other generators can use different architectures and objectives. This course focuses on text-oriented systems and their application boundaries rather than claiming one mechanism describes every generator. Start with a user task, accepted inputs, required evidence, prohibited effects, and a way to decide whether the result is useful. A fluent paragraph is not itself proof of correctness. A model component can help an application without being the component that decides identity, grants permission, commits a transaction, or certifies a factual claim.
Understand the autoregressive text example
Many language generators predict a distribution over the next token conditioned on available context and then select a continuation through a decoding procedure. Repeating this process builds an output sequence. Training objectives, post-training, model architecture, and decoding influence behavior, but next-token likelihood is not the same as external truth. The first offline example applies temperature to two invented logits. It demonstrates how a distribution can become flatter without executing a neural network. It contains no real tokenizer, trained weights, benchmark result, or claim about a specific hosted model. Other generation families need their own explanatory models.
import math
def probabilities(logits, temperature):
scaled = [value / temperature for value in logits]
maximum = max(scaled)
weights = [math.exp(value - maximum) for value in scaled]
return [round(value / sum(weights), 3) for value in weights]
print(probabilities([2.0, 1.0], 1.0))
print(probabilities([2.0, 1.0], 2.0))
[0.731, 0.269]
[0.622, 0.378]The invented two-choice distribution becomes flatter at temperature two. These logits are hand-authored and no actual model, sampling run, or quality comparison is performed.
Separate training, inference, and application state
Training adjusts model parameters from data under an objective; inference uses a selected model to compute outputs. Supplying a document in one request does not automatically retrain the model or guarantee permanent memory. An application may separately store history, retrieval indexes, caches, and user records. Those stores have privacy, retention, authorization, and deletion responsibilities independent of the model. Record which state is persistent and who owns it. Do not assume a conversational interface remembers everything, forgets everything, or keeps every input confidential without a verified system and provider contract. No external model or storage is used here.
Build a pipeline with explicit trust boundaries
The second example is vendor-neutral pseudocode separating authorization, data selection, generation, validation, and approved response. It is deliberately non-runnable because no real generator is supplied. A generated instruction or tool proposal remains untrusted input to a deterministic policy layer. A model should not turn retrieved text into new authority merely because the text sounds confident. Keep user identity and permissions outside the prompt as enforceable application state. Validate outputs before they reach templates, database queries, tools, or other consumers that can cause effects. Schema validity is useful but cannot establish factual support or authorization by itself.
trusted_actor = authenticate_by_application_policy(request)
allowed_evidence = retrieve_authorized_records(trusted_actor, request)
candidate = generate_candidate(request, allowed_evidence)
validated = validate_shape_and_supported_claims(candidate)
return release_under_application_policy(trusted_actor, validated)
The named operations are conceptual boundaries, not an SDK or executable program. In particular, generation does not authenticate the actor or authorize a tool action.
State what the evidence actually establishes
Offline arithmetic can verify a formula, parser, budget, or control-flow rule. It cannot demonstrate a model’s accuracy, safety, semantic quality, latency, or cost in production. Every synthetic output in this course is labeled as a fixture or deterministic computation rather than a sampled model response. No API key, model download, paid invocation, network service, deployment, or load experiment is required. Use primary research to understand an idea in its tested setting and current official documentation when adopting a particular runtime. A system decision needs task-specific evidence and maintained controls, not a broad claim that generative technology is reliable or unreliable for every use.
Common errors and fixes
Fluent output is treated as verified truth
- Why it happens
- Language likelihood replaced external evidence.
- How to fix it
- Check task facts and provenance independently.
- Prevent it next time
- Define correctness criteria before selecting generation.
A request document is called permanent model training
- Why it happens
- Context, parameters, and application storage were conflated.
- How to fix it
- Document the actual state and retention paths.
- Prevent it next time
- Review training and inference responsibilities separately.
A generated proposal bypasses permission checks
- Why it happens
- Text was promoted into trusted authority.
- How to fix it
- Apply deterministic authorization before any effect.
- Prevent it next time
- Keep actor identity and tool permissions outside model discretion.
Check your understanding
1. A model returns a confident explanation of an unseen company policy. What has been established?
Show worked solution
Only a candidate explanation exists. Confidence in phrasing does not establish access to the current policy or correctness. Obtain authorized evidence, verify the relevant clauses, and allow an uncertainty or abstention result when support is missing.
2. Which layer should decide whether a generated payment proposal may execute?
Show worked solution
The application’s trusted authorization and transaction policy must decide using verified actor, resource, amount, and approval state. The model can propose structured data but cannot grant itself authority. Validate and record the decision before any real side effect.
Key takeaways
- Generative AI produces candidate content under a learned model and supplied conditions.
- Training adjusts model parameters from data under an objective; inference uses a selected model to compute outputs.
- Offline arithmetic can verify a formula, parser, budget, or control-flow rule.