Resposta curta e em inglês, para ler, fechar e treinar em voz alta.
A resposta, como você fala na entrevista
Repeatable evaluations of the system's behavior on representative and adversarial cases. Define acceptance criteria before tuning, version the dataset, and keep held-out cases to check generalization.
- Reference cases: questions with acceptable facts, evidence, constraints, or rubrics. Many questions have several valid answers, so exact string matching is not always appropriate.
- Retrieval: recall@k is the fraction of relevant items found among the top k. Reciprocal rank is 1 divided by the rank of the first relevant item, or zero if none is found. MRR averages that value across queries.
- Generation: check grounding in supplied evidence, relevance, task correctness, and appropriate abstention. Groundedness alone does not prove that the source is true.
- LLM-as-judge: a model grades against a rubric. Compare its decisions with human labels, test order/style biases, and keep critical deterministic checks outside the judge.
Run appropriate evaluations on prompt, model, retrieval, and tool changes. Add production failures after review, while retaining an independent test set. A finite green dataset is evidence for those cases, not proof of universal correctness.
Ler é o primeiro passo. No app, essa pergunta vira flashcard com repetição espaçada, e o módulo tem quiz e simulado para testar o que ficou.
Treinar essa pergunta no app