★ Unidad 24

Usability testing and evaluation

Find out whether an AI product really works for people. Plan evaluations that combine what people do, what the numbers say and how the model behaves; test non-deterministic features without fooling yourself; measure AI UX; build evals, rubrics and LLM judges you can trust; red-team for safety; run fair experiments; and turn findings into decisions. Every lesson has real prompts to run with your own AI key.

🔒 La unidad 24 es para miembros

Las unidades 7 a 25 son para miembros del bootcamp. Inicia sesión si eres estudiante, o postula para unirte.

Iniciar sesiónVolver al curso gratis

1Evaluation strategy: what to measure, and whenTurn “is our AI feature any good?” into a written plan that combines people, product and model evidence, with success criteria agreed before you look at results.
2Usability testing AI featuresPlan and run sessions for features that answer differently every time: realistic tasks, planned failures, a choice between Wizard of Oz and the live model, and a sharp eye for over- and under-trust.
3Benchmarks and UX metrics for AIMeasure usability with standard instruments, scored correctly; choose UX metrics that fit AI features, such as acceptance, edits and regenerations; and report numbers honestly when samples are small.
4Evals for LLM featuresBuild a golden dataset of test cases, decide how each case is graded, and run regression evals so a prompt or model change never quietly breaks what used to work.
5LLM as judge, and human ratingWrite rubrics that people and models can apply the same way, know the biases of LLM judges, and check a judge against human ratings with simple agreement measures.
6Red-teaming and safety testingTest an AI feature the way an adversary, a careless input or an unlucky customer would: jailbreaks, prompt injection, data leaks, unsafe actions and unfair answers, and report what you find responsibly.
7Experiments and A/B testsKnow how a fair A/B test works, choose a primary metric and guardrails, size it before you start, see through novelty effects, and recognise when an experiment is the wrong tool.
8Reporting and acting on findingsRate AI findings by severity and reach, design dashboards people can read at a glance, give a readout that leads with the decision, and turn findings into roadmap items you can check later.
★An evaluation of an AI feature, from plan to readoutProyecto

Usability testing and evaluation

🔒 La unidad 24 es para miembros

Las unidades 7 a 25 son para miembros del bootcamp. Inicia sesión si eres estudiante, o postula para unirte.

Iniciar sesiónVolver al curso gratis