Testing, Validation Criteria and Copilot Prompt Quality
This topic covers test process and metrics, validation criteria for custom AI models, Copilot prompt best practices, end-to-end testing across Dynamics 365 apps, and test-case strategy using Copilot.
How to study for Microsoft AB-100
Treat each item as an architecture decision: identify the business process, select the Microsoft AI pattern, then add ALM, telemetry, security, responsible AI, and governance controls.
Core concepts
Concept 1
Agent testing should include defined test cases, metrics, expected outcomes, edge cases, failure modes, and regression checks.
Exam cue: Use validation criteria when custom model readiness must be judged.
Concept 2
Custom AI models require validation criteria tied to business purpose, data quality, performance, safety, and acceptance thresholds.
Exam cue: Use end-to-end scenarios when multiple Dynamics 365 apps or agent actions are involved.
Concept 3
End-to-end tests should cover multi-app business flows, data movement, prompts, actions, integrations, and user roles.
Exam cue: Use prompt best practices when bad outputs are caused by unclear instructions or missing context.
Risk pitfalls and guardrails
Testing only the model and not the full business solution.
Guardrail: Avoid defaulting to custom agents, skipping grounding-data checks, or treating prompts and connectors as outside the release process.
Approving a custom model without explicit acceptance criteria.
Guardrail: Avoid defaulting to custom agents, skipping grounding-data checks, or treating prompts and connectors as outside the release process.
Using Copilot-generated test cases without human review or coverage analysis.
Guardrail: Avoid defaulting to custom agents, skipping grounding-data checks, or treating prompts and connectors as outside the release process.
Memory anchors
Agent Test Case
An agent test case defines input, expected behavior, metrics, and pass or fail criteria.
Validation Criteria
Validation criteria define what a model or AI solution must prove before deployment.
Prompt Best Practice
Prompt best practice improves clarity, context, constraints, examples, and safety of prompts.
End-to-End Test
An end-to-end test validates a complete business flow across apps, data, agents, and users.
Regression Test
A regression test checks that changes did not break previously working behavior.
Acceptance Threshold
An acceptance threshold defines the minimum acceptable metric or behavior.
Test Coverage
Test coverage checks whether important scenarios, roles, and edge cases were tested.
Copilot Test Strategy
A Copilot test strategy uses AI to help draft tests while humans verify relevance and completeness.
Checkpoint rule
Do the check-up only after you can summarize each concept in one sentence and identify one dangerous pitfall from memory.
Knowledge Check (after reading)
Short check-up to confirm understanding of this module.
Check-up Questions
A team begins testing an agent by asking ten ad hoc questions in the chat pane. What should it do next to make testing repeatable?
An evaluation case sends a question to an authenticated agent that uses connectors. What must the test profile represent?
Answer all questions to submit.
Next step personalized recommendations
Continue learning
Move forward only after this module is stable.
What is Pass Harbor?
Completely free exam prep for 317 U.S. exams.
- Practice questions
- Flashcards
- Study guides
- Mock exams
- No registration
- No paywall
- Start instantly
“No more expensive exam prep. Quality study tools should be accessible to everyone.”