FM Training and Fine-Tuning
This topic covers pre-training, fine-tuning, continuous pre-training, instruction tuning, transfer learning, data curation, representativeness, labeling, and RLHF.
How to study for AWS Certified AI Practitioner
Treat each question as a business and governance decision: identify the AI pattern, choose the right AWS capability, then add responsible AI, cost, security, and evaluation controls.
Core concepts
Concept 1
Pre-training builds broad model capability, while fine-tuning adapts a model to a task, domain, style, or instruction pattern.
Exam cue: Use fine-tuning when repeated task behavior or domain adaptation is needed.
Concept 2
Fine-tuning data must be curated, governed, representative, labeled where required, and appropriate for the target behavior.
Exam cue: Use RAG before fine-tuning when the main need is access to factual private knowledge.
Concept 3
Customization choices should be weighed against RAG, in-context learning, distillation, cost, risk, and governance needs.
Exam cue: Check data rights, quality, representativeness, and labeling before model customization.
Risk pitfalls and guardrails
Fine-tuning on low-quality or ungoverned data.
Guardrail: Avoid choosing GenAI because it sounds modern, trusting fluent output without validation, or ignoring privacy, cost, and governance requirements.
Using continuous pre-training when prompt examples would solve the task.
Guardrail: Avoid choosing GenAI because it sounds modern, trusting fluent output without validation, or ignoring privacy, cost, and governance requirements.
Ignoring data rights and provenance during model customization.
Guardrail: Avoid choosing GenAI because it sounds modern, trusting fluent output without validation, or ignoring privacy, cost, and governance requirements.
Memory anchors
Pre-Training
Pre-training teaches a model broad patterns from large datasets before task-specific adaptation.
Fine-Tuning
Fine-tuning adapts a model for a task, domain, or response style using additional examples.
Instruction Tuning
Instruction tuning trains a model to follow task instructions more effectively.
Transfer Learning
Transfer learning reuses learned model capability for a related task.
Data Curation
Data curation selects, cleans, and organizes training data for quality and fit.
Representativeness
Representativeness means data reflects the population or use cases the model will encounter.
RLHF
Reinforcement learning from human feedback uses human preferences to improve model behavior.
Continuous Pre-Training
Continuous pre-training continues broad model training on additional domain data.
Checkpoint rule
Do the check-up only after you can summarize each concept in one sentence and identify one dangerous pitfall from memory.
Knowledge Check (after reading)
Short check-up to confirm understanding of this module.
Check-up Questions
What generally happens during foundation model pre-training?
How does fine-tuning differ from foundation model pre-training?
Answer all questions to submit.
Next step personalized recommendations
Continue learning
Move forward only after this module is stable.
What is Pass Harbor?
Completely free exam prep for 317 U.S. exams.
- Practice questions
- Flashcards
- Study guides
- Mock exams
- No registration
- No paywall
- Start instantly
“No more expensive exam prep. Quality study tools should be accessible to everyone.”