Customization techniques - Amazon SageMaker AI
Services or capabilities described in AWS documentation might vary by Region. To see the differences applicable to the AWS European Sovereign Cloud Region, see the AWS European Sovereign Cloud User Guide.

Customization techniques

Customization techniques define how your model learns from data. Each technique requires a different dataset format and has its own hyperparameters. Choose based on what data you have and what behavior you want to achieve.

For Python SDK examples, see the example section in each technique page and Serverless trainers.

  • SFT (Supervised Fine-Tuning) — Train on labeled prompt-response pairs. Best for teaching a model specific style, format, or domain knowledge. For the expected training data structure, see Dataset format.

  • DPO (Direct Preference Optimization) — Train on preferred vs rejected response pairs. Best for aligning with human preferences and avoiding undesirable outputs. For dataset requirements, see Dataset format.

  • RFT (Reinforcement Fine-Tuning) — Optimize via reward signals: code-based verification (RLVR) or an LLM judge (RLAIF). Best for improving factual accuracy or subjective quality. For RLVR dataset requirements, see Dataset format. For RLAIF dataset requirements, see Dataset format.

  • Sequence lengths — Maximum sequence lengths supported for each open weight model, technique, and training type in serverless customization.

  • Multi-turn reinforcement learning (Multi-Turn Reinforcement Learning) — Train agents for multi-step agentic tasks with growing context. For setup requirements, see Prerequisites.

  • Continuous customization (Continuous Customization) — Chain multiple techniques sequentially (for example, SFT → DPO → RLVR). Requires LoRA training type.

  • Distillation — Transfer knowledge from a teacher model to a student model. Best when you have a specific high-volume task that the student can be trained on. When the student is smaller than the teacher, you also get lower inference latency of the student model.

Each technique can be combined with either LoRA or FFT training type. See Training types for details on choosing between them.

Technique Dataset format SDK trainer FFT Notes
SFT messages or prompt/completion JSONL SFTTrainer Yes LoRA and FFT use the same dataset schema. See Dataset format.
DPO chosen/rejected message pairs or prompt/chosen/rejected JSONL DPOTrainer Yes LoRA and FFT use the same dataset schema. See Dataset format.
RLVR prompt with reward_model metadata JSONL RLVRTrainer Yes LoRA and FFT use the same dataset schema. See Dataset format.
RLAIF prompt with LLM-judge reward_model metadata JSONL RLAIFTrainer Yes LoRA and FFT use the same dataset schema. See Dataset format.
Multi-turn reinforcement learning prompt column in Parquet, JSONL, JSON, or CSV MultiTurnRLTrainer No MTRL training uses prompt datasets and a multi-turn agent environment. See Prompt dataset format and SageMaker AI Python SDK.