Services or capabilities described in AWS documentation might vary by Region. To see the differences applicable to the AWS European Sovereign Cloud Region, see the AWS European Sovereign Cloud User Guide.

RFT

Reinforcement Fine-Tuning (RFT) uses reinforcement learning to optimize model behavior based on reward signals rather than explicit input-output examples. Amazon SageMaker AI supports two RFT variants: RLVR (Reinforcement Learning with Verifiable Rewards) and RLAIF (Reinforcement Learning from AI Feedback).

RLVR

RLVR uses a code-based reward function that programmatically verifies whether model outputs are correct. Best suited for tasks with objectively right or wrong answers.

When to use

Dataset format

Each record contains a prompt and reward model metadata. The reward function evaluates model-generated responses during training.

{
  "data_source": "openai/gsm8k",
  "prompt": [
    {
      "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\".",
      "role": "user"
    }
  ],
  "ability": "math",
  "reward_model": {
    "ground_truth": "72",
    "style": "rule"
  }
}

Required fields:

To implement your own Lambda-based reward function, see RLVR - Custom Lambda. For the supported Lambda response envelopes, see Sample Lambda Output Payload.

Preset reward functions

Python SDK example

Use RLVRTrainer for serverless RLVR customization jobs:

from sagemaker.train import RLVRTrainer

trainer = RLVRTrainer(
    model_id="meta-textgeneration-llama-3-1-8b-instruct",
    train_data="s3://my-bucket/rlvr-train.jsonl",
    hyperparameters={
        "preset_reward_function": "gsm8k",
        "learning_rate": 1e-5,
        "max_epochs": 1,
        "global_batch_size": 128,
        "lora_rank": 16,
    },
)

trainer.train()

To run RLVR with FFT, use the same dataset format and remove LoRA-specific hyperparameters such as lora_rank and lora_alpha.

RLVR LoRA hyperparameters

RLVR FFT hyperparameters

Same parameters as RLVR LoRA without lora_rank and lora_alpha.

RLAIF

RLAIF uses another LLM as a judge to evaluate model responses based on a natural language reward prompt. Best suited for tasks with subjective quality criteria that are difficult to evaluate programmatically.

When to use

Dataset format

Each record contains a prompt and reward model metadata. The LLM judge evaluates model-generated responses during training.

{
  "data_source": "WeOpenML/PandaLM",
  "prompt": [
    {
      "role": "user",
      "content": "Below are two responses for a given task...Evaluate the responses and generate a reference answer.\n\n### Instruction:\nCompare the given products..."
    }
  ],
  "ability": "pairwise-judging",
  "reward_model": {
    "style": "llmj",
    "ground_truth": "2\n\n### Reason: Response 2 provides a more detailed comparison..."
  }
}

Required fields:

Judge templates

The following preset judge templates are provided in the training container. Set judge_prompt_template to one of these values:

To replace a preset template with your own Jinja prompt, see RLAIF - Custom Prompt.

Python SDK example

Use RLAIFTrainer for serverless RLAIF customization jobs:

from sagemaker.train import RLAIFTrainer

trainer = RLAIFTrainer(
    model_id="meta-textgeneration-llama-3-1-8b-instruct",
    train_data="s3://my-bucket/rlaif-train.jsonl",
    hyperparameters={
        "judge_prompt_template": "evaluate.jinja",
        "learning_rate": 1e-5,
        "max_epochs": 1,
        "global_batch_size": 128,
        "lora_rank": 16,
    },
)

trainer.train()

To run RLAIF with FFT, use the same dataset format and remove LoRA-specific hyperparameters such as lora_rank and lora_alpha.

RLAIF LoRA hyperparameters

Same parameters as RLVR LoRA with the following difference:

RLAIF FFT hyperparameters

Same parameters as RLAIF LoRA without lora_rank and lora_alpha.