SamplingParams
The parameters controlling the text generation sampling strategy.
Contents
- maxTokens
-
The maximum number of tokens to generate per sample.
Type: Integer
Valid Range: Minimum value of 512. Maximum value of 131072.
Required: Yes
- seed
-
The random seed for reproducible sampling. When set, the same seed with the same input produces deterministic outputs.
Type: Integer
Required: No
- stop
-
The list of stop sequences that terminate generation when encountered in the output.
Type: Array of strings
Required: No
- temperature
-
The sampling temperature controlling randomness. Higher values produce more diverse outputs.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 2.
Required: No
- topK
-
The number of highest-probability vocabulary tokens to keep for top-k filtering. Set to -1 to disable.
Type: Integer
Required: No
- topP
-
The cumulative probability threshold for nucleus (top-p) sampling. Only tokens with cumulative probability below this threshold are considered.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: