AdamParamsConfig
The Adam optimizer hyperparameters used during an optimizer step.
Contents
- learningRate
-
The learning rate for the optimizer. Must be greater than 0.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 0.01.
Required: Yes
- beta1
-
The exponential decay rate for the first moment estimates.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 0.999999.
Required: No
- beta2
-
The exponential decay rate for the second moment estimates.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 0.999999.
Required: No
- eps
-
A small constant added to the denominator for numerical stability.
Type: Double
Valid Range: Minimum value of 1.0e-16. Maximum value of 0.01.
Required: No
- gradClipNorm
-
The maximum gradient norm for gradient clipping. Gradients are scaled down if their norm exceeds this value.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 100.
Required: No
- loraPlusLrRatio
-
The learning rate multiplier for LoRA+ B matrices relative to A matrices. When set, B matrices use
learningRate * loraPlusLrRatiowhile A matrices use the base learning rate.Type: Double
Valid Range: Minimum value of 1.0. Maximum value of 100.0.
Required: No
- weightDecay
-
The weight decay (L2 regularization) coefficient.
Type: Double
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: