Services or capabilities described in AWS documentation might vary by Region. To see the differences applicable to the AWS European Sovereign Cloud Region, see the AWS European Sovereign Cloud User Guide.

GuardDuty AI Protection finding types

When you enable AI Protection, GuardDuty analyzes AWS CloudTrail management events and data events from Amazon Bedrock and Amazon SageMaker AI. GuardDuty uses this data to detect potentially suspicious activity that targets your AI workloads. When GuardDuty detects a potential threat, it generates one of the following finding types. The Resource type for these findings is AccessKey, and the finding identifies the IAM identity that invoked the model.

Each finding also includes details about the affected AI resources in the resource object of the finding. The Impact:IAMUser/AnomalousModelInvocation and Impact:IAMUser/CostHarvesting findings include a resource.modelDetails list that identifies the invoked Amazon Bedrock or Amazon SageMaker AI models. The Impact:IAMUser/PromptInjection.Direct finding includes a resource.bedrockGuardrailDetails object that describes the Amazon Bedrock Guardrail that intervened. You can view these details in the GuardDuty console or in the finding JSON.

The Impact:IAMUser/AnomalousModelInvocation and Impact:IAMUser/CostHarvesting finding types apply to both Amazon Bedrock and Amazon SageMaker AI model invocations. The Impact:IAMUser/PromptInjection.Direct finding type applies to Amazon Bedrock workloads that use Amazon Bedrock Guardrails.

Impact:IAMUser/AnomalousModelInvocation

An IAM identity invoked an Amazon Bedrock or Amazon SageMaker AI model in a way that deviates from the established baseline for the identity or account.

Default severity: Low

This finding informs you that an IAM identity invoked an Amazon Bedrock or Amazon SageMaker AI model in a way that deviates from the historical activity of the identity or account. This activity might indicate that a threat actor is using compromised credentials to access foundation models in your account. For example, the actor might probe a model, craft adversarial inputs, or run automated requests.

GuardDuty establishes a baseline of normal model invocation activity for each IAM identity and AWS account that invokes Amazon Bedrock or Amazon SageMaker AI. GuardDuty profiles features of the invocation activity, including the following:

When GuardDuty observes activity that deviates significantly from this baseline, it generates this finding. Examples include invocations from a previously unseen IP address, a previously unseen user agent, or a model that the identity has never invoked. The finding includes a resource.modelDetails list identifying the models involved in the suspicious activity. Each entry contains a modelId representing an Amazon Bedrock foundation model, inference profile, provisioned, custom, or imported model ARN, Amazon Bedrock Marketplace endpoint, prompt resource, or Amazon SageMaker AI endpoint name.

This finding maps to the MITRE ATLAS technique AML.T0040 - AI Model Inference API Access on the MITRE ATLAS website.

Remediation recommendations:

If this activity is unexpected, your credentials may be compromised. For more information, see Remediating potentially compromised AWS credentials.

To reduce the risk of unauthorized model invocations, restrict which identities can invoke models. Scope access to the following model invocation APIs to only the principals and models that require them:

To deny model inference across your organization where it isn't needed, you can use a service control policy (SCP). For more information, see Identity-based policy examples for Amazon Bedrock in the Amazon Bedrock User Guide and Amazon SageMaker AI identity-based policy examples in the Amazon SageMaker AI Developer Guide.

Impact:IAMUser/CostHarvesting

An IAM identity invoked an Amazon Bedrock or Amazon SageMaker AI model with anomalous token volumes that might indicate a cost harvesting attack.

Default severity: Low

This finding informs you that an IAM identity invoked an Amazon Bedrock or Amazon SageMaker AI model with input or output token volumes that deviate significantly from the baseline for the identity or account. In a cost harvesting attack, a threat actor sends computationally expensive inputs to a model to inflate the operating costs of the AI workload, without necessarily exfiltrating data. These inputs might include excessively long or deliberately malformed prompts.

GuardDuty establishes a baseline of average input token volume and average output token volume for each IAM identity and account that invokes Amazon Bedrock or Amazon SageMaker AI models. When GuardDuty observes invocations with token counts that exceed this baseline, and correlates the volume anomaly with other unusual signals, GuardDuty generates this finding. The finding reports the anomalous activity in the service.detection object, including the observed invocation and token volumes that deviated from the baseline. It also includes a resource.modelDetails list identifying the targeted models. Each entry contains a modelId representing an Amazon Bedrock foundation model, inference profile, provisioned, custom, or imported model ARN, Amazon Bedrock Marketplace endpoint, prompt resource, or Amazon SageMaker AI endpoint name.

This finding maps to the MITRE ATLAS technique AML.T0034 - Cost Harvesting on the MITRE ATLAS website.

Remediation recommendations:

If this activity is unexpected for the associated identity, the credentials might have been compromised and used to drive up the cost of your AI workloads. For more information, see Remediating potentially compromised AWS credentials. Review the model invocations associated with the identity, and restrict access to the affected models by scoping access to the following model invocation APIs to only the principals that require them:

To detect and contain abnormal spend, you can use AWS Budgets, including budget actions that apply a restrictive IAM policy when a budget threshold is exceeded, and AWS Cost Anomaly Detection. To attribute Amazon Bedrock costs to specific workloads, you can attach cost allocation tags to an application inference profile.

Impact:IAMUser/PromptInjection.Direct

An Amazon Bedrock Guardrail detected a direct prompt injection attempt in an invocation made by an IAM identity.

Default severity: Low

This finding informs you that an Amazon Bedrock Guardrail configured for an Amazon Bedrock workload in your account detected a direct prompt injection attempt. In a direct prompt injection attack, a threat actor crafts a malicious prompt as input to a foundation model. The prompt makes the model ignore its original instructions and follow the attacker's instructions instead. Prompt injection can give an attacker initial access to carry out additional steps against an AI application, such as exfiltrating data, calling unauthorized tools, or bypassing safety controls.

GuardDuty generates this finding when an Amazon Bedrock Guardrail intervenes on an invocation because it detected a prompt attack with high confidence. To enable GuardDuty to detect direct prompt injection, configure an Amazon Bedrock Guardrail with a prompt attack content filter. To ensure that the guardrail evaluates invocations across your environment, enforce it broadly instead of relying on individual applications to attach it to each request. You can enforce a guardrail for all Amazon Bedrock model invocations in an account, or across your organization by using Amazon Bedrock policies in AWS Organizations. For more information, see Apply cross-account safeguards with Amazon Bedrock Guardrails enforcements in the Amazon Bedrock User Guide. When a guardrail evaluates an invocation, Amazon Bedrock records the evaluation in AWS CloudTrail data events, which GuardDuty analyzes to generate this finding.

The finding includes a resource.bedrockGuardrailDetails object that describes the guardrail that intervened. The following table describes the fields in this object.

This finding maps to the MITRE ATLAS technique AML.T0051 - LLM Prompt Injection on the MITRE ATLAS website.

Remediation recommendations:

Review the prompt that triggered the guardrail intervention and the IAM identity that submitted the prompt. If the activity is unexpected for the associated identity, the credentials might have been compromised. For more information, see Remediating potentially compromised AWS credentials. If the guardrail detected but did not block the prompt attack, consider setting the prompt attack content filter's action to Block rather than Detect so that the guardrail blocks the content. For more information, see Options for handling harmful content detected by Amazon Bedrock Guardrails in the Amazon Bedrock User Guide. You can also restrict access to the Amazon Bedrock invocation APIs for the affected identity.