Help improve this page
To contribute to this user guide, choose the Edit this page on GitHub link that is located in the right pane of every page.
Assign security groups to individual Pods in EKS Auto Mode
Applies to: EKS Auto Mode clusters running Kubernetes 1.36 or later
Security groups for Pods lets you assign dedicated Amazon EC2 security groups to individual Pods, rather than sharing the security groups of the node they run on. EKS Auto Mode implements this through a SecurityGroupPolicy resource, Kubernetes Dynamic Resource Allocation (DRA), and EC2 branch network interfaces. A Pod matched by a SecurityGroupPolicy gets its own branch network interface attached to the trunk interface on its node, carrying only the security groups you specify. Traffic to and from that Pod is evaluated against those security groups at the branch interface, independently of the node’s own security groups.
Note
This feature is different from podSecurityGroupSelectorTerms on a NodeClass. podSecurityGroupSelectorTerms applies one security group set to every Pod on nodes using that NodeClass. SecurityGroupPolicy lets you assign different security groups to different Pods on the same node, selected by Pod or service account labels, matching the behavior of security groups for Pods on non-Auto-Mode EKS clusters.
How it works
A Pod gets a dedicated branch network interface one of two ways:
-
SecurityGroupPolicy(recommended) — You create aSecurityGroupPolicythat selects Pods by label (podSelector), service account (serviceAccountSelector), or both, and references one or more security groups. The admission webhook then injects a resource claim into any new Pod matched by the policy, with no changes needed to the Pod spec itself. -
Native Dynamic Resource Allocation (DRA) — You add a
resourceClaimsentry to the Pod spec that points at theResourceClaimTemplatethe controller generates for an existingSecurityGroupPolicy, bypassing the webhook’spodSelectororserviceAccountSelectormatching and automatic injection. TheSecurityGroupPolicyis still what resolves and supplies the security groups; this path only changes how a Pod gets associated with it. See Alternative: reference the controller-generated claim template with native DRA.
From there, both paths converge:
-
The Kubernetes scheduler allocates a device representing a branch-interface slot on a node, backed by the resolved security groups.
-
EKS Auto Mode creates and attaches a branch network interface carrying those security groups, and the Pod’s networking is configured through it.
Pods with no resource claim, by either path, are unaffected and continue to use the node’s own security groups, as with any other EKS Auto Mode Pod.
How this compares to other ways to assign security groups
Three mechanisms assign security groups to EKS workloads. Two of them use the same SecurityGroupPolicy kind but belong to different API groups and have incompatible specs, so manifests aren’t portable between them.
| Mechanism | API group | Behavior |
|---|---|---|
|
NodeClass |
|
EKS Auto Mode. The same security groups for every Pod on the node. For more information, see Separate subnets and security groups for Pods. |
|
|
|
EKS Auto Mode. Different security groups per Pod. Described on this page. |
|
|
|
Amazon VPC CNI, on clusters that aren’t EKS Auto Mode. Different security groups per Pod. Its spec isn’t compatible with the |
Migrate from the Amazon VPC CNI SecurityGroupPolicy
Both resources can exist in the same cluster. Which one applies depends on the API group you used and the node the Pod lands on: vpcresources.k8s.aws/v1beta1 policies apply to Pods on non-Auto-Mode nodes, such as a managed node group, and eks.amazonaws.com/v1 policies apply to Pods on EKS Auto Mode nodes. Neither enforces the other’s policies, so a Pod that can be scheduled onto either kind of node needs a policy in both API groups to be secured on both.
The field mapping is direct. podSelector and serviceAccountSelector carry over unchanged. Replace spec.securityGroups.groupIds with one securityGroupSelectorTerms entry per group:
# vpcresources.k8s.aws/v1beta1 spec: securityGroups: groupIds: - sg-0123456789abcdef0 - sg-0123456789abcdef1 # eks.amazonaws.com/v1 spec: securityGroupSelectorTerms: - id: sg-0123456789abcdef0 - id: sg-0123456789abcdef1
Before you migrate, check your policies against the scope and limits of the new API, which differ from v1beta1:
-
At most one policy can match a Pod. v1beta1 allows a Pod to match several policies and merges the result. Overlapping selectors must be made disjoint first, or EKS Auto Mode rejects the Pod. For more information, see Considerations.
-
Security group counts differ. v1beta1 caps
groupIdsat 5 per policy. The new API resolves up to 16, but your account’s Security groups per network interface quota still applies, and it defaults to 5. -
Selector terms resolve dynamically.
nameandtagsterms re-resolve as security groups change, so a policy’s effective groups can change without the policy being edited. Directidterms behave like v1beta1. -
Each policy consumes node capacity. Branch network interfaces lower a node’s effective
maxPods, and more than five distinct policies on one node risks scheduling latency. See the considerations that follow.
Create a SecurityGroupPolicy
apiVersion: eks.amazonaws.com/v1 kind: SecurityGroupPolicy metadata: name: my-sgp namespace: my-namespace spec: podSelector: matchLabels: app: my-app securityGroupSelectorTerms: - id: sg-0123456789abcdef0
Self-referencing ingress rule required for Pods under the same policy
A SecurityGroupPolicy selects existing security groups; it doesn’t create them. A security group you create for use with a SecurityGroupPolicy has no ingress rules by default. If Pods matched by the same SecurityGroupPolicy need to communicate with each other — including Pods on the same node — add a self-referencing ingress rule using the following AWS CLI command:
aws ec2 authorize-security-group-ingress --group-id sg-0123456789abcdef0 \ --source-group sg-0123456789abcdef0 --protocol -1
Without this rule, Pods secured by the same policy can’t reach each other, because EKS Auto Mode enforces security groups on same-node Pod-to-Pod traffic as well as traffic that leaves the node.
Apply this configuration with:
kubectl apply -f my-sgp.yaml
Alternative: reference the controller-generated claim template with native DRA
For every SecurityGroupPolicy, the controller generates a matching ResourceClaimTemplate that carries the policy’s resolved security groups. The webhook’s only job is to match Pods against podSelector or serviceAccountSelector and automatically inject a reference to that template. You can reference the same ResourceClaimTemplate yourself, from a Pod’s resourceClaims field, without relying on label-based matching at all:
kubectl get resourceclaimtemplate -n my-namespace my-sgp -o yaml
Use this approach if you want explicit, per-Pod control over which Pods get the security groups a SecurityGroupPolicy resolves, instead of relying on labels the webhook matches against.
A Pod that references the claim template directly in its resourceClaims field bypasses the mutating webhook’s podSelector or serviceAccountSelector matching and automatic injection. Reference exactly one SecurityGroupPolicy claim template per Pod. A Pod gets a single branch network interface, so referencing more than one policy’s claim template in the same Pod isn’t supported. When the webhook matches Pods for you, it rejects this misconfiguration automatically, as described in Considerations.
The SecurityGroupPolicy itself is still required: it’s what resolves and supplies the security groups backing the claim. The considerations on this page about same-node enforcement, the node-local exception list, and maxPods impact still apply, along with the recommendation to use no more than five policies per node for scheduler performance. They describe the branch network interface datapath itself, not the webhook matching layer in front of it.
SecurityGroupPolicy specification
apiVersion: eks.amazonaws.com/v1 kind: SecurityGroupPolicy metadata: name: my-sgp namespace: my-namespace spec: # Required: at least one of podSelector or serviceAccountSelector. # If both are set, a Pod must match BOTH to be selected. podSelector: matchLabels: app: my-app # serviceAccountSelector: # matchLabels: # my-label: my-value # Optional: scope this policy to nodes in a specific NodePool. The only # key this field may reference is karpenter.sh/nodepool. Omit the field # to apply the policy cluster-wide. nodeSelector: matchLabels: karpenter.sh/nodepool: general-purpose # Required: at least one term. Terms are OR'd; id, name, and tags are # mutually exclusive within a single term. securityGroupSelectorTerms: - id: sg-0123456789abcdef0 # Alternative approaches: # - name: "my-security-group" # - tags: # Team: "platform"
-
podSelectorandserviceAccountSelector— At least one is required. If you set both, a Pod must match both to be selected by the policy. -
nodeSelector— Optional. The only label key this field may reference iskarpenter.sh/nodepool, which scopes the policy to Pods scheduled onto a specific NodePool. Omit this field to apply the policy to matching Pods cluster-wide, regardless of NodePool. -
securityGroupSelectorTerms— Required, at least one term, maximum 30 terms. Terms are evaluated as OR — the resolved security group set is the union of every term’s matches. Within a single term,id,name, andtagsare mutually exclusive. Atagsterm allows a maximum of 20 key-value pairs. The resolved set across all terms can’t exceed 16 security groups, the ceiling enforced when the policy resolves. Separately, your account’s Amazon VPC Security groups per network interface quota, which defaults to 5, governs how many a Pod’s branch network interface can actually carry.Security groups per network interface defaults to 5
The default is 5, not 16. 16 is only the ceiling you can raise the quota to, so request an increase before relying on more than five.
The two limits fail differently. More than 16 is caught at resolution:
SecurityGroupsResolved: False, reasonsg_quota_exceeded. More than your quota but no more than 16 resolves successfully, then fails at Amazon EC2 as anENICreationFailedevent on the Pod’s resource claim, with the condition stillTrue.This quota multiplied by Inbound or outbound rules per security group can’t exceed 1,000. At 16 groups the default 60 rules still fits, but rules can’t then exceed 62. For more information, see Amazon VPC quotas.
Check policy status
EKS Auto Mode reports what a policy resolved to in its status field:
kubectl get securitygrouppolicies.eks.amazonaws.com <name> -n <namespace> -o yaml
-
status.securityGroups— The security group IDs the selector terms resolved to. These are the groups a matched Pod’s branch network interface carries. -
status.conditions[]withtype: SecurityGroupsResolved—Trueonce the selector terms resolved successfully. When it’sFalse, the condition’sreasonnames the failure class:-
no_sgs_matched— Every lookup succeeded but no security group matched the terms. -
sg_quota_exceeded— More than 16 security groups resolved. Always means more than 16, never that your account quota is lower than the resolved count. -
invalid_spec— The policy spec itself is rejected. -
resolution_error— An Amazon EC2 call failed while resolving the terms: a service-side problem, not something to fix in your policy.
-
A policy with SecurityGroupsResolved: False doesn’t secure Pods. Matched Pods stay pending rather than being admitted with the node’s security groups. A sg_quota_exceeded reason also records a SecurityGroupQuotaExceeded warning event on the policy; the other reasons record SecurityGroupResolutionFailed.
Considerations
-
At most one
SecurityGroupPolicycan match a Pod. If a Pod’s labels (and, where configured, service account) match more than oneSecurityGroupPolicy, EKS Auto Mode rejects the Pod rather than admitting it with an ambiguous or unsecured configuration. Keep yourpodSelectorandserviceAccountSelectorvalues disjoint across policies. A Pod gets a single branch network interface and can carry only one policy’s security groups, so referencing more than one policy’s claim template yourself inresourceClaimsisn’t supported either. -
Same-node Pod-to-Pod traffic is enforced. Unlike
standardmode in the open source Amazon VPC CNI, EKS Auto Mode evaluates security groups on Pod-to-Pod traffic even when both Pods are on the same node and the same branch network interface. A security group with no self-referencing ingress rule blocks Pods under the same policy from reaching each other on the same node. This is a deliberate security property, not a limitation to work around. -
A fixed set of node-local endpoints bypasses security group enforcement. Traffic between a Pod secured by a
SecurityGroupPolicyand the following node-local endpoints is not evaluated against the Pod’s security groups, because these connections never cross the branch network interface onto the VPC: the EC2 instance metadata service (IMDS), the Amazon EKS Pod Identity Agent, kubelet health and readiness probes from the node’s own IP address, and CoreDNS. This exception set is fixed by EKS Auto Mode and isn’t customer-configurable. -
A Pod can’t reach itself through its own Service. A Pod secured by a
SecurityGroupPolicythat connects to a Kubernetes Service it is itself a backend of needs no additional security group rule for that specific path — EKS Auto Mode delivers that connection locally. This differs from a Pod reaching a different Pod (including a sibling behind the same Service), which is always security-group enforced per the previous points. -
IPv4 egress from an IPv6 cluster isn’t enforced by the Pod’s security group. On an IPv6 cluster with
enableV4Egressenabled (the default), a Pod’s IPv4 egress traffic is translated through the node’s primary network interface and uses the node’s security groups for that path, not the Pod’s. -
Branch network interfaces reduce the IP addresses available for other Pods on the node. EKS Auto Mode calculates
maxPodsbased on the available ENI slots and IP addresses per instance type. A node running any Pod secured by aSecurityGroupPolicyreserves at least one ENI slot for the branch interface. It reserves two slots if you also use separate Pod subnets. This lowers that node’s effectivemaxPodscompared to a node running no secured Pods. This recalculation applies to new nodes; nodes that joined the cluster before anySecurityGroupPolicyexisted don’t have it applied retroactively. If you adopt this feature on an existing NodePool, recycle its nodes somaxPodsreflects actual capacity. If you don’t, those nodes keep advertising their originalmaxPodswhile having fewer slots available, so the scheduler can place more Pods than the node can give interfaces to. The excess Pods stay pending waiting for a branch interface rather than failing outright. -
We strongly recommend no more than five
SecurityGroupPolicyresources targeting Pods on the same node. Each distinct policy on a node requires its own branch network interface accounting. Exceeding five distinct policies on one node risks increased scheduling latency. It also increases Karpenter andkube-schedulerCPU and memory consumption as node density grows, up to and including scheduler instability at scale. Further scheduler-side optimization for this scenario isn’t yet available upstream. -
Compatible with secondary IP mode and separate Pod subnets.
SecurityGroupPolicyworks alongside a NodeClass configured for secondary IP mode (advancedNetworking.ipv4PrefixSize) or separate Pod subnets and security groups (podSubnetSelectorTerms/podSecurityGroupSelectorTerms, see Separate subnets and security groups for Pods). The branch network interface’s security groups still come only from theSecurityGroupPolicy; the NodeClass setting changes where Pod IP addresses and the trunk interface are placed, not which security groups a secured Pod’s branch interface carries. -
Some trunking-compatible instance types have only two total ENI slots. On these instance types, a node can support regular Pod networking plus one
SecurityGroupPolicy-secured branch interface, but not also separate subnets and security groups for Pods at the same time — that configuration needs a third ENI slot. Choose a larger instance type in your NodePool if you need both features together. To check a specific instance type, see the limits.go file on the GitHub website; types with IsTrunkingCompatible: trueand only two interfaces are affected. -
Grant the cluster role
ec2:CreateNetworkInterfacePermission. In addition to theAmazonEKSNetworkingPolicymanaged policy, your cluster’s IAM role needs an inline (or separately managed) policy statement grantingec2:CreateNetworkInterfacePermission, required for the trunk-to-branch interface association. For more information, see Learn about identity and access in EKS Auto Mode. -
Upgrading an existing cluster to Kubernetes 1.36 or later. EKS Auto Mode’s control plane components that implement this feature start on the control plane’s next restart after the version upgrade, which can take up to several minutes. The feature isn’t available on a cluster immediately at the moment the control plane reports the new version.
-
We recommend removing secured Pods and policies before you disable EKS Auto Mode. Before you disable EKS Auto Mode, delete or scale down the workloads whose Pods a
SecurityGroupPolicysecures and let those Pods terminate gracefully. Then delete everySecurityGroupPolicyand wait untilkubectl get securitygrouppolicies.eks.amazonaws.com -Areturns no resources. This lets EKS Auto Mode release each policy’s branch network interfaces and remove its finalizer while EKS Auto Mode is still enabled. -
Not supported with EKS Auto Mode: alternate CNI or network policy plugins (see Alternate CNI plugins for Amazon EKS clusters).
SecurityGroupPolicyonly applies to Pods on EKS Auto Mode’s built-in networking.
Troubleshooting
If a Pod matched by a SecurityGroupPolicy doesn’t get its own security groups, check in order:
-
Does the Pod have an injected resource claim?
kubectl get pod <pod-name> -o jsonpath='{.spec.resourceClaims}'An empty result means the Pod’s labels didn’t match anySecurityGroupPolicyin its namespace, or the admission webhook didn’t fire. -
Did the policy resolve its security groups?
kubectl get securitygrouppolicies.eks.amazonaws.com <name> -n <namespace> -o jsonpath='{.status}'Check forSecurityGroupsResolved: "True".Falseusually means zero security groups matched (no_sgs_matched) or more than 16 resolved (sg_quota_exceeded). If it readsTruebut Pods still get no interface, check your Security groups per network interface quota (default 5): exceeding it leaves the conditionTrueand surfaces as anENICreationFailedevent. -
Is there a
ResourceClaimTemplatefor the policy?kubectl get resourceclaimtemplate -n <namespace>If missing, the policy hasn’t been reconciled yet. Re-apply it to trigger reconciliation. -
Check for a
Warningevent on the Pod’s resource claim. AnENICreationFailedevent names the underlying EC2 error, most often an IAM permissions issue (see theec2:CreateNetworkInterfacePermissionconsideration described earlier in this topic) or a security group that doesn’t exist in the cluster’s VPC. If that keeps failing, EKS Auto Mode stops retrying the security group set and records aSecurityGroupIsolatedevent instead; correcting the security group clears it.