Isolate workloads with AWS Nitro Sandboxes

The Nitro System removes the memory of the Nitro Sandbox from the parent instance, so neither the parent instance nor its administrators can read or change that memory. Each virtual device exchanges data through its own dedicated buffer. A Nitro Security Module (NSM) in the Nitro Sandbox provides cryptographic attestation.

The parent instance still carries the device traffic, chooses the image, and controls when the Nitro Sandbox runs. Encrypt data that leaves the Nitro Sandbox, and use attestation to check its image before you give it secrets.

Use confidential mode for workloads that you keep private from the administrators of the parent instance.

The parent instance shares the memory of the Nitro Sandbox and can read and write it. There is no NSM and no attestation. The virtual devices exchange data directly through the memory of the Nitro Sandbox, without dedicated buffers, and the trust model is equivalent to that of a nested virtual machine.

Use direct mode when the same trusted party operates both the Nitro Sandbox and the parent instance.

A Nitro Sandbox can use block devices on storage of the parent instance and resize them while it runs. Over the network, a workload can use Amazon Simple Storage Service (Amazon S3), Amazon Elastic File System (Amazon EFS), and other network storage through a client in your image.

Each Nitro Sandbox runs its own network stack. The network backend of the VMM on the parent instance decides how that stack connects to the network of the parent instance. A local socket (vsock) connects the Nitro Sandbox to the parent instance.

Nitro Sandboxes are not available on flex instances, such as C7i-flex, M7i-flex, C8i-flex, M8i-flex, and R8i-flex. The following instance types support Nitro Sandboxes.

Each vCPU of a Nitro Sandbox either has a dedicated hardware thread of the parent instance or, with CPU overcommit, shares the vCPUs of the parent instance. You choose this for each Nitro Sandbox at launch.

You can run up to 5 Nitro Sandboxes for each vCPU of the parent instance, and at most 896 Nitro Sandboxes on one parent instance. Each Nitro Sandbox can have at most as many vCPUs as the parent instance. With CPU overcommit, all Nitro Sandboxes together can have up to 5 times as many vCPUs as the parent instance. The following table shows an example.

Without CPU overcommit, the vCPUs of all Nitro Sandboxes together can't exceed the vCPUs of the parent instance, minus the vCPUs of its first core. For more information, see Set the size of a Nitro Sandbox.

On instances with two threads per core, a core runs only one Nitro Sandbox, or only the parent instance, at a time. The compute available to Nitro Sandboxes is therefore bounded by the physical cores, about half the vCPUs. AWS Graviton instances have one thread per core.

Nitro Sandbox memory comes from a memory pool that the parent instance reserves at boot. Each Nitro Sandbox uses its own memory plus 2 MiB of the pool, and in confidential mode another 16 MiB for each virtual device.

Each running Nitro Sandbox also uses one of the Nitro Sandbox slots of the parent instance. The parent instance starts with 4 slots, and you can add more in steps of 8. Each step uses another 128 MiB of the pool. For more information, see Prepare the parent instance.

Give each Nitro Sandbox at least 256 MiB of memory.

Each Nitro Sandbox supports up to 32 virtual devices, including devices that you hot plug. You can't remove a device from a running Nitro Sandbox.

GPU passthrough into a Nitro Sandbox is currently unavailable.

When you stop, reboot, or terminate the parent instance, its Nitro Sandboxes stop too. You can't enable hibernation on a parent instance.

You can't change the vCPUs or memory of a running Nitro Sandbox. Nitro Sandboxes do not support snapshot and restore, pause and resume, or live migration.

The parent instance can read every block device that it presents to a Nitro Sandbox, including encrypted Amazon Elastic Block Store (Amazon EBS) volumes. Amazon EBS encryption and server-side encryption with AWS KMS keys (SSE-KMS) do not protect data from the parent instance. To keep data confidential, encrypt it inside the Nitro Sandbox with a key that AWS KMS releases only to an attested Nitro Sandbox.

A Nitro Sandbox has no instance metadata service (IMDS) of its own. If the parent instance routes traffic from a Nitro Sandbox, for example with NAT, the Nitro Sandbox can reach the IMDS of the parent instance and read its AWS Identity and Access Management (IAM) role credentials. IMDSv2 and the hop limit of 2 that Amazon Linux 2023 AMIs use do not prevent this. To block this access, reject traffic from Nitro Sandboxes to 169.254.169.254 on the parent instance.

One Nitro Sandbox can exhaust network resources that all Nitro Sandboxes share on the parent instance, such as the connection tracking table, so limit these resources for each Nitro Sandbox.

You can't use Nitro Sandboxes or Nitro Enclaves on an instance that has nested virtualization enabled.

You can't use Nitro Sandboxes and Credential Guard on the same instance during the same boot. If Credential Guard starts first, creating a Nitro Sandbox fails. If a Nitro Sandbox starts first, Windows stops with an error when Credential Guard starts. Nitro Enclaves on its own is not affected.