The Data Catalog supports creating partition indexes to provide efficient lookup for specific
partitions. For more information, see Creating partition
indexes. The AWS Glue crawler creates partition indexes for Amazon S3 and Delta Lake targets
by default.
- AWS Management Console
-
-
Sign in to the AWS Management Console and open the AWS Glue console at
https://eusc-de-east-1.console.amazonaws-eusc.eu/glue/.
-
Choose Crawlers under the Data Catalog.
-
When you define a crawler, the option to Create partition indexes
automatically is enabled by default under Advanced
options on the Set output and scheduling
page.
To disable this option, you can unselect the checkbox Create partition indexes
automatically in the console.
-
Complete the crawler configuration and choose Create crawler.
- AWS CLI
-
You can also disable this option by using the
AWS CLI, set the CreatePartitionIndex in the configuration parameter. The
default value is true.
aws glue update-crawler \
--name myCrawler \
--configuration '{"Version": 1.0, "CreatePartitionIndex": false }'
Tables created by the crawler do not have the variable partition_filtering.enabled by default. For more information, see AWS Glue partition indexing and filtering.
Creating partition indexes for encrypted partitions is not supported.