How can I ensure that my Databricks workspace enforces the use of IMDSv2 on all clusters, including any necessary upgrades and testing, while also disabling isolated AWS Glue catalog usage?
1. Upgrade your code and any existing AWS CLIs and SDKs that your workloads use to use IMDSv2.
2. Modify all notebooks in the workspace to remove any existing IMDSv1 usage and replace with IMDSv2 usage.
3. Test your modified code to ensure it works correctly with IMDSv2.
4. As a workspace admin, go to the settings page and click the Compute tab.
5. Click Enforce AWS instance metadata v2.
6. Refresh the page to ensure that the setting took effect.
7. To disable isolation of AWS Glue catalog, add the following Spark conf line to your clusters: `spark.databricks.hive.metastore.glueCatalog.isolation.enabled false`
8. Restart any running clusters to ensure that all EC2 instances have IMDSv2 enforced. If clusters are attached to a fleet instance pool, create a new fleet instance pool and recreate the clusters using the new fleet instance pool.
9. Monitor the CloudWatch metric MetadataNoToken to ensure that your workspace is not making any active IMDSv1 calls.

Note: Enforcing IMDSv2 causes any existing workloads to fail if they use IMDSv1 to fetch instance metadata.