What operations are automatically run by predictive optimization for enabled Delta tables in Databricks?
Predictive optimization in Databricks automatically runs OPTIMIZE and VACUUM operations for enabled Delta tables. The OPTIMIZE operation improves query performance by optimizing file sizes, while the VACUUM operation reduces storage costs by deleting data files that are no longer referenced by the table. However, it's important to note that the VACUUM command's retention window is determined by the 'delta.deletedFileRetentionDuration' table property, which defaults to 7 days. If you'd like to retain data for a longer duration, you must set this table property appropriately before enabling predictive optimization.