How can I ensure that my Delta Lake table commits without reading the table state, and what are the limitations for row-level concurrency when using complex conditional clauses or MERGE commands?
To ensure that your Delta Lake table commits without reading the table state, you can use INSERT or append operations with the following conditions:

1. The logic is expressed using INSERT SQL logic or append mode.
2. The logic contains no subqueries or conditionals that reference the table targeted by the write operation.

However, it's important to note that many common patterns use MERGE operations to insert data based on table conditions. Although it might be possible to rewrite this logic using INSERT statements, if any conditional expression references a column in the target table, these statements have the same concurrency limitations as MERGE.

Regarding row-level concurrency limitations, some limitations apply for row-level concurrency, especially when using complex conditional clauses or MERGE commands:

1. Commands with complex conditional clauses, including conditions on complex data types such as structs, arrays, or maps, conditions using non-deterministic expressions and subqueries, and conditions that contain correlated subqueries.
2. For MERGE commands, you must use an explicit predicate on the target table to filter rows matching the source table. For merge resolution, the filter is used to only scan rows that might conflict based on filter conditions in concurrent operations.

In the case of many concurrent transactions, the writer prioritizes latency over conflict resolution, and conflicts may occur. All limitations for deletion vectors also apply.