Close prepared select statements invalidation loopholes

[wprzytula]

Quoting the umbrella drivers issue:

Prepared statements invalidation used to be completely broken, you can read details on it here Recently core merged PR that fixes problems for select statements. After that PR, if driver supply SCYLLA_USE_METADATA_ID at the startup, when statements is prepared server hands out result queries metadata hash. When driver executes this query it sends metadata id alongside with the request. Server checks driver metadata id and what it has on its side, if there is any difference it will send new metadata and metadata id. Driver suppose to pick it up and update metadata and metadata id on it's side, before deserializing response, solving issue with having outdated metadata on driver side. Unfortunately it is solved only for SELECT statements, UPDATE/INSERT will be solved on separate occasion.

Core changes are in, we need to start working on driver side.

This commit from the Core and this from Python Driver can serve as a reference.

To Be Done:

1. parse SCYLLA_USE_METADATA_ID protocol extension from SUPPORTED frame;
   • if present in SUPPORTED, send it in the STARTUP frame;
2. if the extension is negotiated:
   • store the metadata ID in ResultMetadata
   • read METADATA_CHANGED (=3) flag in result metadata;
   • if the flag is set, read the new metadata ID (short bytes), which is put between the paging state and the global tables spec;
3. if received new result metadata that is different from the cached one:
   • use the new metadata for deserialization instead of the old one;
   • [*] replace the old cached metadata with the new one.

Metadata mutability

If the [*] is not done, we will have a correct implementation that no longer faces the risk of using outdated metadata for deserialization.
However, we will have to receive the updated metadata upon every prepared statement execution, which may be costly for SELECTs with little data returned. It needs to be considered if we can (and should) make result metadata mutable in the PreparedStatement.

Considerations:

• ResultMetadata is already kept behind an Arc. We could use ArcSwap instead to get cheaply-synchronized mutability.
• repreparation could (and should) update the ResultMetadata.

Prepared Metadata Issue

There's no mechanism implemented on the server side to update prepared metadata once it changes due to, e.g., ALTERing some column that has an associated bind parameter. This means that having closed the gap with CQLv5 regarding ResultMetadata, we still have a similar problem with PreparedMetadata. cc @dkropachev @Lorak-mmk @andrzej-jackowski-scylladb

[Lorak-mmk]

Considerations:

ResultMetadata is already kept behind an Arc. We could use ArcSwap instead to get cheaply-synchronized mutability.
repreparation could (and should) update the ResultMetadata.

Let's do this too.

There's no mechanism implemented on the server side to update prepared metadata once it changes due to, e.g., ALTERing some column that has an associated bind parameter. This means that having closed the gap with CQLv5 regarding ResultMetadata, we still have a similar problem with PreparedMetadata.

This is much less sever issue, because there is nothing like SELECT * for insets. Encountering this issue would require dropping a column (that is used as a bound parameter in prepared statement), then re-adding it with different type, and then executing the old prepared statement object. Imo this is not a high-priority scenario.

[dkropachev]

Is there skip metadata flag thing missing ?

[wprzytula]

Is there skip metadata flag thing missing ?

WDYM? There is a setter on PreparedStatement that controls the flag.