Couchbase SDK 3.x CancellationToken Handling

plan.md

Proposed Approach

This proposal is to clarify how we handle various cancellation scenarios which may cancel an in-flight operation.

Idempotent Operations (i.e. GET, LookupIn, etc)

Note: Should also include operations related to bootstrap like hello, auth, select bucket, etc.

Phase	External Cancellation	Timeout Cancellation
Before Sent	OperationCanceledException	UnambiguousTimeoutException
Waiting For Response	OperationCanceledException	UnambiguousTimeoutException
Retry Wait	OperationCanceledException	UnambiguousTimeoutException

Non-Idempotent Operations (i.e. Set, Replace, Touch, etc)

Phase	External Cancellation	Timeout Cancellation
Before Sent	OperationCanceledException	UnambiguousTimeoutException
Waiting For Response	Wait for success/failure	AmbiguousTimeoutException
Retry Wait	OperationCanceledException	UnambiguousTimeoutException

Of particular note is the handling of external cancellation that occurs while waiting for a response from the server. In this scenario, we don't want to throw an OperationCancelledException immediately because it would be ambiguous whether or not the operation may have been completed. Instead, once the operation is sent to the server, we always wait for a response back from the server (up to the timeout). The success/failure will be returned immediately without further retry attempts.

This is in keeping with the idea of a key/value operation being an atomic operation. However, there may be corner cases where the consumer doesn't want this behavior. In this case, they could use other patterns to enforce instant cancellation.

// Example consumer code that will instantly cancel if that edge case is required

var result = collection.UpsertAsync(key, doc, new UpsertOptions().CancellationToken(cancellationToken));
var tcs = new TaskCompletionSource<bool>();
cancellationToken.Register(() => tcs.SetCancelled(cancellationToken));
await Task.WhenAny(result, tcs.Task);