4. APM pattern
IAsyncResult BeginFoo(..., AsyncCallback callback, object state);
void EndFoo(IAsyncResult iar);
Synchronous call:
Foo();
Achieving the same:
EndFoo(BeginFoo(..., null, null));
Leveraging asynchronous calls:
BeginFoo(..., iar => {
T val = EndFoo(iar);
// do stuff ...
});
5. APM – Example
Copy stream to stream:
int bytesRead;
while ((bytesRead = input.Read(buffer)) != 0) {
output.Write(buffer, 0, bytesRead);
}
6. APM – Nesting problem
BeginRead(..., iar => {
int bytesRead = EndRead(iar);
input.BeginWrite(..., iar2 => {
int bytesWritten2 = EndWrite(iar2);
BeginRead(..., iar3 => {
int bytesRead3 = EndRead(iar3);
BeginWrite(..., iar4 => {
// ... again and again
});
});
});
});
7. APM – IsCompletedSynchronously
IAsyncResult r = BeginRead(..., iar => {
if (!iar.IsCompletedSynchronously) {
// ... asynchronous path as shown earlier
}
});
if (r.IsCompletedSynchronously) {
// ... Synchronous path
}
• Even worse in loop
• Overall very complicated
• Queueing on ThreadPool much simpler
8. EAP:
Event-based Asynchronous Pattern
• .NET Framework 2.0
obj.Completed += (sender, eventArgs) => {
// ... my event handler
}
obj.SendPacket(); // returns void
• Did not solve multiple-calls problem, or loops
• Introduced context
9. Task
• .NET Framework 4.0
• MSR project – parallel computing
• Divide & conquer efficiently (e.g. QuickSort)
• Shaped Task – similar to today
• Task – represents general work (compute, I/O bound, etc.)
= promise / future / other terminology
• Task / Task<T> – operation (with optional result T)
1. T … in the case of Task<T>
2. State related to synchronization
3. State related to callback
10. Task / TaskCompletionSource
• Task
• Here's a callback, invoke it when you're done, or right now if you've already
completed
• I want to block here, until your work is done
• Cannot be completed by user directly
• TaskCompletionSource … wrapper for Task
• Holds Task internally and operates on it via internal methods
• Methods:
• SetResult
• SetException
• SetCancelled
11. Task – Consumption
Task<T> t;
Either:
t.Wait(); // Blocks until Task is completed
Or:
t.ContinueWith(callback); // Will be executed after Task is completed
Even multiple times:
t.ContinueWith(callback2);
t.ContinueWith(callback3);
ContinueWith:
• Does not guarantee order of executions
• Always asynchronous (queued to ThreadPool/scheduler in general)
12. Task.Run
We complicated things
Task<T> Task.Run(delegate d)
• Adds field to Task with ‘d’
• Queues work to ThreadPool
• Thread grabs it, executes it, marks task completed
13. Task.Run implementation
Task<T> Run(Func<T> f) {
var tcs = new TaskCompletionSource<T>();
ThreadPool.QueueUserWorkItem(() => {
try {
T result = f();
tcs.SetResult(result);
} catch (ex) {
tcs.SetException(ex);
}
});
return tcs.Task;
}
14. async-await
.NET Framework 4.5 / C# 5
Example of asynchronous code:
Task<int> GetDataAsync();
Task PutDataAsync(int i);
Code:
Task<int> t = GetDataAsync();
t.ContinueWith(a => {
var t2 = PutDataAsync(a.Result);
t2.ContinueWith(b => Console.WriteLine("done"));
});
15. async-await
Task<int> t = GetDataAsync();
t.ContinueWith(a => {
var t2 = PutDataAsync(a.Result);
t2.ContinueWith(b => Console.WriteLine("done"));
});
C# 5 with async-await helps us:
Task<int> t = GetDataAsync();
int aResult = await t;
Task t2 = PutDataAsync(aResult);
await t2;
Console.WriteLine("done");
16. Awaiter pattern
int aResult = await t;
Translated to:
var $awaiter1 = t.GetAwaiter();
if (! $awaiter1.IsCompleted) { // returns bool
// ...
}
int aResult = $awaiter1.GetResult(); // returns void or T
// If exception, it will throw it
17. Awaiter pattern – details
void MoveNext() {
if (__state == 0) goto label0;
if (__state == 1) goto label1;
if (__state == 42) goto label42;
if (! $awaiter1.IsCompleted) {
__state = 42;
$awaiter1.OnCompleted(MoveNext);
return;
}
label42:
int aResult = $awaiter1.GetResult();
}
18. State Machine
string x = Console.ReadLine();
int aResult = await t;
Console.WriteLine("done" + x);
State machine:
struct MethodFooStateMachine {
void MoveNext() { ... }
local1; // would be ‘x’ in example above
local2;
params;
_$awaiter1;
}
19. State Machine – Example
public async Task Foo(int timeout) {
await Task.Delay(timeout);
}
public Task Foo(int timeout) {
FooStateMachine sm = default;
sm._timeout = timeout;
sm._state = 0;
sm.MoveNext();
return ???;
}
struct FooStateMachine {
int _timeout; // param
// locals would be here too
void MoveNext() { ... }
int _state;
TaskAwaiter _$awaiter;
}
20. State Machine – Example
public Task Foo(int timeout) {
FooStateMachine sm = default;
sm._tcs = new TaskCompletionSource();
sm._timeout = timeout;
sm._state = 0;
sm.MoveNext();
return sm._tcs.Task;
}
AsyncValueTaskMethodBuilder.Create();
_tcs.Task -> _builder.Task;
struct FooStateMachine {
int _timeout; // param
// locals would be here too
void MoveNext() {
// ...
_tcs.SetResult(...);
}
int _state;
TaskAwaiter _$awaiter;
TaskCompletionSource _tcs;
}
21. State Machine – Summary
What about Task allocation?
• Builder can reuse known tasks
• Task.CompletedTask (without value)
• boolean – True/False
• int … <-1,8>
• LastCompleted (e.g. on MemoryStream)
• Does not work on SslStream (alternates headers and body)
• Size: 64B (no value) / 72B (with value)
• Azure workloads OK (GC will collect)
• Hot-path: up to 5%-10% via more GCs
22. ValueTask
• .NET Core 2.0
• Also as nuget package down-level
struct ValueTask<T> {
T;
Task<T>;
}
• Only one of them: T+null or default+Task<T>
• NET Core 2.1
ValueTask<int> Stream.ReadAsync(Memory<byte>, ...)
23. ValueTask – Can we improve more?
• What about the 1% asynchronous case?
• .NET Core 2.1
struct ValueTask<T> {
T;
Task<T>;
IValueTaskSource<T>;
}
struct ValueTask {
Task;
IValueTaskSource;
}
Based on internal talk from author of Task, async and all good things around – Stephen Toub, architect of BCL
IAsyncResult
AsyncWaitHandle – ManualResetEvent or AutoResetEvent
Across BCL
Usage either:
Wait for callback to be called, or
Call EndFoo which will block until completed
Single operation works fine, but in reality you do more – e.g. in a loop
Manually it does not work – somehow turn it into loop
It’s possible but extremely long and tricky
Further complications with IsCompletedSynchronously
For perf reasons
In the loop it is even more complicated
However:
On MemoryStream, the data is already available … instead of ThreadPool, call delegate immediately
-> Leads to recursive calls -> 10K StackOverflow
Bottom part:
Even BCL lots of wrappers (e.g. in Networking: LazyAsyncResult) with lots of specializations
Very complicated
Straightforward idea – Completed event
Kick off operation, then Completed handler is invoked (generally on ThreadPool)
5-10 classes in BCL … like SmtpMail, TcpClient, BackgroundWorker
Downsides:
We shipped it in .NET Framework 2.0 and quickly realized that it is interesting experiment, but not exactly addressing real needs
90% right, 10% keeps Toub awake at night even after 10 years and would love to change it
NOT tied to ThreadPool – not tied to executing delegate
Shove result into it
Can be completed
Can wake up someone waiting on it
Task – something to consume - hook up to, not to change directly (no control)
TaskCompletionSource – can alter state of Task … has control over Task
Lazy initialization (something over network) … you are in charge who can change complete the work
Wait - creates ManualResetEvent which will be signaled when one of the SetResult/SetException/SetCancelled is called
Option: TaskExecutionOption to do it synchronously
APM - IAsyncResult … no shared implementation
Everyone had to have their own implementation
Task model - you don't pass delegate at creation, but you can walk up on any of them and say "call me when you're done"
Abstractions enabled - like async-await
await hooks up the callback
Sets completed = execute callback, waking up things waiting on it, etc.
TaskCompletionSource creates Task
Returns Task to be awaited on, etc.
Now we implemented Task.Run without storing any delegate on Task
GetData/PutData … maybe across the wire
Compiler translates it to the code above (hand-waving involved)
Compiler does not treat Task specially, but it just looks for pattern (awaiter pattern)
Bold methods are pattern matching
“! IsCompleted” part is complicated – I have to hook up code that comes back here when task completes
All of it is part of MoveNext method -- it is a state machine, every await in method is state in state machine (hand waving a bit)
OnCompleted has slightly more complicated signature
How does ‘x’ survive continuation? (it is just on stack) – need to capture it
Same in continuations - C# compiler lifts it to keep it on heap allocated object (floats through closures)
Same in state machine
Compiler optimizes - stores here things only crossing await boundary
In debug -- struct is class -- for debuggability, but for perf struct
Why? These async methods often complete synchronously – example:
BufferedStream … large buffer behind with inner stream
If I ask for 1B, but it reads 10K in bulk, then lots of calls end up synchronously
If it was class, then we would allocate per call
_tcs on state machine is logically there
problem 2 allocations – TaskCompletionSource and Task (inside)
For the synchronous case we want 0 allocations ideally (BufferedStream example)
Even the Task/TaskCompletionSource is problematic, because it is anything Task-like
Each Task-like type (except Task) has attribute defining builder - builder pattern
ValueTask has one -> AsyncValueTaskMethodBuilder
instead of new TaskCompletionSource() -> AsyncValueTaskMethodBuilder.Create();
We have internally in System.Runtime.CompilerServices structs:
AsyncTaskMethodBuilder, AsyncTaskMethodBuilder<T>, AsyncVoidMethodBuilder
instead of _tcs.Task -> _builder.Task
We eliminated TaskCompletionSource allocation
What about the Task?
Methods are 1-liners (if Task<T> == null, do something, else something else)
Nicely handles synchronously completing case
Note: Non-generic ValueTask does not make sense - only Task inside … we have CompletedTask
.NET Core 2.1
Luckily we introduced Memory<T> at the same time, as we cannot overload on return type
That's why sometimes in PRs we wrap byte[] in Memory first … to use the ValueTask
Design-guidelines: Start with Task … use ValueTask only in hot-path scenarios
IValueTaskSource – complicated interface
almost the awaiter pattern:
Are you completed?
Hook up a call back
Get a result
All implementations on ValueTask are now ternary
Value: You can implement it however you want, incl. reset (reuse)
Complicated to do, so not everywhere
Socket.SendAsync/ReceiveAsync
Object per send/receive … if one at a time (typical)
0 allocation for loop around Send/Receive on Socket
Same: NetworkStream, Pipelines, Channels