1.
Exploring .NET
memory management
A trip down memory lane
Maarten Balliauw
@maartenballiauw
—

2.
Garbage Collector

3.
The goal of managed memory
“Virtually unlimited memory for our applications”
.NET CLR manages a big chunk of pre-allocated memory
Allocations in that chunk
Garbage Collector (GC) reclaims unused memory

4.
.NET memory management 101
Memory allocation
Objects allocated in “managed heap” (big chunk of memory)
Cheap and fast, it’s just adding a pointer
Memory release or “Garbage Collection” (GC)
Generations
Large Object Heap

5.
.NET memory management 101
Memory allocation
Memory release or “Garbage Collection” (GC)
GC releases memory that’s no longer in use
Expensive!
Pause application
Build a graph of objects
Remove unreachable objects
Compact memory
Generations
Large Object Heap

6.
.NET memory management 101
Memory allocation
Memory release or “Garbage Collection” (GC)
Generations
Large Object Heap
Generation 0 Generation 1 Generation 2
Short-lived objects (e.g. Local
variables)
In-between objects Long-lived objects (e.g. App’s
main form)

7.
.NET memory management 101
Memory allocation
Memory release or “Garbage Collection” (GC)
Generations
Large Object Heap (LOH)
Large objects (>85KB)
Collected only during full garbage collection
Not compacted (by default)
Fragmentation can cause OutOfMemoryException

8.
The .NET garbage collector
Runs often for gen0, less often for higher generations
Background GC (enabled by default)
Concurrent with application threads, pauses usually just for one thread
When does it run? Vague… But usually:
Out of memory condition – when the system fails to allocate or re-allocate memory
After some significant allocation – if X memory is allocated since previous GC
Profiler, forced, internal events
GC is not guaranteed to run
http://blogs.msdn.com/b/oldnewthing/archive/2010/08/09/10047586.aspx
http://blogs.msdn.com/b/abhinaba/archive/2008/04/29/when-does-the-net-compact-framework-garbage-collector-run.aspx

9.
Throughput
More allocations, more garbage collections
Garbage collection means pauses happen
Pauses are bad
Less powerful devices (mobile)
Games (lag)
Server (“resource sharing” with others)
Trading
…

10.
https://www.youtube.com/watch?v=BTHimgTauwQ – © Age of Ascent

11.
Can we help the GC avoid pauses?
Allocating is cheap, collecting is expensive
Use struct when it makes sense, Span<T>, ValueTuple<T>, object pooling, …
Make use of IDisposable / using statement & clean up manually
Weak references
Allow the GC to collect these objects, no need for checks
Finalizers
Beware! Moved to finalizer queue -> always gen++

12.
Helping the GC
DEMO
https://github.com/maartenba/memory-demos

13.
Allocations

14.
Types
REFERENCE TYPES
Variable is a pointer to an object
Passed around by reference
Assignment copies the reference
Allocated on heap
GC involved, plenty of space
VALUE TYPES
Variable is the value
Passed around by value (copied)
Assignment copies the value
Allocated on stack
No GC involved, limited space
int, bool, struct, decimal, enum, float, byte, long, …class, string, new

15.
Hidden allocations!
Boxing
Lambda’s/closures
Allocate compiler-generated DisplayClass to capture state
Params arrays (depending on compiler version)
Async/await
...
int i = 42;
// boxing - wraps the value type in an "object box"
// (allocating a System.Object)
object o = i;

16.
How to find them?
Intermediate Language (IL)
Profiler
“Heap allocations viewer”
ReSharper Heap Allocations Viewer plugin
Roslyn’s Heap Allocation Analyzer

17.
Hidden allocations
DEMO
https://github.com/maartenba/memory-demos

18.
Measure!
Don’t do premature optimization – measure!
Allocations don’t always matter (that much)
Performance counters:
How frequently are we allocating?
How frequently are we collecting?
What generation do we end up on?
Are our allocations introducing pauses?
Or use a profiler:
www.jetbrains.com/dotmemory (and www.jetbrains.com/dottrace)

19.
Always Be Measuring
DEMO
https://github.com/maartenba/memory-demos

20.
[
{ ... },
{
"name": "Westmalle Tripel",
"brewery": "Brouwerij der Trappisten van Westmalle",
"votes": 17658,
"rating": 4.7
},
{ ... }
]

21.
Object pools / object re-use
Re-use objects / collections (when it makes sense)
Fewer allocations, fewer objects for the GC to clean
Less memory traffic (maybe no need for a full GC)
Object pooling - object pool pattern
System.Buffers.ArrayPool
“Borrow objects that have been allocated before”

22.
Garbage Collector summary
Don’t fear allocations!
GC is optimized for high memory traffic in short-lived objects
Don’t optimize what should not be optimized…
Know when allocations happen
GC is awesome
Gen2 collection that stop the world not so much…
Measure!

23.
Strings

24.
Strings are objects
.NET tries to make them look like a value type, but they are a reference type
Read-only collection of char
Length property
A bunch of operator overloading
Allocated on the managed heap
var a = new string('-', 25);
var b = a.Substring(5);
var c = httpClient.GetStringAsync("http://blog.maartenballiauw.be");

25.
String literals
Are all strings on the heap? Are all strings duplicated?
var a = "Hello, World!";
var b = "Hello, World!";
Console.WriteLine(a == b);
Console.WriteLine(Object.ReferenceEquals(a, b));
Prints true twice. So “Hello World” only in memory once?

26.
Portable Executable (PE)
#UserStrings
DEMO
https://github.com/maartenba/memory-demos

27.
String literals in #US
Compile-time optimization
Store literals only once in PE header metadata stream ECMA-335 standard, section II.24.2.4
Reference literals (IL: ldstr)
var a = Console.ReadLine();
var b = Console.ReadLine();
Console.WriteLine(a == b);
Console.WriteLine(Object.ReferenceEquals(a, b));

28.
String duplicates
Any .NET application has them (System.Globalization duplicates quite a few)
Are they bad?
.NET GC is fast for short-lived objects, so meh.
Don’t waste memory with string duplicates on gen2
(but: it’s okay to have strings there)

29.
String interning
Store (and read) strings from the intern pool
Simply call String.Intern when “allocating” or reading the string
Scans intern pool and returns reference
var url = "https://blog.maartenballiauw.be";
var stringList = new List<string>();
for (int i = 0; i < 1000000; i++)
{
stringList.Add(string.Intern(url + "/"));
}

30.
String interning caveats
Why are not all strings interned by default?
CPU vs. memory
Not on the heap but on intern pool
No GC on intern pool – all strings in memory for AppDomain lifetime!
Rule of thumb
Lot of long-lived, few unique -> interning good
Lot of long-lived, many unique -> no benefit, memory growth
Lot of short-lived -> trust the GC
Measure!

31.
Exploring the heap
for fun and profit

32.
How would you...
…build a managed type system, store in memory, CPU/memory friendly
Probably:
Store type info (what’s in there, what’s the offset of fieldN, …)
Store field data (just data)
Store method pointers
Inheritance information

33.
Managed Heap
(scroll down for more...)

34.
.Managed heap is like a database
Pointer to an “instance”
Instance
Pointer to Runtime Type Information (RTTI)
Field values (which can be pointers in turn)
RunTime Type Information
Interface addresses
Instance method addresses
Static method addresses
…

35.
Theory is nice...
Microsoft.Diagnostics.Runtime (ClrMD)
“ClrMD is a set of advanced APIs for programmatically inspecting a crash dump of
a .NET program much in the same way that the SOS Debugging Extensions (SOS)
do. This allows you to write automated crash analysis for your applications as well
as automate many common debugger tasks. In addition to reading crash dumps
ClrMD also allows supports attaching to live processes.”
“LINQ-to-heap”
Maarten’s definition

36.
ClrMD
DEMO
https://github.com/maartenba/memory-demos

37.
But... Why?
Programmatic insight into memory space of a running project
Unit test critical paths and assert behavior (did we clean up what we expected?)
Capture memory issues in running applications
Other (easier) options in this space
dotMemory Unit (JetBrains)
Benchmark.NET

38.
dotMemory Unit
DEMO
https://github.com/maartenba/memory-demos

39.
Conclusion

40.
Conclusion
Garbage Collector (GC) optimized for high memory traffic + short-lived objects
Don’t fear allocations! But beware of gen2 “stop the world”
Don’t optimize what should not be optimized…
Measure!
Using a profiler/memory analysis tool
ClrMD to automate inspections
dotMemory Unit, Benchmark.NET, … to profile unit tests
Blog series: https://blog.maartenballiauw.be

41.
Thank you!
Maarten Balliauw
@maartenballiauw
—