Collections and LINQ
Which collection to pick, what happens inside List<T> and Dictionary, when a LINQ query actually runs, and why IQueryable<T> is dangerous in EF Core. These questions come up in almost every interview and quickly show whether a candidate understands the cost of their code.
Before this topic: Types and memory, The C# language.
What collections does .NET have, and how do you choose one?
Section titled “What collections does .NET have, and how do you choose one?”What it is
A collection holds a group of items. Operation speed is written with O: O(1) time does not depend on the number of items, O(n) grows in proportion to it, O(log n) grows very slowly.
How to use
| Task | Collection | Main operation |
|---|---|---|
| Fixed-size set | array T[] |
index — O(1) |
| Growing list | List<T> |
Add and index — O(1) |
| Value by key | Dictionary<TKey, TValue> |
lookup — O(1) on average |
| Unique values | HashSet<T> |
Contains — O(1) on average |
| Queue: first in, first out | Queue<T> |
Enqueue, Dequeue — O(1) |
| Stack: last in, first out | Stack<T> |
Push, Pop — O(1) |
| Keys in order | SortedDictionary<TKey, TValue> |
lookup — O(log n) |
Common mistake
Keeping everything in a List<T> and calling Contains in a loop. Each check is O(n), while HashSet<T> does it in O(1).
Takeaway: choose the collection by the operation the code performs most often.
How it works
List<T>,Queue<T>andStack<T>wrap an array; the queue uses it as a circular buffer.Dictionary<TKey, TValue>andHashSet<T>are hash tables: an item’s slot is computed from the key’s hash code.SortedDictionary<TKey, TValue>andSortedSet<T>are a red-black tree.SortedList<TKey, TValue>is two sorted arrays: keys and values.LinkedList<T>is a doubly linked list of separate nodes.PriorityQueue<TElement, TPriority>(.NET 6+) is an array-backed heap: the item with the lowest priority comes out first.
When to choose what
| Pair | First wins | Second wins |
|---|---|---|
List<T> / LinkedList<T> |
almost always | O(1) insert if the node is already found |
SortedList / SortedDictionary |
less memory, data arrives sorted | frequent inserts in random order |
Dictionary / SortedDictionary |
O(1) lookup | you need to walk keys in order |
Pitfalls
List<T>.Insert(0, x)andRemoveshift elements — O(n).- The enumeration order of
DictionaryandHashSetis not guaranteed. LinkedList<T>has no index: finding a node is O(n).
Rule of thumb
Start with List<T> or Dictionary. Reach for a specialized collection when you can name the operation it speeds up.
Takeaway: pick a collection by its dominant operation and that operation’s complexity, not by habit.
Under the hood
- An array and a
List<T>are one contiguous block of memory. Iteration walks neighbouring addresses and hits the CPU cache well. - Node-based collections (
LinkedList<T>,SortedDictionary,SortedSet) create an object per item: a header, references to neighbours, scattered across the heap. - In a
List<T>of structs the values live right inside the array, with no boxing.
Cost
- O(1) for a hash table is the average case. A bad hash or crafted collisions turn lookup into O(n).
- For small n, constants matter more than asymptotics. A linear scan over a dozen items can beat hashing.
- Every node is an allocation and work for the GC; with millions of items it shows up in collection pauses.
Trade-offs
| Scenario | Choice |
|---|---|
| Built at startup, read-only afterwards | FrozenDictionary, FrozenSet (.NET 8+) |
| Insertion order plus key lookup | OrderedDictionary<TKey, TValue> (.NET 9+) |
| Prioritized work items | PriorityQueue<TElement, TPriority> |
| Access from several threads | System.Collections.Concurrent |
How to check in production
Hot collection operations show up in a CPU profile (dotnet-trace, PerfView). Node allocations are visible in BenchmarkDotNet with [MemoryDiagnoser] on realistic data sizes.
Takeaway: asymptotics drive the choice, while memory locality and allocations decide how fast it really is.
var list = new List<int> { 3, 1, 2 };var byId = new Dictionary<int, string> { [1] = "Alice", [2] = "Bob" };var seen = new HashSet<string>();var queue = new Queue<string>();var stack = new Stack<string>();var sorted = new SortedDictionary<string, int>();
Console.WriteLine(list.Contains(2)); // True — linear search, O(n)if (byId.TryGetValue(2, out var name)) Console.WriteLine(name); // Bob — key lookup, O(1)Console.WriteLine(seen.Add("x")); // TrueConsole.WriteLine(seen.Add("x")); // False — duplicate not added
queue.Enqueue("a"); queue.Enqueue("b");stack.Push("a"); stack.Push("b");Console.WriteLine($"{queue.Dequeue()} {stack.Pop()}"); // a b
sorted["b"] = 2; sorted["a"] = 1;Console.WriteLine(string.Join(",", sorted.Keys)); // a,b — keys in order- Why is
LinkedList<T>rarely faster thanList<T>? — Finding a node is O(n), nodes are scattered across the heap, no locality. - How does
SortedListdiffer fromSortedDictionary? — Arrays versus a tree: less memory, but O(n) inserts. - In what order does a
Dictionaryenumerate? — The order is not guaranteed; if you need order, useSortedDictionaryorOrderedDictionary<TKey, TValue>. - When is
HashSet<T>better thanList<T>.Contains? — When you check membership often: O(1) versus O(n). - What type should a method parameter accept? — The narrowest interface that is enough:
IEnumerable<T>,IReadOnlyCollection<T>,IReadOnlyList<T>.
How does List<T> grow, and what is Capacity?
Section titled “How does List<T> grow, and what is Capacity?”What it is
Countis how many items are in the list.Capacityis the length of the internal array, never less thanCount.
Why
An array cannot be extended. So List<T> keeps spare room and grows when that room runs out.
How to use
- While
Countis less thanCapacity,Addjust writes the item into the array. - When the array is full, the list creates a new one, about twice as large, and copies the items.
- If you know the size, pass it to the constructor:
new List<Order>(orders.Length). Clear()resetsCountbut keepsCapacity. You can shrink the array withTrimExcess().
Common mistake
Treating Capacity as a size limit. The list grows on its own; Capacity only shows how much room is already allocated.
Takeaway: Count is how many items there are, Capacity is how many fit without copying.
How it works
new List<T>()points to a shared empty array and allocates nothing.- The first
Addcreates an array of 4 items. - Each subsequent growth doubles the length: 8, 16, 32 and so on.
- If more is needed at once (
AddRangeof a large collection), the required size is used.
With doubling, n additions copy fewer than n items in total. So Add is O(1) on average, even though a single call can take O(n).
When to choose what
| Situation | What to do |
|---|---|
| Size is known | new List<T>(n) or EnsureCapacity(n) (.NET 6+) |
Source is an ICollection<T> |
new List<T>(source) or AddRange: one allocation plus CopyTo |
| The list will not grow again | TrimExcess() — works if less than 90 % is used |
Pitfalls
Insert(0, x)andRemoveAt(0)shift the whole array — O(n). For a queue there isQueue<T>.RemoveandCleardo not reduceCapacity.- Cap any capacity that comes from user input, or a hostile request can allocate a gigantic array.
Rule of thumb
Know the size — set it right away. Don’t know it — don’t guess, and let the list grow.
Takeaway: doubling makes Add cheap on average, and a preset capacity removes copying entirely.
Under the hood
- The
DefaultCapacity = 4constant and the2 * _items.Lengthformula are CoreLib implementation details. Code must not rely on them. - Growth is capped by
Array.MaxLength, and the new capacity is never below the required one. CollectionsMarshal.AsSpan(list)(.NET 5+) returns aSpan<T>over the internal array without copying. Do not change the list’s size while the span is in use.CollectionsMarshal.SetCount(.NET 8+) setsCountdirectly. When increasing it, it exposes uninitialized slots.
Cost
- During growth both arrays are alive. For a
List<long>with a million items the last growth allocates 8 MB while 4 MB are still live. - Arrays of 85,000 bytes or more go to the LOH. It is not compacted by default, which leads to fragmentation and a larger working set.
- Right after growth up to half of the array is empty.
Trade-offs
- The factor of 2 balances the number of copies against wasted memory.
- Reusing a list via
Clear()saves allocations but holds on to a large array. - For temporary buffers on a hot path, an array pool such as
ArrayPool<T>.Sharedfits better.
How to check in production
dotnet-counters shows the allocation rate and LOH size, and a dotnet-trace trace shows where allocations come from. In BenchmarkDotNet the cost of growth appears in the Allocated column with [MemoryDiagnoser].
Takeaway: list growth means copying and garbage, so on hot paths set the capacity up front.
var list = new List<int>();Console.WriteLine(list.Capacity); // 0 — no array allocated yet
int last = -1;for (int i = 0; i < 20; i++){ list.Add(i); if (list.Capacity != last) { Console.WriteLine($"Count={list.Count}, Capacity={list.Capacity}"); last = list.Capacity; }}// Count=1, Capacity=4// Count=5, Capacity=8// Count=9, Capacity=16// Count=17, Capacity=32
list.TrimExcess(); // 20 is below 90 % of 32 — the array shrinksConsole.WriteLine(list.Capacity); // 20
var presized = new List<int>(1000); // one allocation instead of nine arraysConsole.WriteLine(presized.Capacity); // 1000- What is
Capacityafter adding 1025 items with no initial capacity? — 2048: almost half the array is empty. - Does
Capacityshrink afterRemoveorClear? — No, only afterTrimExcessor assigningCapacityexplicitly. - Why is
Insert(0, x)slow? — It shifts the whole array, O(n); for a queue there isQueue<T>. - How many allocations does
new List<T>(array)make? — One: for anICollection<T>theCountis known and items are copied withCopyTo. - What is dangerous about
CollectionsMarshal.AsSpan? — If the list grows, the span keeps pointing at the old array.
What is deferred execution in LINQ, and why is enumerating a query twice dangerous?
Section titled “What is deferred execution in LINQ, and why is enumerating a query twice dangerous?”What it is
A LINQ query is a recipe, not a result. Until it is enumerated, not a single item has been read.
Why
You can build a query piece by piece without spending memory on intermediate lists.
How to use
| Kind of method | Examples | When it runs |
|---|---|---|
| Deferred | Where, Select, OrderBy, Take |
when the result is enumerated |
| Immediate | Count(), First(), Sum(), Any() |
right away |
| Materializing | ToList(), ToArray(), ToDictionary() |
right away; the result is stored |
Common mistake
Calling Count() and then foreach over the same query. The query runs twice. If you need the result several times, store it with ToList().
Takeaway: a query runs on every enumeration, not when it is declared.
How it works
- A deferred operator returns an object that remembers the source and the lambda.
foreachasks for items one at a time, and each operator in the chain processes them in turn.- Streaming operators (
Where,Select,Take) yield an item right away. - Non-streaming ones (
OrderBy,GroupBy,Reverse) read the whole source into a buffer on the first request.
When to choose what
| Situation | Decision |
|---|---|
| The result is read once | keep it lazy |
| Needed several times, or you need a snapshot | ToList() or ToArray() |
Data is needed after the DbContext or file is closed |
materialize before closing |
Pitfalls
- A closure captures the variable, not its value. Change the variable before enumeration and the query uses the new value.
- An exception from a lambda is thrown during enumeration, not on the line with
Where. - Generators (
Random,DateTime.Now, network reads) give a different result on a second pass. - Modifying a
List<T>while enumerating it throwsInvalidOperationException.
Rule of thumb
If you accept an IEnumerable<T>, enumerate it exactly once. Need more — materialize at the start of the method.
Takeaway: a lazy query runs as many times as it is enumerated.
Under the hood
- Each operator wraps its source in an object that yields items one at a time via
MoveNextandCurrent. - These internal LINQ classes (in .NET 9 they were consolidated under a common
Iterator<TSource>base) know each other’s shape.Where(...).Select(...)fuses into one object, andOrderBy(...).First()finds the minimum in O(n) without sorting. List<T>keeps a version number.AddorRemoveduring enumeration changes it, and the nextMoveNextthrows.
Cost
- A second pass repeats everything: delegates, sorting, the SQL query.
- An EF Core query returned as
IEnumerable<T>runs after the context’sDisposeand throwsObjectDisposedException. - Side effects in lambdas fire on every pass and only for the items actually read.
Trade-offs
| Approach | Pro | Con |
|---|---|---|
| Lazy result | O(1) memory, early exit | repeated work, lifetime surprises |
| Materialization | one pass, a data snapshot | memory for the whole result |
If you return a lazy result, document it. If you need Count and several passes, accept IReadOnlyCollection<T>.
How to check in production
- Analyzer CA1851 (since .NET 7, off by default) finds possible multiple enumeration.
- Repeated SQL queries show up in EF Core logs.
- In a CPU profile, extra passes look like unexpectedly frequent lambda calls.
Takeaway: laziness saves memory but hands control over when and how often the query runs to the consumer.
var numbers = new List<int> { 1, 2, 3 };var calls = 0;
IEnumerable<int> query = numbers.Where(n =>{ calls++; return n > 1;});
numbers.Add(4); // the query has not run yetConsole.WriteLine(query.Count()); // 3 — sees the 4 added after declarationConsole.WriteLine(query.Max()); // 4 — a second pass over the sourceConsole.WriteLine(calls); // 8 — the predicate ran twice per item
var snapshot = query.ToList(); // materialization: one pass, then a ready listnumbers.Add(5);Console.WriteLine(snapshot.Count); // 3 — the snapshot does not change- When does
OrderByactually run? — On the firstMoveNext: it reads the whole source and sorts. - How does
ToList()differ fromAsEnumerable()? — The first runs the query and stores the result; the second runs nothing and only changes the static type. - What happens if you return an EF Core query from a method with
using var db? — Enumerating after the context’sDisposethrowsObjectDisposedException. - Why is an exception from
Selectthrown somewhere other than theSelectline? — The lambda runs during enumeration. - How do you stop callers from running an expensive query twice? — Return a materialized collection, such as
IReadOnlyList<T>.
How does Dictionary<TKey, TValue> work, and why must Equals and GetHashCode agree?
Section titled “How does Dictionary<TKey, TValue> work, and why must Equals and GetHashCode agree?”What it is
Dictionary<TKey, TValue> stores key-value pairs and finds a value by key in O(1) on average. Keys are unique and cannot be null.
How to use
dict[key] = valueadds a pair or overwrites the value.Addwith an existing key throwsArgumentException.- Reading a missing key through the indexer throws
KeyNotFoundException, so read withTryGetValue.
Common mistake
Overriding only Equals in a key class. The dictionary compares hash codes from GetHashCode first, and two “equal” objects with different hashes never find each other. The easiest fix is a record: it generates both methods for you.
Takeaway: a dictionary searches by GetHashCode and confirms with Equals, so the two must agree.
How it works
- The key’s
GetHashCodereturns a number — the hash code. - The remainder of the hash divided by the table size picks a bucket.
- The dictionary walks that bucket’s chain of entries: it compares stored hashes and calls
Equalswhen they match. - The comparison is done by an equality comparer,
IEqualityComparer<TKey>. By default it isEqualityComparer<TKey>.Default, which usesIEquatable<T>if the type implements it.
The contract: if a.Equals(b), the hashes are equal. The reverse is not required: different keys with the same hash are ordinary collisions.
When to choose what
| Key | How to define equality |
|---|---|
| Your own immutable type | record, record struct, or IEquatable<T> plus HashCode.Combine |
| A type you cannot change | your own IEqualityComparer<TKey> passed to the constructor |
| Case-insensitive strings | StringComparer.OrdinalIgnoreCase |
Pitfalls
- A mutable key: change a field the hash depends on and the entry stays in the dictionary but becomes unreachable.
EqualswithoutGetHashCodeonly produces compiler warning CS0659.ContainsKeyfollowed by the indexer is two lookups instead of one (CA1854). UseTryGetValue.Addduringforeachover a dictionary throwsInvalidOperationException, whileRemovehas been allowed since .NET Core 3.0.
Rule of thumb
Keys are immutable, and Equals and GetHashCode use the same fields.
Takeaway: the hash picks the bucket, Equals confirms the match, and a bug in either one breaks lookup.
Under the hood
int[] _bucketsstores the index of the first entry in each chain, andEntry[] _entriesstores entries{ hashCode, next, key, value }.- Chains are indices inside a single array, not node objects. This is separate chaining with no per-item allocation and good locality.
- The table size is a prime. When
_entriesis full, arrays of at least double the size are created and every entry is redistributed. - In a 64-bit process the modulo is replaced with a multiplication (
FastMod). - Removed entries go to a free list and are taken by the next
Addcalls.
Cost
- Growth is O(n) plus an allocation spike, so a known size is passed to the constructor.
- Enumeration order is slot order. Without removals it matches insertion order; after removals it does not.
- A bad hash piles keys into one chain, and lookup becomes O(n).
Trade-offs
string.GetHashCodeis randomized per process. That protects against HashDoS — an attack that crafts collisions.Dictionary<string, T>with the default or an ordinal comparer starts with a faster non-randomized hash. If a chain grows longer than 100 during an insert, it switches to the randomized hash and rehashes the table.- Your own key types get no such protection: a predictable hash over user input is a vulnerability.
How to check in production
Signs of a bad hash are expensive TryGetValue calls and many Equals calls in a CPU profile (dotnet-trace, PerfView). Check the hash distribution with a test and the speed with BenchmarkDotNet.
Takeaway: a Dictionary is fast as long as the hash spreads keys well and the keys themselves do not change.
var key = new MutableKey { Id = 1 };var dict = new Dictionary<MutableKey, string> { [key] = "first" };
key.Id = 2; // the key's hash changedConsole.WriteLine(dict.ContainsKey(key)); // False — the entry sits under the old hashConsole.WriteLine(dict.ContainsKey(new MutableKey { Id = 1 })); // False — and Equals no longer matchesConsole.WriteLine(dict.Count); // 1 — the entry exists but is unreachable
var prices = new Dictionary<Sku, decimal> { [new Sku(1, "A-100")] = 9.99m };Console.WriteLine(prices.ContainsKey(new Sku(1, "A-100"))); // True — record struct provides Equals and GetHashCode
class MutableKey{ public int Id { get; set; } public override bool Equals(object? obj) => obj is MutableKey k && k.Id == Id; public override int GetHashCode() => Id.GetHashCode();}
readonly record struct Sku(int ShopId, string Code);- What happens if you override
Equalsbut notGetHashCode? — Equal objects land in different buckets and are not found; the compiler warns with CS0659. - Can
GetHashCodereturn a constant? — It is correct, but every key ends up in one chain: O(n) lookup. - Why is a dictionary’s enumeration order not guaranteed? — New entries take the slots of removed ones.
- Why is
ContainsKeyplus the indexer worse thanTryGetValue? — Two lookups instead of one (CA1854). - How do you make a dictionary with case-insensitive keys? — Pass
StringComparer.OrdinalIgnoreCaseto the constructor.
How does IEnumerable<T> differ from IQueryable<T>?
Section titled “How does IEnumerable<T> differ from IQueryable<T>?”What it is
IEnumerable<T>means “I can iterate items”: lists, arrays, the result ofyield return.IQueryable<T>means “I can build a query against a source”. In EF Core,DbSet<User>implementsIQueryable<User>.
Why
To filter data where it lives. db.Users.Where(u => u.IsActive) becomes SQL with a WHERE, and only active users come back from the database.
How to use
- Build the query on
IQueryable<T>:Where,OrderBy,Select. - Execute it at the end:
ToListAsync(),FirstOrDefaultAsync().
Common mistake
Calling ToList() or AsEnumerable() before the filter. Then the whole table is loaded and Where runs in memory.
Takeaway: IEnumerable<T> runs LINQ in your process, IQueryable<T> runs it in the data source.
How it works
- A lambda compiles differently depending on the parameter type: into executable code for
Func<T, bool>, into a tree of objects you can inspect forExpression<Func<T, bool>>. IQueryable<T>inheritsIEnumerable<T>. WhichWhereis called is decided at compile time by the variable’s static type.- On enumeration the EF Core provider translates the tree into SQL. Captured variables become SQL parameters.
When to choose what
| Query step | Where it runs |
|---|---|
Where, OrderBy, Select before materialization |
in the database |
Everything after AsEnumerable() or ToList() |
in memory |
An untranslatable method in the final Select |
in memory, only that part |
Pitfalls
- A method taking a
Func<User, bool>callsEnumerable.Whereeven on aDbSet, and the whole table is loaded. - Since EF Core 3.0 an untranslatable expression in
WhereorOrderBythrows. Client evaluation is allowed only in the finalSelect. - String comparison and
nullhandling are defined by the database, not by C#.
Rule of thumb
Accept filters as Expression<Func<T, bool>>, and switch to memory explicitly and as late as possible.
Takeaway: the static type of the query decides whether LINQ runs in the database or in memory.
Under the hood
- The compiler builds the tree with calls such as
Expression.LambdaandExpression.Propertyright at the query site — allocations on every execution. Queryable.Whereadds a method-call node and callsProvider.CreateQuery.- On enumeration EF Core looks the query up in its cache by tree shape. On a miss it translates the tree into SQL and compiles the materialization code.
Cost
- Constants baked into the tree when building it dynamically with
Expression.Constantproduce a new shape for every value. That means recompilation in EF and a polluted plan cache in the database. AsQueryable()over an in-memory collection gives anEnumerableQuery<T>: the tree is compiled into a delegate at execution time, which is slower than plain LINQ to Objects.
Trade-offs
EF.CompileQueryandEF.CompileAsyncQueryremove building and comparing the tree. Limits: a single model and simple scalar parameters.- Returning
IQueryable<T>from a repository is flexible, but storage details and theDbContextlifetime leak out. A common compromise is materialized DTOs or specifications built onExpression<Func<T, bool>>.
How to check in production
ToQueryString()and logging viaLogToshow the generated SQL.- An EF Core Query Cache Hit Rate metric that stays below 100 % means something is defeating the query cache.
Takeaway: IQueryable<T> moves work to the database but requires watching the tree’s shape and the server-client boundary.
using System.Linq.Expressions;
IQueryable<int> source = new[] { 1, 2, 3, 4 }.AsQueryable();
Expression<Func<int, bool>> expr = n => n > 2; // expression tree: code as dataFunc<int, bool> func = n => n > 2; // a plain delegate
IQueryable<int> q1 = source.Where(expr); // Queryable.Where: the filter is part of the queryIEnumerable<int> q2 = source.Where(func); // Enumerable.Where: the filter runs in memory
Console.WriteLine(q1.Expression); // System.Int32[].Where(n => (n > 2))Console.WriteLine(q2 is IQueryable<int>); // False — the provider knows nothing about the filterusing System.Linq.Expressions;
// db is a DbContext with DbSet<User> Users// 1. Filter in SQL: SELECT ... WHERE IsActivevar active = await db.Users.Where(u => u.IsActive).ToListAsync();
// 2. Filter in memory: the whole table is read from the databasevar slow = db.Users.AsEnumerable().Where(u => u.IsActive).ToList();
// 3. The same trap, but hidden: Func picks Enumerable.WhereList<User> Find(Func<User, bool> filter) => db.Users.Where(filter).ToList();
// 4. Correct: Expression keeps the query IQueryableList<User> FindQ(Expression<Func<User, bool>> filter) => db.Users.Where(filter).ToList();
Console.WriteLine(db.Users.Where(u => u.IsActive).ToQueryString()); // the SQL text- What runs on the server in
db.Users.AsEnumerable().Where(...)? — Only aSELECTof the whole table; the filter runs in memory. - Why does a method with a
Func<T, bool>parameter load the whole table? — AFuncis not an expression tree, soEnumerable.Whereis chosen. - What does EF Core do with a local variable used in a lambda? — Turns it into a SQL parameter, so the query plan is reused.
- How do you see the generated SQL? —
ToQueryString()or logging viaLogTo. - Should a repository return
IQueryable<T>? — It is flexible, but storage details and the context lifetime leak out.
How does yield return work, and what does the compiler generate?
Section titled “How does yield return work, and what does the compiler generate?”What it is
An iterator is a method that returns IEnumerable<T> and produces items with yield return. yield break ends the sequence early.
Why
- No intermediate list: items are produced on demand.
- You can describe an infinite sequence and take only the beginning with
Take.
How to use
- Calling the method only creates a sequence object.
foreachruns the body up to the firstyield returnand receives an item.- The next loop step resumes the body where it stopped.
- A new
foreachstarts everything from the beginning.
Common mistake
Expecting an argument check at the top of an iterator to fire on the call. The exception is thrown only on the first enumeration.
Takeaway: an iterator is a lazy method that runs in chunks between yield return statements.
How it works
- The compiler creates a nested class that implements
IEnumerable<T>,IEnumerator<T>andIDisposable. - Parameters and locals that live across a
yieldbecome its fields. - An enumerator is an object with
MoveNext()andCurrent. HereMoveNext()is aswitchon the state number: a jump to the point after the lastyield return. Dispose()runsfinallyandusingblocks if enumeration was interrupted.foreachalways calls it, even onbreak.Reset()on the generated enumerator throwsNotSupportedException.
When to choose what
| Task | Tool |
|---|---|
| A lazy in-memory sequence | IEnumerable<T> with yield return |
| Items arrive asynchronously (network, database) | async IAsyncEnumerable<T> and await foreach (C# 8+) |
| The result is needed in full and several times | a method returning List<T> |
The asynchronous version is called an async stream; cancellation is passed with WithCancellation.
Pitfalls
yieldis not allowed in methods withref,outorinparameters, in lambdas, incatchandfinallyblocks, or in atrythat has acatch. Atrywith only afinallyis fine.- Calling
GetEnumerator()by hand withoutusingskipsDispose, andfinallydoes not run.
Rule of thumb
Split an iterator into two methods: a public one validates arguments immediately and returns the result of a private iterator, for example a local function.
Takeaway: an iterator is a compiler-generated state machine class, and all its effects happen during enumeration.
Under the hood
- Fields get unspeakable names:
<>1__state(state),<>2__current(current item),<>l__initialThreadId(creating thread). - The original parameter values are kept separately, for example
<>3__max, so every new enumerator starts from them. - The first
GetEnumeratoron the creating thread returns the object itself. A repeated call, or a call from another thread, creates a copy, so parallelforeachloops over one result do not interfere.
Cost
- Every iterator call is an object allocation.
- Every item is two interface calls:
MoveNextandCurrent. - A resource in a
usinginside an iterator stays alive until the consumer reads to the end or callsDispose.
Trade-offs
- BCL collections hand-write their enumerators as structs:
foreachover aList<T>does not allocate when the type is known statically. - In .NET 9 the group enumerator behind
GroupByandToLookupwas rewritten by hand: compiler-generated code was slower on that hot path. - Since C# 13 iterators can declare
reflocals and useunsafeblocks, as long as they do not cross ayield.
How to check in production
Iterator allocations are visible in BenchmarkDotNet with [MemoryDiagnoser] and in a dotnet-trace trace. You inspect the generated code with a decompiler: ILSpy or SharpLab.
Takeaway: yield return is a convenient code generator, and you pay for the convenience with an allocation and interface calls.
var seq = Evens(-1); // no exception: the body has not startedforeach (var n in Evens(10)){ Console.WriteLine(n); if (n == 2) break; // foreach calls Dispose, and finally runs}// start// 0// 2// finally
static IEnumerable<int> Evens(int max){ if (max < 0) throw new ArgumentOutOfRangeException(nameof(max)); // fires on the first MoveNext Console.WriteLine("start"); try { for (int i = 0; i <= max; i += 2) yield return i; } finally { Console.WriteLine("finally"); }}using System.Collections;
// A simplified equivalent of the iterator: for (i = 0; i <= max; i += 2) yield return i;sealed class EvensIterator(int max) : IEnumerable<int>, IEnumerator<int>{ private int _state; // 0 — not started, 1 — after yield, -1 — finished private int _i; // the local variable became a field public int Current { get; private set; } object IEnumerator.Current => Current;
public bool MoveNext() { switch (_state) { case 0: _i = 0; break; // first call: start of the body case 1: _i += 2; break; // resume after yield return default: return false; } if (_i <= max) { Current = _i; _state = 1; return true; } _state = -1; return false; }
public IEnumerator<int> GetEnumerator() => this; // real code creates a copy if this is already in use IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); public void Dispose() => _state = -1; public void Reset() => throw new NotSupportedException();}- Why doesn’t an argument check in an iterator fire on the call? — The body starts only on the first
MoveNext; validation goes into a wrapper. - Does
finallyrun if you leave aforeachwithbreak? — Yes:foreachcallsDispose, which runsfinally. - Can you write
yield returninside atrywith acatch? — No, only inside atrywith afinally. - How many times does the iterator body run with two
foreachloops? — Twice: every enumeration starts from the beginning. - How does
IAsyncEnumerable<T>differ fromTask<List<T>>? — It hands out items as they become ready rather than all at once at the end.
How do ReadOnly, Immutable and Frozen collections differ?
Section titled “How do ReadOnly, Immutable and Frozen collections differ?”What it is
| Kind | Examples | What it guarantees |
|---|---|---|
| Read-only | IReadOnlyList<T>, ReadOnlyCollection<T> |
cannot be changed through it |
| Immutable | ImmutableArray<T>, ImmutableList<T> |
never changes |
| Frozen (.NET 8+) | FrozenDictionary<TKey, TValue>, FrozenSet<T> |
never changes and looks up fast |
Why
To hand data out and be sure nobody corrupts it.
How to use
list.AsReadOnly()— a wrapper for a public property.ToImmutableArray(),ToImmutableList()— an immutable copy.ToFrozenDictionary(),ToFrozenSet()— a lookup table you build once.
Common mistake
Calling Add on an immutable collection and ignoring the result. The original collection does not change: Add returned a new one.
Takeaway: read-only protects data from the consumer, immutable protects it from everyone, and frozen also speeds up lookup.
How it works
IReadOnlyList<T>is only an interface.List<T>implements it, and the cast(List<T>)viewremoves the protection.ReadOnlyCollection<T>wraps anIList<T>. It protects against casting, but changes to the underlying list are visible through it.ImmutableArray<T>is a struct over an array: O(1) indexing, any change copies the whole array.ImmutableList<T>is an AVL tree: indexing,AddandRemoveare O(log n).- Frozen collections analyze the keys at creation and choose a specialized lookup implementation.
When to choose what
| Scenario | Choice |
|---|---|
| A property that must not be changed from outside | ReadOnlyCollection<T> |
| A snapshot with rare changes | ImmutableArray<T> |
| Frequent changes with versions kept | ImmutableList<T>, ImmutableDictionary |
| A lookup table for the whole lifetime | FrozenDictionary, FrozenSet |
Pitfalls
default(ImmutableArray<T>)is uninitialized:IsDefaultreturnstrue, and enumerating it throwsNullReferenceException.- A
forloop over anImmutableList<T>costs O(n log n). Iterate withforeach. - A series of changes to an immutable collection creates many intermediate copies. Use
ToBuilder()andToImmutable()for that.
Rule of thumb
The docs recommend ImmutableArray<T> when changes are rare or there are few items, and ImmutableList<T> when changes are frequent.
Takeaway: read-only is an access restriction, immutable is a data guarantee, frozen is the guarantee plus read speed.
Under the hood
ImmutableList<T>stores an AVL tree. A “change” creates O(log n) new nodes along the path from the root; the other nodes are shared with the old version.FrozenDictionarypicks an implementation based on the data. For strings it buckets by length or hashes a unique substring; for dense integer keys and enums in .NET 10 it is a plain array.- In .NET 9 frozen collections gained alternate lookup: searching by
ReadOnlySpan<char>without creating a string.
Cost
- Immutable trees mean an object per item, pointers instead of an array, O(log n) reads and noticeably more memory than a
List<T>. - A frozen collection is expensive to build. The docs warn to use it only with trusted keys, since the keys affect construction time.
Trade-offs
- A configuration snapshot that is swapped as a whole —
ImmutableArray<T>orFrozen*: readers work without locks. - State history (undo, versions) —
ImmutableList<T>orImmutableDictionary. - A public API —
IReadOnlyList<T>in the signature. If the caller cannot be trusted, useReadOnlyCollection<T>or a copy.
How to check in production
The gain from Frozen depends on the keys, so confirm the decision with BenchmarkDotNet: TryGetValue versus Dictionary, plus construction cost multiplied by the number of rebuilds. The memory taken by immutable tree nodes shows up in a dotnet-gcdump heap snapshot.
Takeaway: choose by the guarantee you need and the load profile: read-heavy — an array or frozen, write-heavy — a tree.
using System.Collections.Frozen;using System.Collections.Immutable;using System.Collections.ObjectModel;
var source = new List<int> { 1, 2, 3 };
IReadOnlyList<int> view = source; // just an interfaceReadOnlyCollection<int> wrapper = source.AsReadOnly(); // a wrapper without mutating methodsImmutableArray<int> snapshot = source.ToImmutableArray(); // a copy that never changesFrozenSet<int> frozen = source.ToFrozenSet(); // a copy optimized for lookup
source.Add(4);Console.WriteLine($"{view.Count} {wrapper.Count} {snapshot.Length} {frozen.Count}"); // 4 4 3 3
((List<int>)view).Add(5); // the interface does not stop a castImmutableArray<int> next = snapshot.Add(10); // a new instance; snapshot is unchangedConsole.WriteLine($"{snapshot.Length} {next.Length}"); // 3 4Console.WriteLine(frozen.Contains(2)); // True- Can a list exposed as
IReadOnlyList<T>be modified? — Yes, by casting toList<T>;ReadOnlyCollection<T>or a copy protects against that. - Why is
ImmutableList<T>slower thanList<T>in aforloop? — The indexer walks down the tree, O(log n). - When should you not use
FrozenDictionary? — When the collection is rebuilt often: construction is expensive. - How do you efficiently build an immutable collection from many items? — With a
BuilderorCreateRange, not a chain ofAddcalls. - What is
default(ImmutableArray<T>)? — An uninitialized struct:IsDefaultreturnstrue, enumeration fails.
When is LINQ slow, and how do you measure it?
Section titled “When is LINQ slow, and how do you measure it?”What it is
LINQ is a convenient layer over loops. You pay for the convenience with helper objects and a lambda call for each item.
Why
In ordinary code the difference is invisible. In a loop with millions of iterations it adds up to milliseconds and megabytes of garbage.
How to use
Any()instead ofCount() > 0:Any()stops at the first item.- The
CountorLengthproperty instead of theCount()method on lists and arrays. ToList()if you need the query more than once.HashSet<T>instead ofList<T>ifContainsis called insideWhere.
Common mistake
Measuring speed with Stopwatch in a Debug build. The result will be wrong; the right tool is BenchmarkDotNet in Release.
Takeaway: LINQ is not expensive by itself, only in hot loops and when it does extra work.
How it works
- Every operator is an iterator object.
- A lambda is a delegate. If it captures a variable, a closure object is created too.
- Every item costs
MoveNextandCurrentcalls through an interface plus a delegate call. Count()checks whether the source implementsICollection<T>and then readsCountwithout enumerating. A lazy query is enumerated in full.
When to choose what
| Instead of | Better |
|---|---|
Count() > 0 |
Any(), or Count > 0 on a collection |
GroupBy(x => x.Key).Select(g => (g.Key, g.Count())) |
CountBy(x => x.Key) (.NET 9+) |
Where(x => list.Contains(x.Id)) |
a HashSet<T> for Contains |
| LINQ in a loop with millions of iterations | for or foreach, if a benchmark confirms it |
Pitfalls
Analyzers catch the typical mistakes: CA1829 (the Length or Count property instead of Count()), CA1827 (Any() instead of Count()), CA1860 (a property instead of Any() on collections), CA1851 (multiple enumeration).
Rule of thumb
Rewrite only what you have measured:
- Move the code into a class with
[Benchmark]methods and the[MemoryDiagnoser]attribute. - Run it in Release without a debugger.
- Compare LINQ and the loop on realistic data sizes using
[Params].
Takeaway: first remove extra work, then measure, and only then rewrite LINQ.
Under the hood
- LINQ’s internal iterators pass information to each other:
Where(...).Select(...)is one object, andSkip(...).Take(...)over aList<T>works by index. - In .NET 9,
Any,All,Count,FirstandSinglewith a predicate gained a fast path throughReadOnlySpan<T>for arrays andList<T>. - In .NET 10,
Containsgained specializations for chains of operators. - Also in .NET 10, the JIT devirtualizes array interface methods, and escape analysis in many cases stack-allocates delegates and enumerators of arrays and
List<T>that do not leave the method.
Cost
- The JIT in .NET 10 does not yet remove the closure object.
- The compiler caches a non-capturing lambda in a static field. A lambda that captures
thisor a local creates a delegate on every call. - A static lambda (
static x => ..., C# 9) guarantees that nothing is captured.
Trade-offs
- For Native AOT, LINQ is built in a size-optimized flavour by default, without some optimizations. In .NET 10 this is switched with the
UseSizeOptimizedLinqproperty. - A loop wins on speed but loses on readability; only proven hot code is rewritten.
How to check in production
- dotnet-counters: allocation rate and generation 0 collection frequency.
- dotnet-trace or PerfView: hot stacks and allocation types.
- BenchmarkDotNet with several runtimes (
--runtimes net8.0 net10.0): results on .NET 8 and .NET 10 can differ noticeably.
Takeaway: “LINQ is slow” is a hypothesis you test with a profile and a benchmark on the target .NET version.
// NuGet: BenchmarkDotNetusing BenchmarkDotNet.Attributes;using BenchmarkDotNet.Running;
BenchmarkRunner.Run<LinqVsLoop>();
[MemoryDiagnoser] // adds the Allocated columnpublic class LinqVsLoop{ private readonly int[] _data = Enumerable.Range(0, 10_000).ToArray(); private readonly int _threshold = 5_000;
[Benchmark(Baseline = true)] public int Loop() { int sum = 0; foreach (var x in _data) if (x > _threshold) sum += x; return sum; // 0 bytes allocated }
[Benchmark] public int Linq() => SumAbove(_data, _threshold);
private static int SumAbove(int[] data, int threshold) => data.Where(x => x > threshold).Sum(); // a closure, a delegate and a Where iterator}- Does
Count()enumerate the items of aList<T>? — No, it readsCountviaICollection<T>, but CA1829 still recommends the property. - How does a static lambda help performance? — It guarantees no capture: the delegate is cached and no closure is created.
- What does the Allocated column in BenchmarkDotNet show? — Bytes on the managed heap per operation with
[MemoryDiagnoser]. - What replaces
GroupBy(...).Select(g => g.Count())for counting? —CountByfrom .NET 9: no intermediate groups. - Why might a benchmark on .NET 8 disagree with .NET 10? — New LINQ and JIT optimizations: devirtualization and stack allocation.