List size limitation in C#

by

A List<int> is backed by an int[]. You will fail as soon as a larger backing array cannot be allocated – and bear in mind that:

  • There’s a 2GB per-object limit in the CLR even in 64 bits (EDIT: as of .NET 4.5, this can be avoided for the 64-bit CLR – see <gcAllowVeryLargeObjects>)
  • The list will try to allocate a backing array which is larger than what it immediately requires, in order to accommodate later Add requests without reallocation.
  • During the reallocation, there has to be enough total memory for both the old and the new arrays.

Setting the Capacity to a value which will put the backing array near the theoretical limit may get you a higher cutoff point than the natural growth, but that limit will certainly come.

I would expect a limit of around 229 elements (536,870,912) – I’m slightly surprised you haven’t managed to get beyond 134,217,728. How much memory do you actually have? What version of .NET are you using, and on what architecture? (It’s possible that the per-object limit is 1GB for a 32-bit CLR, I can’t remember for sure.)

Note that even if the per-object limit wasn’t a problem, as soon as you got above 231 elements you’d have problems addressing those elements directly with List<T>, as the indexer takes an int value.

Basically, if you want a collection with more than int.MaxValue elements, you’ll need to write your own, probably using multiple backing arrays. You might want to explicitly prohibit removals and arbitrary insertions 🙂

More Related Contents:

  1. 2 ways of list making in c# [duplicate]
  2. What is the best way to organize a lot of data which contains multiple conditions?
  3. C# List sorting with multiple range
  4. List or IList [closed]
  5. Compare two List objects for equality, ignoring order [duplicate]
  6. Create an array or List of all dates between two dates [duplicate]
  7. C# List of objects, how do I get the sum of a property
  8. How do arrays in C# partially implement IList?
  9. How can I get every nth item from a List?
  10. How can I add an item to a IEnumerable collection?
  11. Binding List to DataGridView in WinForm
  12. List OrderBy Alphabetical Order
  13. Linq select objects in list where exists IN (A,B,C)
  14. How to initialize a list of strings (List) with many string values
  15. Converting from IEnumerable to List [duplicate]
  16. Does Distinct() method keep original ordering of sequence intact?
  17. Split string, convert ToList() in one line
  18. When a class is inherited from List, XmlSerializer doesn’t serialize other attributes
  19. Check if list contains any of another list
  20. How can I sort a List by multiple T.attributes?
  21. C# – elegant way of partitioning a list?
  22. Console.WriteLine and generic List
  23. Remove duplicate items from list in C#, according to one of their properties
  24. Parallel.ForEach with adding to list
  25. Sort one list by another
  26. Cast Object to Generic List
  27. C# Convert List to Dictionary
  28. Sorting a List
  29. Why am I getting System.Collections.Generic.List`1[System.String] instead of the list’s contents? [duplicate]
  30. The name ‘…’ does not exist in the current context

Leave a Comment