How to loop through IEnumerable in batches [duplicate]

by

You can use MoreLINQ’s Batch operator (available from NuGet):

foreach(IEnumerable<User> batch in users.Batch(1000))
   // use batch

If simple usage of library is not an option, you can reuse implementation:

public static IEnumerable<IEnumerable<T>> Batch<T>(
        this IEnumerable<T> source, int size)
{
    T[] bucket = null;
    var count = 0;

    foreach (var item in source)
    {
       if (bucket == null)
           bucket = new T[size];

       bucket[count++] = item;

       if (count != size)                
          continue;

       yield return bucket.Select(x => x);

       bucket = null;
       count = 0;
    }

    // Return the last bucket with all remaining elements
    if (bucket != null && count > 0)
    {
        Array.Resize(ref bucket, count);
        yield return bucket.Select(x => x);
    }
}

BTW for performance you can simply return bucket without calling Select(x => x). Select is optimized for arrays, but selector delegate still would be invoked on each item. So, in your case it’s better to use

yield return bucket;

More Related Contents:

  1. Dynamic LINQ OrderBy on IEnumerable / IQueryable
  2. IEnumerable vs List – What to Use? How do they work?
  3. Convert IEnumerable to DataTable
  4. Can anyone explain IEnumerable and IEnumerator to me? [closed]
  5. Optimal LINQ query to get a random sub collection – Shuffle
  6. How can I add an item to a IEnumerable collection?
  7. Pair-wise iteration in C# or sliding window enumerator
  8. Passing a single item as IEnumerable
  9. ICollection Vs List in Entity Framework
  10. Differences between IQueryable, List, IEnumerator?
  11. Converting from IEnumerable to List [duplicate]
  12. IEnumerable to string [duplicate]
  13. What is the difference between IEnumerator and IEnumerable? [duplicate]
  14. Performance of nested yield in a tree
  15. How to concatenate two IEnumerable into a new IEnumerable?
  16. Collection was modified; enumeration may not execute error when removing a ListItem from a LIstBox
  17. IEnumerable vs IReadonlyCollection vs ReadonlyCollection for exposing a list member
  18. Convert DataTable to IEnumerable
  19. Is there an IEnumerable implementation that only iterates over it’s source (e.g. LINQ) once?
  20. Recreating a Dictionary from an IEnumerable
  21. Get previous and next item in a IEnumerable using LINQ
  22. Pair-wise iteration in C#, or sliding window enumerator
  23. What is the purpose of AsQueryable()?
  24. Casting IEnumerable to List
  25. IEnumerable and order
  26. IEnumerable as return type
  27. Finding symmetric difference with LINQ
  28. Why classes that implement variant interfaces remain invariant?
  29. Why use .AsEnumerable() rather than casting to IEnumerable?
  30. generic NOT constraint where T : !IEnumerable

Leave a Comment