Split an IEnumerable into fixed-sized chunks (return an IEnumerable where the inner sequences are of fixed length) [duplicate]

by

You could try to implement Batch method mentioned above on your own like this:

    static class MyLinqExtensions 
    { 
        public static IEnumerable<IEnumerable<T>> Batch<T>( 
            this IEnumerable<T> source, int batchSize) 
        { 
            using (var enumerator = source.GetEnumerator()) 
                while (enumerator.MoveNext()) 
                    yield return YieldBatchElements(enumerator, batchSize - 1); 
        } 

        private static IEnumerable<T> YieldBatchElements<T>( 
            IEnumerator<T> source, int batchSize) 
        { 
            yield return source.Current; 
            for (int i = 0; i < batchSize && source.MoveNext(); i++) 
                yield return source.Current; 
        } 
    }

I’ve grabbed this code from http://blogs.msdn.com/b/pfxteam/archive/2012/11/16/plinq-and-int32-maxvalue.aspx.

UPDATE: Please note, that this implementation not only lazily evaluates batches but also items inside batches, which means it will only produce correct results when batch is enumerated only after all previous batches were enumerated. For example: 

public static void Main(string[] args)
{
    var xs = Enumerable.Range(1, 20);
    Print(xs.Batch(5).Skip(1)); // should skip first batch with 5 elements
}

public static void Print<T>(IEnumerable<IEnumerable<T>> batches)
{
    foreach (var batch in batches)
    {
        Console.WriteLine($"[{string.Join(", ", batch)}]");
    }
}

will output:

[2, 3, 4, 5, 6] //only first element is skipped.
[7, 8, 9, 10, 11]
[12, 13, 14, 15, 16]
[17, 18, 19, 20]

So, if you use case assumes batching when batches are sequentially evaluated, then lazy solution above will work, otherwise if you can’t guarantee strictly sequential batch processing (e.g. when you want to process batches in parallel), you will probably need a solution which eagerly enumerates batch content, similar to one mentioned in the question above or in the MoreLINQ

More Related Contents:

  1. Dynamic LINQ OrderBy on IEnumerable / IQueryable
  2. Returning IEnumerable vs. IQueryable
  3. IEnumerable vs List – What to Use? How do they work?
  4. Optimal LINQ query to get a random sub collection – Shuffle
  5. How to get the index of an element in an IEnumerable?
  6. How to check if IEnumerable is null or empty?
  7. Differences between IQueryable, List, IEnumerator?
  8. Enumerating Collections that are not inherently IEnumerable?
  9. Is there an IEnumerable implementation that only iterates over it’s source (e.g. LINQ) once?
  10. Get previous and next item in a IEnumerable using LINQ
  11. Pair-wise iteration in C#, or sliding window enumerator
  12. how do I chunk an enumerable?
  13. What is the purpose of AsQueryable()?
  14. Finding symmetric difference with LINQ
  15. Why use .AsEnumerable() rather than casting to IEnumerable?
  16. Performance of Skip (and similar functions, like Take)
  17. Map two lists into a dictionary in C#
  18. Entity Framework/Linq EXpression converting from string to int
  19. Difference between Lookup() and Dictionary(Of list())
  20. How to convert an expression tree to a partial SQL query?
  21. Check for any element that exists in two collections
  22. Convert list to dictionary using linq and not worrying about duplicates
  23. How does LINQ expression syntax work with Include() for eager loading
  24. Convert / Cast IEnumerable to IEnumerable
  25. Select multiple fields group by and sum
  26. Where can I find the System.Linq.Dynamic dll?
  27. Linq – Top value from each group
  28. How can I convert a DataTable into a Dynamic object?
  29. C# Searching for files and folders except in certain folders
  30. What is the difference between LINQ ToDictionary and ToLookup

Leave a Comment