Run async method 8 times in parallel

I coded it in the assumption that asynchronous and parallel processing would be the same

Asynchronous processing and parallel processing are quite different. If you don’t understand the difference, I think you should first read more about it (for example what is the relation between Asynchronous and parallel programming in c#?).

Now, what you want to do is actually not that simple, because you want to process a big collection asynchronously, with a specific degree of parallelism (8). With synchronous processing, you could use Parallel.ForEach() (along with ParallelOptions to configure the degree of parallelism), but there is no simple alternative that would work with async.

In your code, this is complicated by the fact that you expect everything to execute on the UI thread. (Though ideally, you shouldn’t access the UI directly from your computation. Instead, you should use IProgress, which would mean the code no longer has to execute on the UI thread.)

Probably the best way to do this in .Net 4.5 is to use TPL Dataflow. Its ActionBlock does exactly what you want, but it can be quite verbose (because it’s more flexible than what you need). So it makes sense to create a helper method:

public static Task AsyncParallelForEach<T>(
    IEnumerable<T> source, Func<T, Task> body,
    int maxDegreeOfParallelism = DataflowBlockOptions.Unbounded,
    TaskScheduler scheduler = null)
{
    var options = new ExecutionDataflowBlockOptions
    {
        MaxDegreeOfParallelism = maxDegreeOfParallelism
    };
    if (scheduler != null)
        options.TaskScheduler = scheduler;

    var block = new ActionBlock<T>(body, options);

    foreach (var item in source)
        block.Post(item);

    block.Complete();
    return block.Completion;
}

In your case, you would use it like this:

await AsyncParallelForEach(
    threads, async url => await DownloadUrl(url), 8,
    TaskScheduler.FromCurrentSynchronizationContext());

Here, DownloadUrl() is an async Task method that processes a single URL (the body of your loop), 8 is the degree of parallelism (probably shouldn’t be a literal constant in real code) and FromCurrentSynchronizationContext() makes sure the code executes on the UI thread.

Leave a Comment