C# Parallel foreach - System.AggregateException：发生一个或多个错误。---> System.IndexOutOfRangeException

C# Parallel foreach - System.AggregateException: One or more errors occurred. ---> System.IndexOutOfRangeException

提问by Murtaza Mandvi

Found the answer, ... The mistake in the code was that I was adding data to the wrong list

找到了答案，...代码中的错误是我将数据添加到错误的列表中

---

---

I know this has been asked a few times but I am simply unable to figure out why this will give an exception, am I missing a lock somewhere ?

我知道这已经被问过几次了，但我根本无法弄清楚为什么这会产生异常，我是否在某处丢失了锁？

var sendsToday = new List<TSend>();

var threads = _maxNumberOfThreads;
if (threads <= 0)
    threads = 1;

Parallel.ForEach(_subscribers,
    new ParallelOptions { MaxDegreeOfParallelism = threads },
    () => new ContentSendLocalStorage<TSend, TMedium>(_contentServices, _logService),
    (subscriber, loopState, localStorage) =>
    {
        localStorage.LogService.Warning(string.Format("Choosing content for subscriber {0}", subscriber.SubscriberId));
        foreach (var newsletterId in subscriber.SubscribedNewsletterIds)
        {
            localStorage.LogService.Warning(string.Format("Choosing content for newsletter {0}", newsletterId));
            var clicks = StateBag.Get<LookupList>(StateKeys.LookupList).Clicks.Where(c => c.Subscriber.SubscriberId == subscriber.SubscriberId).Select(c => c.Content.ContentId).ToArray();

            foreach (var contentService in _contentServices.Where(contentService => contentService.Contents.Count != 0))
            {
                subscriber.UrlsClicked = contentService.Contents
        .Where(c => clicks.Contains(c.ContentId))
        .GroupBy(g => g.Page.Url)
        .ToDictionary(k => k.Key, v => 1);

                var best = contentService.GetBestForSubscriber(subscriber, new TMedium { MediumId = int.Parse(newsletterId) });
                if (best != null)
        sendsToday.Add(best);
            }
        }

        localStorage.LogService.Warning(string.Format("Done choosing content for subscriber {0}", subscriber.SubscriberId));
        return localStorage;
    },
    finalStorage =>
    {
        lock (ContentSendLock)
        {
            sendsToday.AddRange(finalStorage.SubscriberSends);
        }
    });

I keep getting the exception below :

我不断收到以下异常：

System.AggregateException: One or more errors occurred. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.List`1.Add(T item)
   at WFM.Newsletter.Business.CORE.OptimizationEngine`2.<>c__DisplayClassd.<Start>b__4(Subscriber subscriber, ParallelLoopState loopState, ContentSendLocalStorage`2 localStorage)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass21`2.<ForEachWorker>b__1a(Int32 i, ParallelLoopState state, TLocal local)
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass7.<ExecuteSelfReplicating>b__6(Object )
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](TSource[] array, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Func`1 localInit, Func`4 body, Action`1 localFinally)
   at WFM.Newsletter.Business.CORE.OptimizationEngine`2.Start()
   at WFM.Newsletter.Business.CORE.OptimizationService.Start(IEnumerable`1 subscribers, IEnumerable`1 newsletters, LookupList lookupList, LocalConfig config, ILogging logService, Int32 maxNumberOfThreads, String subscriberClicksSourceTableName)
---> (Inner Exception #0) System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.List`1.Add(T item)
   at WFM.Newsletter.Business.CORE.OptimizationEngine`2.<>c__DisplayClassd.<Start>b__4(Subscriber subscriber, ParallelLoopState loopState, ContentSendLocalStorage`2 localStorage)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass21`2.<ForEachWorker>b__1a(Int32 i, ParallelLoopState state, TLocal local)
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass7.<ExecuteSelfReplicating>b__6(Object )<---

---> (Inner Exception #1) System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.List`1.Add(T item)
   at WFM.Newsletter.Business.CORE.OptimizationEngine`2.<>c__DisplayClassd.<Start>b__4(Subscriber subscriber, ParallelLoopState loopState, ContentSendLocalStorage`2 localStorage)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass21`2.<ForEachWorker>b__1a(Int32 i, ParallelLoopState state, TLocal local)
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass7.<ExecuteSelfReplicating>b__6(Object )<---

---> (Inner Exception #2) System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.List`1.Add(T item)
   at WFM.Newsletter.Business.CORE.OptimizationEngine`2.<>c__DisplayClassd.<Start>b__4(Subscriber subscriber, ParallelLoopState loopState, ContentSendLocalStorage`2 localStorage)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass21`2.<ForEachWorker>b__1a(Int32 i, ParallelLoopState state, TLocal local)
   at System.Threading.Tasks.Parallel.<>c__DisplayClassf`1.<ForWorker>b__c()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass7.<ExecuteSelfReplicating>b__6(Object )<---

采纳答案by Murtaza Mandvi

Found the answer, neither did I have to change my logic to introduce ConcurrentBag nor did I have issues of large volume insert.

找到了答案，我既不必更改逻辑来引入 ConcurrentBag，也没有大批量插入的问题。

The mistake in the code was that I was adding data to the wrong list i.e. sendsToday.Add(best);

代码中的错误是我将数据添加到错误的列表中，即 sendToday.Add(best);

Instead I had to add it to "localstorage" which is handled with a lock by .Net itself in my Final storage.

相反，我不得不将它添加到“localstorage”中，它由 .Net 本身在我的最终存储中用锁处理。

So instead of

所以代替

 if (best != null)
        sendsToday.Add(best);

The correct code is

正确的代码是

if (best != null)
localStorage.SubscriberSends.Add(best);  

and final storage finally merges each with a locking mechanism.

最终存储最终将每个合并与锁定机制。

Thanks anyways for all the support !

总之谢谢大家的支持！

回答by Mike Parkhill

Inside of your inner foreach you are adding to the sendsToday which is a List<>. This is occurring on every thread.

在您的内部 foreach 中，您将添加到sendToday，它是一个列表<>。这发生在每个线程上。

if (best != null)
    sendsToday.Add(best);
}

Instead of using List<>, try using one of the System.Collections.Concurrentimplementations (e.g. ConcurrentBag). It handles all the locking internally for you and is specifically designed for threading.

不要使用 List<>，而是尝试使用System.Collections.Concurrent实现之一（例如 ConcurrentBag）。它在内部为您处理所有锁定，并且专为线程设计。

var sendsToday = new ConcurrentBag<TSend>();

回答by D Stanley

Since the method that generates the exception is List<T>.Add()my guess the exception is thrown here:

由于生成异常的方法是List<T>.Add()我的猜测，这里抛出异常：

sendsToday.Add(best);

Add a lockaround the list access and see if that helps.

lock在列表访问周围添加一个，看看是否有帮助。