C# 如何有效地计算移动标准差

How to efficiently calculate a moving Standard Deviation

提问by Christiaan Wevers

Below you can see my C# method to calculate Bollinger Bands for each point (moving average, up band, down band).

您可以在下面看到我的 C# 方法来计算每个点的布林带（移动平均线、上行带、下行带）。

As you can see this method uses 2 for loops to calculate the moving standard deviation using the moving average. It used to contain an additional loop to calculate the moving average over the last n periods. This one I could remove by adding the new point value to total_average at the beginning of the loop and removing the i - n point value at the end of the loop.

如您所见，此方法使用 2 个 for 循环来使用移动平均值计算移动标准差。它曾经包含一个额外的循环来计算过去 n 个周期的移动平均线。我可以通过在循环开始时将新点值添加到 total_average 并在循环结束时删除 i - n 点值来删除这个值。

My question now is basically: Can I remove the remaining inner loop in a similar way I managed with the moving average?

我现在的问题基本上是：我可以用移动平均线管理的类似方式删除剩余的内循环吗？

    public static void AddBollingerBands(SortedList<DateTime, Dictionary<string, double>> data, int period, int factor)
    {
        double total_average = 0;

        for (int i = 0; i < data.Count(); i++)
        {
            total_average += data.Values[i]["close"];

            if (i >= period - 1)
            {
                double total_bollinger = 0;
                double average = total_average / period;

                for (int x = i; x > (i - period); x--)
                {
                    total_bollinger += Math.Pow(data.Values[x]["close"] - average, 2);
                }

                double stdev = Math.Sqrt(total_bollinger / period);

                data.Values[i]["bollinger_average"] = average;
                data.Values[i]["bollinger_top"] = average + factor * stdev;
                data.Values[i]["bollinger_bottom"] = average - factor * stdev;

                total_average -= data.Values[i - period + 1]["close"];
            }
        }
    }

采纳答案by RBarryYoung

The answer is yes, you can. In the mid-80's I developed just such an algorithm (probably not original) in FORTRAN for a process monitoring and control application. Unfortunately, that was over 25 years ago and I do not remember the exact formulas, but the technique was an extension of the one for moving averages, with second order calculations instead of just linear ones.

答案是肯定的，你可以。在 80 年代中期，我在 FORTRAN 中为过程监视和控制应用程序开发了这样一种算法（可能不是原创的）。不幸的是，那是在 25 年前，我不记得确切的公式，但该技术是移动平均线技术的扩展，使用二阶计算而不仅仅是线性计算。

---

---

After looking at your code some, I am think that I can suss out how I did it back then. Notice how your inner loop is making a Sum of Squares?:

看了你的代码之后，我想我可以弄清楚我当时是怎么做的。请注意您的内部循环是如何计算平方和的？：

            for (int x = i; x > (i - period); x--)
            {
                total_bollinger += Math.Pow(data.Values[x]["close"] - average, 2);
            }

in much the same way that your average must have originally had a Sum of Values? The only two differences are the order (its power 2 instead of 1) and that you are subtracting the average each value before you square it. Now that might look inseparable, but in fact they can be separated:

与您的平均值最初必须具有值总和的方式大致相同？唯一的两个区别是顺序（它的幂是 2 而不是 1），以及在平方之前减去每个值的平均值。现在看起来可能是分不开的，但实际上它们可以分开：

SUM(i=1; n){ (v[i] - k)^2 }

is

是

SUM(i=1..n){v[i]^2 -2*v[i]*k + k^2}

which becomes

变成

SUM(i=1..n){v[i]^2 -2*v[i]*k} + k^2*n

which is

这是

SUM(i=1..n){v[i]^2} + SUM(i=1..n){-2*v[i]*k} + k^2*n

which is also

这也是

SUM(i=1..n){v[i]^2} + SUM(i=1..n){-2*v[i]}*k + k^2*n

Now the first term is just a Sum of Squares, you handle that in the same way that you do the sum of Values for the average. The last term (k^2*n) is just the average squared times the period. Since you divide the result by the period anyway, you can just add the new average squared without the extra loop.

现在第一项只是平方和，您可以像处理平均值的值之和一样处理它。最后一项 ( k^2*n) 只是平均平方乘以period。由于无论如何您将结果除以周期，您只需添加新的平均平方而无需额外循环。

Finally, in the second term (SUM(-2*v[i]) * k), since SUM(v[i]) = total = k*nyou can then change it into this:

最后，在第二项 ( SUM(-2*v[i]) * k) 中，因为SUM(v[i]) = total = k*n您可以将其更改为：

-2 * k * k * n

or just -2*k^2*n, which is -2 times the average squared, once the period (n) is divided out again. So the final combined formula is:

或者只是-2*k^2*n，即平均平方的 -2 倍，一旦n再次划分句点 ( )。所以最终的组合公式为：

SUM(i=1..n){v[i]^2} - n*k^2

or

或者

SUM(i=1..n){values[i]^2} - period*(average^2)

(be sure to check the validity of this, since I am deriving it off the top of my head)

（一定要检查这个的有效性，因为我是从头顶上推导出来的）

And incorporating into your code should look something like this:

并入您的代码应该是这样的：

public static void AddBollingerBands(ref SortedList<DateTime, Dictionary<string, double>> data, int period, int factor)
{
    double total_average = 0;
    double total_squares = 0;

    for (int i = 0; i < data.Count(); i++)
    {
        total_average += data.Values[i]["close"];
        total_squares += Math.Pow(data.Values[i]["close"], 2);

        if (i >= period - 1)
        {
            double total_bollinger = 0;
            double average = total_average / period;

            double stdev = Math.Sqrt((total_squares - Math.Pow(total_average,2)/period) / period);
            data.Values[i]["bollinger_average"] = average;
            data.Values[i]["bollinger_top"] = average + factor * stdev;
            data.Values[i]["bollinger_bottom"] = average - factor * stdev;

            total_average -= data.Values[i - period + 1]["close"];
            total_squares -= Math.Pow(data.Values[i - period + 1]["close"], 2);
        }
    }
}

回答by Jason

I've used commons-math (and contributed to that library!) for something very similar to this. It's open-source, porting to C# should be easy as store-bought pie (have you tried making a pie from scratch!?). Check it out: http://commons.apache.org/math/api-3.1.1/index.html. They have a StandardDeviation class. Go to town!

我已经使用了 commons-math（并为该库做出了贡献！）与此非常相似。它是开源的，移植到 C# 应该像商店买的馅饼一样容易（您是否尝试过从头开始制作馅饼！？）。检查一下：http: //commons.apache.org/math/api-3.1.1/index.html。他们有一个 StandardDeviation 类。去镇上！

回答by Joni

The problem with approaches that calculate the sum of squares is that it and the square of sums can get quite large, and the calculation of their difference may introduce a verylarge error, so let's think of something better. For why this is needed, see the Wikipedia article on Algorithms for computing varianceand John Cook on Theoretical explanation for numerical results)

计算平方和的方法的问题在于它和和的平方可能会变得非常大，并且计算它们的差值可能会引入非常大的误差，所以让我们考虑更好的方法。关于为什么需要这样做，请参阅维基百科关于计算方差的算法和约翰库克关于数值结果的理论解释）

First, instead of calculating the stddev let's focus on the variance. Once we have the variance, stddev is just the square root of the variance.

首先，我们不计算标准差，而是关注方差。一旦我们有了方差，stddev 就是方差的平方根。

Suppose the data are in an array called x; rolling an n-sized window by one can be thought of as removing the value of x[0]and adding the value of x[n]. Let's denote the averages of x[0]..x[n-1]and x[1]..x[n]by μ and μ' respectively. The difference between the variances of x[0]..x[n-1]and x[1]..x[n]is, after canceling out some terms and applying (a2-b2) = (a+b)(a-b):

假设数据在一个名为x;的数组中。将 n 大小的窗口滚动 1 可以被认为是删除 的值x[0]并添加 的值x[n]。让我们来表示的平均值x[0]..x[n-1]，并x[1]..x[n]通过μ分别μ”。x[0]..x[n-1]和的方差之间的差异x[1]..x[n]是，在取消一些条款并应用 之后(a2-b2) = (a+b)(a-b)：

Var[x[1],..,x[n]] - Var[x[0],..,x[n-1]] 
= (\sum_1^n x[i]2 - n μ'2)/(n-1) - (\sum_0^{n-1} x[i]2 - n μ2)/(n-1)
= (x[n]2 - x[0]2 - n(μ'2 - μ2))/(n-1) 
= (x[n]-μ' + x[0]-μ)(x[n]-x[0])/(n-1)

Therefore the variance is perturbed by something that doesn't require you to maintain the sum of squares, which is better for numerical accuracy.

因此，方差会被不需要你保持平方和的东西所扰乱，这对数值精度更好。

You can calculate the mean and variance once in the beginning with a proper algorithm (Welford's method). After that, every time you have to replace a value in the window x[0]by another x[n]you update the average and variance like this:

您可以在开始时使用适当的算法（韦尔福德方法）计算一次均值和方差。之后，每次必须x[0]用另一个值替换窗口中的值时，x[n]您都会更新平均值和方差，如下所示：

new_Avg = Avg + (x[n]-x[0])/n
new_Var = Var + (x[n]-new_Avg + x[0]-Avg)(x[n] - x[0])/(n-1)
new_StdDev = sqrt(new_Var)

回答by marco

Most important information has already been given above --- but maybe this is still of general interest.

上面已经给出了最重要的信息——但也许这仍然是普遍感兴趣的。

A tiny Java library to calculate moving average and standard deviation is available here: https://github.com/tools4j/meanvar

这里提供了一个用于计算移动平均值和标准偏差的小型 Java 库：https: //github.com/tools4j/meanvar

The implementation is based on a variant of Welford's method mentioned above. Methods to remove and replace values have been derived that can be used for moving value windows.

该实现基于上述 Welford 方法的变体。已导出可用于移动值窗口的删除和替换值的方法。