Python scipy 未优化并返回“由于精度损失不一定达到所需的错误”
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/24767191/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
scipy is not optimizing and returns "Desired error not necessarily achieved due to precision loss"
提问by felix
I have the following code which attempts to minimize a log likelihood function.
我有以下代码试图最小化对数似然函数。
#!/usr/bin/python
import math
import random
import numpy as np
from scipy.optimize import minimize
def loglikelihood(params, data):
(mu, alpha, beta) = params
tlist = np.array(data)
r = np.zeros(len(tlist))
for i in xrange(1,len(tlist)):
r[i] = math.exp(-beta*(tlist[i]-tlist[i-1]))*(1+r[i-1])
loglik = -tlist[-1]*mu
loglik = loglik+alpha/beta*sum(np.exp(-beta*(tlist[-1]-tlist))-1)
loglik = loglik+np.sum(np.log(mu+alpha*r))
return -loglik
atimes = [ 148.98894201, 149.70253172, 151.13717804, 160.35968355,
160.98322609, 161.21331798, 163.60755544, 163.68994973,
164.26131871, 228.79436067]
a= 0.01
alpha = 0.5
beta = 0.6
print loglikelihood((a, alpha, beta), atimes)
res = minimize(loglikelihood, (0.01, 0.1,0.1), method = 'BFGS',args = (atimes,))
print res
It gives me
它给了我
28.3136498357
./test.py:17: RuntimeWarning: invalid value encountered in log
loglik = loglik+np.sum(np.log(mu+alpha*r))
status: 2
success: False
njev: 14
nfev: 72
hess_inv: array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
fun: 32.131359359964378
x: array([ 0.01, 0.1 , 0.1 ])
message: 'Desired error not necessarily achieved due to precision loss.'
jac: array([ -2.8051672 , 13.06962156, -48.97879982])
Notice that it hasn't managed to optimize the parameters at all and the minimized value 32 is bigger than 28 which is what you get with a= 0.01, alpha = 0.5, beta = 0.6 . It's possible this problem could be avoided by choosing better initial guesses but if so, how can I do this automatically?
请注意，它根本没有设法优化参数，并且最小化值 32 大于 28，这是您在 a= 0.01, alpha = 0.5, beta = 0.6 时得到的值。可以通过选择更好的初始猜测来避免这个问题，但如果是这样，我如何自动执行此操作？
采纳答案by JimmyK
I copied your example and tried a little bit. Looks like if you stick with BFGS solver, after a few iteration the mu+ alpha * rwill have some negative numbers, and that's how you get the RuntimeWarning.
我复制了你的例子并尝试了一点。看起来如果你坚持使用 BFGS 求解器，经过几次迭代后，mu+ alpha * r会有一些负数，这就是你获得 RuntimeWarning 的方式。
The easiest fix I can think of is to switch to Nelder Mead solver.
我能想到的最简单的解决方法是切换到 Nelder Mead 求解器。
res = minimize(loglikelihood, (0.01, 0.1,0.1), method = 'Nelder-Mead',args = (atimes,))
And it will give you this result:
它会给你这个结果：
28.3136498357
status: 0
nfev: 159
success: True
fun: 27.982451280648817
x: array([ 0.01410906, 0.68346023, 0.90837568])
message: 'Optimization terminated successfully.'
nit: 92
回答by optimizer
Watch out for negative values of the log() function, resolve them and tell the optimizer that they are bad, by adding a penalty:
注意 log() 函数的负值，解决它们并通过添加惩罚来告诉优化器它们是坏的：
#!/usr/bin/python
import math
import random
import numpy as np
from scipy.optimize import minimize
def loglikelihood(params, data):
(mu, alpha, beta) = params
tlist = np.array(data)
r = np.zeros(len(tlist))
for i in xrange(1,len(tlist)):
r[i] = math.exp(-beta*(tlist[i]-tlist[i-1]))*(1+r[i-1])
loglik = -tlist[-1]*mu
loglik += alpha/beta*sum(np.exp(-beta*(tlist[-1]-tlist))-1)
argument = mu + alpha * r
limit = 1e-6
if np.min(argument) < limit:
# add a penalty for too small argument of log
loglik += np.sum(np.minimum(0.0, argument - limit)) / limit
# keep argument of log above the limit
argument = np.maximum(argument, limit)
loglik += np.sum(np.log(argument))
return -loglik
atimes = [ 148.98894201, 149.70253172, 151.13717804, 160.35968355,
160.98322609, 161.21331798, 163.60755544, 163.68994973,
164.26131871, 228.79436067]
a= 0.01
alpha = 0.5
beta = 0.6
print loglikelihood((a, alpha, beta), atimes)
res = minimize(loglikelihood, (0.01, 0.1,0.1), method = 'BFGS',args = (atimes,))
print res
回答by Sahar
Facing the same warning, I solved it by rewriting the log-likelihood function to get log(params)and log(data)as arguments, instead of params and data.
面对同样的警告，我通过重写对数似然函数来解决它，以获取log(params)和log(data)作为参数，而不是参数和数据。
Thus, I avoid using np.log()in the likelihood function or Jacobian, if possible.
因此，np.log()如果可能，我避免在似然函数或雅可比矩阵中使用。
回答by Cam.Davidson.Pilon
Another solution (that worked for me) is to scale your function (and gradients) to values closer to 0. For example, my problem came up when I had to evaluate a log-likelihood of 60k points. This meant that my log-likelihood was a very large number. Conceptually, the log-likelihood was a very very spikey function.
另一个解决方案（对我有用）是将您的函数（和梯度）缩放到接近 0 的值。例如，当我必须评估 60k 点的对数似然时，我的问题就出现了。这意味着我的对数似然是一个非常大的数字。从概念上讲，对数似然是一个非常非常尖峰的函数。
The gradients started off large (to climb this spikey mountain), and then became moderately small, but never less than the default gtolparameter in the BGFS routine (which is the threshold that all gradients must be below for termination). Also, at this time I had essentially arrived at the correct values (I was using generated data so I knew the true values).
梯度开始很大（攀登这座尖峰山），然后变得适度小，但绝不会小于gtolBGFS 例程中的默认参数（这是所有梯度必须低于终止的阈值）。此外，此时我基本上已经得出了正确的值（我使用了生成的数据，所以我知道了真实值）。
What was happening was that my gradients were approx. 60k * average individual gradient value, and even if the average individual gradient valuewas small, say less than 1e-8, 60k * 1e-8 > gtol. So I was never satisfying the threshold even though I had arrived at the solution.
发生的事情是我的梯度大约是。60k * average individual gradient value，即使average individual gradient value很小，也可以说小于 1e-8， 60k * 1e-8 > gtol。因此，即使我已经找到了解决方案，我也从未满足过阈值。
Conceptually, because of this very spikey mountain, the algorithm was making small steps, but stepping overthe true minimum and never achieved average individual gradient << 1e-8which implies my gradients never went under gtol.
从概念上讲，由于这座山峰非常陡峭，算法正在迈出小步，但越过真正的最小值并且从未达到过average individual gradient << 1e-8，这意味着我的梯度从未下降过gtol。
Two solutions:
两种解决方案：
1) Scale your log-likelihood and gradients by a factor, like 1/nwhere nis the number of samples.
1）按一个因子缩放您的对数似然和梯度，例如样本数量1/n在哪里n。
2) Scale your gtol: for example "gtol": 1e-7 * n
2）扩展你的gtol：例如"gtol": 1e-7 * n
