Python 约束非线性优化
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/21765794/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Python constrained non-linear optimization
提问by akhil
What's the recommended package for constrained non-linear optimization in python ?
python中约束非线性优化的推荐包是什么？
The specific problem I'm trying to solve is this:
我试图解决的具体问题是：
I have an unknown X(Nx1), I have M(Nx1) uvectors and M(NxN) smatrices.
我有一个未知数X（Nx1），我有M（Nx1）个u向量和M（NxN）个s矩阵。
max [5th percentile of (ui_T*X), i in 1 to M]
st
0<=X<=1 and
[95th percentile of (X_T*si*X), i in 1 to M]<= constant
When I started out the problem I only had one point estimate for uand sand I was able to solve the problem above with cvxpy.
当我开始解决这个问题时，我只有一点估计u，s并且我能够用cvxpy.
I realized that instead of one estimate for uand s, I had the entire distribution of values so I wanted to change my objective function so that I could use the entire distribution. The problem description above is my attempt to include that information in a meaningful way.
我意识到，我拥有整个值的分布，而不是对u和的一个估计，s所以我想改变我的目标函数，以便我可以使用整个分布。上面的问题描述是我尝试以有意义的方式包含该信息。
cvxpycannot be used to solve this, I've tried scipy.optimize.anneal, but I can't seem to set bounds on the unknown values. I've looked at pulptoo but it doesnt allow nonlinear constraints.
cvxpy不能用来解决这个问题，我试过了scipy.optimize.anneal，但我似乎无法设置未知值的界限。我也看过，pulp但它不允许非线性约束。
采纳答案by Slater Victoroff
scipyhas a spectacular package for constrained non-linear optimization.
scipy有一个用于约束非线性优化的壮观包。
You can get started by reading the optimizedoc, but here's an example with SLSQP:
您可以通过阅读optimizedoc开始，但这里有一个 SLSQP 示例：
minimize(func, [-1.0,1.0], args=(-1.0,), jac=func_deriv, constraints=cons, method='SLSQP', options={'disp': True})
回答by pseudocubic
Typically for fitting you can use scipy.optimizefunctions, or lmfitwhich simply extends the scipy.optimize package to make it easier to pass things like bounds. Personally, I like using kmpfit, part of the kapteyn library and is based on the C implementation of MPFIT.
通常为了拟合，您可以使用scipy.optimize函数，或者lmfit简单地扩展 scipy.optimize 包以使其更容易传递诸如边界之类的东西。就个人而言，我喜欢使用kmpfitkapteyn 库的一部分，它基于 MPFIT 的 C 实现。
scipy.optimize.minimize()is probably the most easy to obtain and is commonly used.
scipy.optimize.minimize()可能是最容易获得的，也是最常用的。
回答by John Hedengren
As others have commented as well, the SciPy minimize package is a good place to start. I've included an example below (Hock Schittkowski #71 benchmark) that includes an objective function, equality constraint, and inequality constraint.
正如其他人所评论的，SciPy 最小化包是一个很好的起点。我在下面包含了一个示例（Hock Schittkowski #71 benchmark），其中包括目标函数、等式约束和不等式约束。
import numpy as np
from scipy.optimize import minimize
def objective(x):
return x[0]*x[3]*(x[0]+x[1]+x[2])+x[2]
def constraint1(x):
return x[0]*x[1]*x[2]*x[3]-25.0
def constraint2(x):
sum_eq = 40.0
for i in range(4):
sum_eq = sum_eq - x[i]**2
return sum_eq
# initial guesses
n = 4
x0 = np.zeros(n)
x0[0] = 1.0
x0[1] = 5.0
x0[2] = 5.0
x0[3] = 1.0
# show initial objective
print('Initial SSE Objective: ' + str(objective(x0)))
# optimize
b = (1.0,5.0)
bnds = (b, b, b, b)
con1 = {'type': 'ineq', 'fun': constraint1}
con2 = {'type': 'eq', 'fun': constraint2}
cons = ([con1,con2])
solution = minimize(objective,x0,method='SLSQP',\
bounds=bnds,constraints=cons)
x = solution.x
# show final objective
print('Final SSE Objective: ' + str(objective(x)))
# print solution
print('Solution')
print('x1 = ' + str(x[0]))
print('x2 = ' + str(x[1]))
print('x3 = ' + str(x[2]))
print('x4 = ' + str(x[3]))
There is also a more comprehensive discussion thread on nonlinear programming solvers for Pythonif SLSQP can't solve your problem. My course material on Engineering Design Optimizationis available if you need additional information on the solver methods.
如果 SLSQP 无法解决您的问题，还有一个关于 Python 非线性规划求解器的更全面的讨论线程。如果您需要有关求解器方法的更多信息，可以使用我的工程设计优化课程材料。
回答by Mike McKerns
While the SLSQPalgorithm in scipy.optimize.minimizeis good, it has a bunch of limitations. The first of which is it's a QPsolver, so it works will for equations that fit well into a quadratic programming paradigm. But what happens if you have functional constraints? Also, scipy.optimize.minimizeis not a global optimizer, so you often need to start very close to the final results.
虽然SLSQP算法scipy.optimize.minimize很好，但它有很多限制。第一个是它是一个QP求解器，因此它适用于非常适合二次规划范式的方程。但是如果您有功能限制会发生什么？此外，scipy.optimize.minimize不是全局优化器，因此您通常需要在非常接近最终结果的情况下开始。
There is a constrained nonlinear optimization package (called mystic) that has been around for nearly as long as scipy.optimizeitself -- I'd suggest it as the go-to for handling any general constrained nonlinear optimization.
有一个约束非线性优化包（称为mystic），它的存在时间几乎与scipy.optimize它本身一样长——我建议将它作为处理任何一般约束非线性优化的首选。
For example, your problem, if I understand your pseudo-code, looks something like this:
例如，你的问题，如果我理解你的伪代码，看起来像这样：
import numpy as np
M = 10
N = 3
Q = 10
C = 10
# let's be lazy, and generate s and u randomly...
s = np.random.randint(-Q,Q, size=(M,N,N))
u = np.random.randint(-Q,Q, size=(M,N))
def percentile(p, x):
x = np.sort(x)
p = 0.01 * p * len(x)
if int(p) != p:
return x[int(np.floor(p))]
p = int(p)
return x[p:p+2].mean()
def objective(x, p=5): # inverted objective, to find the max
return -1*percentile(p, [np.dot(np.atleast_2d(u[i]), x)[0] for i in range(0,M-1)])
def constraint(x, p=95, v=C): # 95%(xTsx) - v <= 0
x = np.atleast_2d(x)
return percentile(p, [np.dot(np.dot(x,s[i]),x.T)[0,0] for i in range(0,M-1)]) - v
bounds = [(0,1) for i in range(0,N)]
So, to handle your problem in mystic, you just need to specify the bounds and the constraints.
因此，要处理 中的问题mystic，您只需要指定边界和约束。
from mystic.penalty import quadratic_inequality
@quadratic_inequality(constraint, k=1e4)
def penalty(x):
return 0.0
from mystic.solvers import diffev2
from mystic.monitors import VerboseMonitor
mon = VerboseMonitor(10)
result = diffev2(objective, x0=bounds, penalty=penalty, npop=10, gtol=200, \
disp=False, full_output=True, itermon=mon, maxiter=M*N*100)
print result[0]
print result[1]
The result looks something like this:
结果如下所示：
Generation 0 has Chi-Squared: -0.434718
Generation 10 has Chi-Squared: -1.733787
Generation 20 has Chi-Squared: -1.859787
Generation 30 has Chi-Squared: -1.860533
Generation 40 has Chi-Squared: -1.860533
Generation 50 has Chi-Squared: -1.860533
Generation 60 has Chi-Squared: -1.860533
Generation 70 has Chi-Squared: -1.860533
Generation 80 has Chi-Squared: -1.860533
Generation 90 has Chi-Squared: -1.860533
Generation 100 has Chi-Squared: -1.860533
Generation 110 has Chi-Squared: -1.860533
Generation 120 has Chi-Squared: -1.860533
Generation 130 has Chi-Squared: -1.860533
Generation 140 has Chi-Squared: -1.860533
Generation 150 has Chi-Squared: -1.860533
Generation 160 has Chi-Squared: -1.860533
Generation 170 has Chi-Squared: -1.860533
Generation 180 has Chi-Squared: -1.860533
Generation 190 has Chi-Squared: -1.860533
Generation 200 has Chi-Squared: -1.860533
Generation 210 has Chi-Squared: -1.860533
STOP("ChangeOverGeneration with {'tolerance': 0.005, 'generations': 200}")
[-0.17207128 0.73183465 -0.28218955]
-1.86053344078
mysticis very flexible, and can handle any type of constraints (e.g. equalities, inequalities) including symbolic and functional constraints.
I specified the constraints as "penalties" above, which is the traditional way, in that they apply a penalty to the objective when the constraint is violated.
mysticalso provides nonlinear kernel transformations, which constrain solution space by reducing the space of valid solutions (i.e. by a spatial mapping or kernel transformation).
mystic非常灵活，可以处理任何类型的约束（例如等式、不等式），包括符号和功能约束。我在上面将约束指定为“惩罚”，这是传统方式，因为当违反约束时，它们会对目标应用惩罚。
mystic还提供非线性核变换，通过减少有效解的空间（即通过空间映射或核变换）来约束解空间。
As an example, here's mysticsolving a problem that breaks a lot of QP solvers, since the constraints are not in the form of a constraints matrix. It's optimizing the design of a pressure vessel.
举个例子，这里mystic解决了一个破坏了很多 QP 求解器的问题，因为约束不是约束矩阵的形式。它正在优化压力容器的设计。
"Pressure Vessel Design"
def objective(x):
x0,x1,x2,x3 = x
return 0.6224*x0*x2*x3 + 1.7781*x1*x2**2 + 3.1661*x0**2*x3 + 19.84*x0**2*x2
bounds = [(0,1e6)]*4
# with penalty='penalty' applied, solution is:
xs = [0.72759093, 0.35964857, 37.69901188, 240.0]
ys = 5804.3762083
from mystic.symbolic import generate_constraint, generate_solvers, simplify
from mystic.symbolic import generate_penalty, generate_conditions
equations = """
-x0 + 0.0193*x2 <= 0.0
-x1 + 0.00954*x2 <= 0.0
-pi*x2**2*x3 - (4/3.)*pi*x2**3 + 1296000.0 <= 0.0
x3 - 240.0 <= 0.0
"""
cf = generate_constraint(generate_solvers(simplify(equations)))
pf = generate_penalty(generate_conditions(equations), k=1e12)
if __name__ == '__main__':
from mystic.solvers import diffev2
from mystic.math import almostEqual
from mystic.monitors import VerboseMonitor
mon = VerboseMonitor(10)
result = diffev2(objective, x0=bounds, bounds=bounds, constraints=cf, penalty=pf, \
npop=40, gtol=50, disp=False, full_output=True, itermon=mon)
assert almostEqual(result[0], xs, rel=1e-2)
assert almostEqual(result[1], ys, rel=1e-2)
Find this, and roughly 100 examples like it, here: https://github.com/uqfoundation/mystic.
在这里找到这个和大约 100 个类似的例子：https: //github.com/uqfoundation/mystic。
I'm the author, so I am slightly biased. However, the bias is very slight. mysticis both mature and well-supported, and is unparalleled in capacity to solve hard constrained nonlinear optimization problems.
我是作者，所以我有点偏见。然而，偏差是非常轻微的。mystic既成熟又得到很好的支持，并且在解决硬约束非线性优化问题方面具有无与伦比的能力。
