Python 无法使用 MLPRegressor 拟合简单数据

Trouble fitting simple data with MLPRegressor

提问by Robert Altena

I am trying out Python and scikit-learn. I cannot get MLPRegressor to come even close to the data. Where is this going wrong?

我正在尝试 Python 和 scikit-learn。我无法让 MLPRegressor 接近数据。这是哪里出错了？

from sklearn.neural_network import MLPRegressor
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0.0, 1, 0.01).reshape(-1, 1)
y = np.sin(2 * np.pi * x).ravel()

reg = MLPRegressor(hidden_layer_sizes=(10,),  activation='relu', solver='adam',    alpha=0.001,batch_size='auto',
               learning_rate='constant', learning_rate_init=0.01, power_t=0.5, max_iter=1000, shuffle=True,
               random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9,
               nesterovs_momentum=True, early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999,
               epsilon=1e-08)

reg = reg.fit(x, y)

test_x = np.arange(0.0, 1, 0.05).reshape(-1, 1)
test_y = reg.predict(test_x)

fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(x, y, s=10, c='b', marker="s", label='real')
ax1.scatter(test_x,test_y, s=10, c='r', marker="o", label='NN Prediction')

plt.show()

The result is not very good: Thank you.

结果不是很好： 谢谢。

回答by JiaweiZhuang

You just need to

你只需要

• change the solver to 'lbfgs'. The default'adam'is a SGD-like method, which is effective for large & messy data but pretty useless for this kind of smooth & small data.
• use a smooth activation function such as tanh. reluis almost linear, not suited for learning this simple non-linear function.

• 将求解器更改为'lbfgs'. 默认'adam'是类似 SGD 的方法，它对大而杂乱的数据很有效，但对这种平滑和小数据来说却毫无用处。
• 使用平滑的激活函数，例如tanh。relu几乎是线性的，不适合学习这种简单的非线性函数。

Here're the resultand the complete code. Even just 3 hidden neurons can achieve very high accuracy.

这是结果和完整的代码。即使只有 3 个隐藏的神经元也可以达到非常高的准确率。

from sklearn.neural_network import MLPRegressor
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0.0, 1, 0.01).reshape(-1, 1)
y = np.sin(2 * np.pi * x).ravel()

nn = MLPRegressor(hidden_layer_sizes=(3), 
                  activation='tanh', solver='lbfgs')

n = nn.fit(x, y)
test_x = np.arange(-0.1, 1.1, 0.01).reshape(-1, 1)
test_y = nn.predict(test_x)
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(x, y, s=5, c='b', marker="o", label='real')
ax1.plot(test_x,test_y, c='r', label='NN Prediction')

plt.legend()
plt.show()

回答by Oleg Melnikov

There are too few points to fit for this non-nonlinear model, so the fit is sensitive to the seed. A good seed helps, but it is not known a priori. You can also add more data points.

适合这个非线性模型的点太少，所以拟合对种子很敏感。一个好的种子有帮助，但它不是先验的。您还可以添加更多数据点。

By iterating through various seeds, I determined random_state=9to work well. Surely there are others.

通过迭代各种种子，我决定random_state=9好好工作。当然还有其他人。

from sklearn.neural_network import MLPRegressor
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0.0, 1, 0.01).reshape(-1, 1)
y = np.sin(2 * np.pi * x).ravel()

nn = MLPRegressor(
    hidden_layer_sizes=(10,),  activation='relu', solver='adam', alpha=0.001, batch_size='auto',
    learning_rate='constant', learning_rate_init=0.01, power_t=0.5, max_iter=1000, shuffle=True,
    random_state=9, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True,
    early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08)

n = nn.fit(x, y)
test_x = np.arange(0.0, 1, 0.05).reshape(-1, 1)
test_y = nn.predict(test_x)
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(x, y, s=1, c='b', marker="s", label='real')
ax1.scatter(test_x,test_y, s=10, c='r', marker="o", label='NN Prediction')
plt.show()

Here are the absolute errors of fits for seed integers i = 0..9:

以下是种子整数拟合的绝对误差i = 0..9：

print(i, sum(abs(test_y - np.sin(2 * np.pi * test_x).ravel())))

which yields:

产生：

0 13.0874999193
1 7.2879574143
2 6.81003360188
3 5.73859777885
4 12.7245375367
5 7.43361211586
6 7.04137436733
7 7.42966661997
8 7.35516939164
9 2.87247035261

Now, we can still improve fitting even with random_state=0by increasing number of target points from 100 to 1000 and the size of hidden layers from 10 to 100:

现在，即使random_state=0将目标点的数量从 100 增加到 1000，隐藏层的大小从 10 增加到 100 ，我们仍然可以改进拟合：

from sklearn.neural_network import MLPRegressor
import numpy as np
import matplotlib.pyplot as plt

x = np.arange(0.0, 1, 0.001).reshape(-1, 1)
y = np.sin(2 * np.pi * x).ravel()

nn = MLPRegressor(
    hidden_layer_sizes=(100,),  activation='relu', solver='adam', alpha=0.001, batch_size='auto',
    learning_rate='constant', learning_rate_init=0.01, power_t=0.5, max_iter=1000, shuffle=True,
    random_state=0, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True,
    early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08)

n = nn.fit(x, y)
test_x = np.arange(0.0, 1, 0.05).reshape(-1, 1)
test_y = nn.predict(test_x)
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(x, y, s=1, c='b', marker="s", label='real')
ax1.scatter(test_x,test_y, s=10, c='r', marker="o", label='NN Prediction')
plt.show()

Yielding:

产量：

Btw, some parameters are unnecessary in your MLPRegressor(), such as momentum, nesterovs_momentum, etc. Check documentation. Also, it helps to seed your examples to make sure the results are reproducible ;)

顺便说一句，有些参数是在你不需要的MLPRegressor()，比如momentum，nesterovs_momentum等检查文档。此外，它有助于为您的示例提供种子以确保结果可重复；)