Python 如何在服务器上运行 OpenAI Gym .render()

Question

提问by Toke Faurby

I am running a python 2.7 script on a p2.xlarge AWSserver through Jupyter (Ubuntu 14.04). I would like to be able to render my simulations.

我正在通过 Jupyter（Ubuntu 14.04）在 p2.xlarge AWS服务器上运行 python 2.7 脚本。我希望能够渲染我的模拟。

Minimal working example

最小工作示例

import gym
env = gym.make('CartPole-v0')
env.reset()
env.render()

env.render()makes (among other things) the following errors:

env.render()使（除其他外）以下错误：

...
HINT: make sure you have OpenGL install. On Ubuntu, you can run 
'apt-get install python-opengl'. If you're running on a server, 
you may need a virtual frame buffer; something like this should work: 
'xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>'")
...
NoSuchDisplayException: Cannot connect to "None"

I would like to some how be able to see the simulations. It would be ideal if I could get it inline, but any display method would be nice.

我想知道一些如何能够看到模拟。如果我可以将它内联，那将是理想的，但任何显示方法都很好。

Edit: This is only an issue with some environments, like classic control.

编辑：这只是某些环境的问题，例如经典控件。

Update I

更新我

Inspired by thisI tried the following, instead of the xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>(which I couldn't get to work).

灵感来自这个我尝试以下，而不是xvfb-run -s \"-screen 0 1400x900x24\" python <your_script.py>（我不能去工作）。

xvfb-run -a jupyter notebook

Running the original script I now get instead

运行我现在得到的原始脚本

GLXInfoException: pyglet requires an X server with GLX

Update II

更新二

Issue #154seems relevant. I tried disabling the pop-up, and directly creating the RGB colors

问题#154似乎相关。我尝试禁用弹出窗口，并直接创建 RGB 颜色

import gym
env = gym.make('CartPole-v0')
env.reset()

img = env.render(mode='rgb_array', close=True)  
print(type(img)) # <--- <type 'NoneType'>

img = env.render(mode='rgb_array', close=False) # <--- ERROR
print(type(img))

I get ImportError: cannot import name gl_info.

我明白了ImportError: cannot import name gl_info。

Update III

更新三

With inspiration from @TorxedI tried creating a video file, and then rendering it (a fully satisfying solution).

受@ Torxed 的启发，我尝试创建一个视频文件，然后渲染它（一个完全令人满意的解决方案）。

Using the code from 'Recording and uploading results'

使用“记录和上传结果”中的代码

import gym

env = gym.make('CartPole-v0')
env.monitor.start('/tmp/cartpole-experiment-1', force=True)
observation = env.reset()
for t in range(100):
#    env.render()
    print(observation)
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done:
        print("Episode finished after {} timesteps".format(t+1))
        break

env.monitor.close()

I tried following your suggestions, but got ImportError: cannot import name gl_infofrom when running env.monitor.start(....

我尝试按照您的建议进行操作，但是ImportError: cannot import name gl_info从运行env.monitor.start(....

From my understanding the problem is that OpenAI uses pyglet, and pyglet'needs' a screen in order to compute the RGB colors of the image that is to be rendered. It is therefore necessary to trick python to think that there is a monitor connected

根据我的理解，问题在于 OpenAI 使用pyglet, 并且pyglet“需要”一个屏幕来计算要渲染的图像的 RGB 颜色。因此有必要欺骗python认为有一个监视器连接

Update IV

更新四

FYI there are solutions online using bumblebee that seem to work. This should work if you have control over the server, but since AWS run in a VM I don't think you can use this.

仅供参考，有一些使用大黄蜂的在线解决方案似乎有效。如果您可以控制服务器，这应该可以工作，但是由于 AWS 在 VM 中运行，我认为您不能使用它。

Update V

更新五

Just if you have this problem, and don't know what to do (like me) the state of most environments are simple enough that you can create your own rendering mechanism. Not very satisfying, but.. you know.

如果你有这个问题，并且不知道该怎么做（像我一样）大多数环境的状态很简单，你可以创建自己的渲染机制。不是很满意，但是……你知道。

Answer 1

回答by Andrew Schreiber

Got a simple solution working:

有一个简单的解决方案：

CartPole

如果在 linux 服务器上，请使用以下命令打开 jupyter

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

在 Jupyter 中

import matplotlib.pyplot as plt
%matplotlib inline
from IPython import display

每一步之后

def show_state(env, step=0, info=""):
    plt.figure(3)
    plt.clf()
    plt.imshow(env.render(mode='rgb_array'))
    plt.title("%s | Step: %d %s" % (env._spec.id,step, info))
    plt.axis('off')

    display.clear_output(wait=True)
    display.display(plt.gcf())

Note: if your environment is not unwrapped, pass env.envto show_state.

注意：如果您的环境不是unwrapped，则传递env.env到show_state。

Answer 2

回答by Nathan

ThisGitHub issue gave an answer that worked great for me. It's nice because it doesn't require any additional dependencies (I assume you already have matplotlib) or configuration of the server.

这个GitHub 问题给出了一个对我很有用的答案。这很好，因为它不需要任何额外的依赖项（我假设您已经有了matplotlib）或服务器配置。

Just run, e.g.:

只需运行，例如：

import gym
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0') # insert your favorite environment
render = lambda : plt.imshow(env.render(mode='rgb_array'))
env.reset()
render()

Using mode='rgb_array'gives you back a numpy.ndarraywith the RGB values for each position, and matplotlib's imshow(or other methods) displays these nicely.

Usingmode='rgb_array'为您numpy.ndarray提供每个位置的 RGB 值，并且matplotlib's imshow（或其他方法）可以很好地显示这些值。

Note that if you're rendering multipletimes in the same cell, this solution will plot a separate image each time. This is probably not what you want. I'll try to update this if I figure out a good workaround for that.

请注意，如果您需要将多个在同一细胞时，这种解决方案将每次绘制一个单独的图像。这可能不是您想要的。如果我找到一个好的解决方法，我会尝试更新它。

Update to render multiple times in one cell

更新以在一个单元格中多次渲染

Based on thisStackOverflow answer, here's a working snippet (note that there may be more efficient ways to do this with an interactive plot; this way seems a little laggy on my machine):

基于此StackOverflow 答案，这是一个工作片段（请注意，使用交互式绘图可能有更有效的方法来执行此操作；这种方式在我的机器上似乎有点滞后）：

import gym
from IPython import display
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
for _ in range(100):
    plt.imshow(env.render(mode='rgb_array'))
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Update to increase efficiency

更新以提高效率

On my machine, this was about 3x faster. The difference is that instead of calling imshoweach time we render, we just change the RGB data on the original plot.

在我的机器上，这大约快了 3 倍。不同之处在于，不是imshow每次渲染时都调用，我们只是更改原始图上的 RGB 数据。

import gym
from IPython import display
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
img = plt.imshow(env.render(mode='rgb_array')) # only call this once
for _ in range(100):
    img.set_data(env.render(mode='rgb_array')) # just update the data
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Answer 3

回答by Van

I managed to run and render openai/gym (even with mujoco) remotely on a headless server.

我设法在无头服务器上远程运行和渲染 openai/gym（甚至使用 mujoco）。

# Install and configure X window with virtual screen
sudo apt-get install xserver-xorg libglu1-mesa-dev freeglut3-dev mesa-common-dev libxmu-dev libxi-dev
# Configure the nvidia-x
sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024
# Run the virtual screen in the background (:0)
sudo /usr/bin/X :0 &
# We only need to setup the virtual screen once

# Run the program with vitural screen
DISPLAY=:0 <program>

# If you dont want to type `DISPLAY=:0` everytime
export DISPLAY=:0

Usage:

用法：

DISPLAY=:0 ipython2

Example:

例子：

import gym
env = gym.make('Ant-v1')
arr = env.render(mode='rgb_array')
print(arr.shape)
# plot or save wherever you want
# plt.imshow(arr) or scipy.misc.imsave('sample.png', arr)

Answer 4

回答by Tr?n Lê Thái S?n

I think we should just capture renders as video by using OpenAI Gym wrappers.Monitorand then display it within the Notebook.

我认为我们应该使用 OpenAI Gym 将渲染捕获为视频wrappers.Monitor，然后将其显示在 Notebook 中。

Example:

例子：

Dependencies

依赖关系

!apt install python-opengl
!apt install ffmpeg
!apt install xvfb
!pip3 install pyvirtualdisplay

# Virtual display
from pyvirtualdisplay import Display

virtual_display = Display(visible=0, size=(1400, 900))
virtual_display.start()

Capture as video

捕获为视频

import gym
from gym import wrappers

env = gym.make("SpaceInvaders-v0")
env = wrappers.Monitor(env, "/tmp/SpaceInvaders-v0")

for episode in range(2):
    observation = env.reset()
    step = 0
    total_reward = 0

    while True:
        step += 1
        env.render()
        action = env.action_space.sample()
        observation, reward, done, info = env.step(action)
        total_reward += reward
        if done:
            print("Episode: {0},\tSteps: {1},\tscore: {2}"
                  .format(episode, step, total_reward)
            )
            break
env.close()

Display within Notebook

在笔记本内显示

import os
import io
import base64
from IPython.display import display, HTML

def ipython_show_video(path):
    """Show a video at `path` within IPython Notebook
    """
    if not os.path.isfile(path):
        raise NameError("Cannot access: {}".format(path))

    video = io.open(path, 'r+b').read()
    encoded = base64.b64encode(video)

    display(HTML(
        data="""
        <video alt="test" controls>
        <source src="data:video/mp4;base64,{0}" type="video/mp4" />
        </video>
        """.format(encoded.decode('ascii'))
    ))

ipython_show_video("/tmp/SpaceInvaders-v0/openaigym.video.4.10822.video000000.mp4")

I hope it helps. ;)

我希望它有帮助。;)

Answer 5

回答by mdaoust

There's also this solutionusing pyvirtualdisplay(an Xvfb wrapper). One thing I like about this solution is you can launch it from inside your script, instead of having to wrap it at launch:

还有这个解决方案使用pyvirtualdisplay（一个 Xvfb 包装器）。我喜欢这个解决方案的一件事是您可以从脚本内部启动它，而不必在启动时包装它：

from pyvirtualdisplay import Display
display = Display(visible=0, size=(1400, 900))
display.start()

Answer 6

回答by I_like_foxes

I ran into this myself. Using xvfb as X-server somehow clashes with the Nvidia drivers. But finally thispost pointed me into the right direction. Xvfb works without any problems if you install the Nvidia driver with the -no-opengl-filesoption and CUDA with --no-opengl-libsoption. If you know this, it should work. But as it took me quite some time till I figured this out and it seems like I'm not the only one running into problems with xvfb and the nvidia drivers.

我自己遇到了这个。使用 xvfb 作为 X 服务器在某种程度上与 Nvidia 驱动程序发生冲突。但最后这篇文章为我指明了正确的方向。如果您安装带有-no-opengl-files选项的 Nvidia 驱动程序和带有--no-opengl-libs选项的CUDA，Xvfb 可以正常工作。如果你知道这一点，它应该工作。但是因为我花了很长时间才弄明白这一点，而且似乎我不是唯一遇到 xvfb 和 nvidia 驱动程序问题的人。

I wrote down all necessary steps to set everything up on an AWS EC2 instance with Ubuntu 16.04 LTS here.

我在这里写下了在带有 Ubuntu 16.04 LTS 的 AWS EC2 实例上设置所有内容的所有必要步骤。

Answer 7

回答by Doug Blank

I avoided the issues with using matplotlib by simply using PIL, Python Image Library:

我通过简单地使用 PIL、Python 图像库避免了使用 matplotlib 的问题：

import gym, PIL
env = gym.make('SpaceInvaders-v0')
array = env.reset()
PIL.Image.fromarray(env.render(mode='rgb_array'))

I found that I didn't need to set the XV frame buffer.

我发现我不需要设置 XV 帧缓冲区。

Answer 8

回答by martinenzinger

I was looking for a solution that works in Colaboratory and ended up with this

我一直在寻找一个适用于 Colaboratory 的解决方案，最终得到了这个

from IPython import display
import numpy as np
import time

import gym
env = gym.make('SpaceInvaders-v0')
env.reset()

import PIL.Image
import io


def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

while True:
    time.sleep(0.01)
    env.step(env.action_space.sample()) # take a random action
    display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')

EDIT 1:

编辑 1：

You could use xvfbwrapper for the Cartpole environment.

您可以将 xvfbwrapper 用于 Cartpole 环境。

from IPython import display
from xvfbwrapper import Xvfb
import numpy as np
import time
import pyglet
import gym
import PIL.Image
import io    

vdisplay = Xvfb(width=1280, height=740)
vdisplay.start()

env = gym.make('CartPole-v0')
env.reset()

def showarray(a, fmt='png'):
    a = np.uint8(a)
    f = io.BytesIO()
    ima = PIL.Image.fromarray(a).save(f, fmt)
    return f.getvalue()

imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


for _ in range(1000):
  time.sleep(0.01)
  observation, reward, done, info = env.step(env.action_space.sample()) # take a random action
  display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')


vdisplay.stop()

If you're working with standard Jupyter, there's a better solution though. You can use the CommManager to send messages with updated Data URLs to your HTML output.

如果您使用标准 Jupyter，则有更好的解决方案。您可以使用 CommManager 将带有更新数据 URL 的消息发送到您的 HTML 输出。

IPython Inline Screen Example

IPython 内联屏幕示例

In Colab the CommManager is not available. The more restrictive output module has a method called eval_js() which seems to be kind of slow.

在 Colab 中，CommManager 不可用。更具限制性的输出模块有一个名为 eval_js() 的方法，它似乎有点慢。

Answer 9

回答by Eoin Murray

Referencing my other answer here: Display OpenAI gym in Jupyter notebook only

在此处引用我的其他答案：仅在 Jupyter 笔记本中显示 OpenAI 健身房

I made a quick working example here which you could fork: https://kyso.io/eoin/openai-gym-jupyterwith two examples of rendering in Jupyter - one as an mp4, and another as a realtime gif.

我在这里做了一个快速的工作示例，你可以分叉：https: //kyso.io/eoin/openai-gym-jupyter 有两个在 Jupyter 中渲染的示例 - 一个作为 mp4，另一个作为实时 gif。

The .mp4 example is quite simple.

.mp4 示例非常简单。

import gym
from gym import wrappers

env = gym.make('SpaceInvaders-v0')
env = wrappers.Monitor(env, "./gym-results", force=True)
env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done: break
env.close()

Then in a new cell Jupyter cell, or download it from the server onto some place where you can view the video.

然后在一个新的单元格 Jupyter 单元格中，或将它从服务器下载到某个可以观看视频的地方。

import io
import base64
from IPython.display import HTML

video = io.open('./gym-results/openaigym.video.%s.video000000.mp4' % env.file_infix, 'r+b').read()
encoded = base64.b64encode(video)
HTML(data='''
    <video width="360" height="auto" alt="test" controls><source src="data:video/mp4;base64,{0}" type="video/mp4" /></video>'''
.format(encoded.decode('ascii')))

If your on a server with public access you could run python -m http.serverin the gym-results folder and just watch the videos there.

如果您在具有公共访问权限的服务器上，您可以python -m http.server在健身房结果文件夹中运行并在那里观看视频。

Answer 10

回答by Costa Huang

This might be a complete workaround, but I used a docker image with a desktop environment, and it works great. The docker image is at https://hub.docker.com/r/dorowu/ubuntu-desktop-lxde-vnc/

这可能是一个完整的解决方法，但我在桌面环境中使用了 docker 映像，并且效果很好。docker 镜像位于https://hub.docker.com/r/dorowu/ubuntu-desktop-lxde-vnc/

The command to run is

要运行的命令是

docker run -p 6080:80 dorowu/ubuntu-desktop-lxde-vnc

Then browse http://127.0.0.1:6080/to access the Ubuntu desktop.

然后浏览http://127.0.0.1:6080/访问 Ubuntu 桌面。

Below are a gif showing it the Mario bros gym environment running and being rendered. As you can see, it is fairly responsive and smooth.

下面是一个 gif，显示了正在运行和渲染的马里奥兄弟健身房环境。如您所见，它相当灵敏且流畅。

Python 如何在服务器上运行 OpenAI Gym .render()

提问by Toke Faurby

回答by Andrew Schreiber

回答by Nathan

Update to render multiple times in one cell

更新以在一个单元格中多次渲染

Update to increase efficiency

更新以提高效率

回答by Van

回答by Tr?n Lê Thái S?n

Dependencies

依赖关系

Capture as video

捕获为视频

Display within Notebook

在笔记本内显示

回答by mdaoust

回答by I_like_foxes

回答by Doug Blank

回答by martinenzinger

回答by Eoin Murray

回答by Costa Huang

相关推荐

最近更新

标签

Python 如何在服务器上运行 OpenAI Gym .render()

提问by Toke Faurby

回答by Andrew Schreiber

回答by Nathan

Update to render multiple times in one cell

更新以在一个单元格中多次渲染

Update to increase efficiency

更新以提高效率

回答by Van

回答by Tr?n Lê Thái S?n

Dependencies

依赖关系

Capture as video

捕获为视频

Display within Notebook

在笔记本内显示

回答by mdaoust

回答by I_like_foxes

回答by Doug Blank

回答by martinenzinger

回答by Eoin Murray

回答by Costa Huang

相关推荐

如何在 Visual Studio 2017 中向 python 添加包

Python 使用相同的图形在 TensorFlow 中显示训练和验证的准确性

Python 无法在 Windows 10 上为 GPU 导入 Tensorflow

Python 如何在 Airflow 中运行 bash 脚本文件

相关推荐

最近更新

标签