如何使用 JMX 连接到在 EC2 上运行的 Java 实例

How to connect to Java instances running on EC2 using JMX

提问by Gray

We are having problem connecting to our Java applications running in Amazon's EC2 cluster. We definitely have allowed both the "JMX port" (which is usually the RMI registry port) andthe server port (which does most of the work) to the security-group for the instances in question. Jconsole connects but seems to hang and never show any information.

我们在连接到在 Amazon 的 EC2 集群中运行的 Java 应用程序时遇到问题。我们确实允许“JMX 端口”（通常是 RMI 注册端口）和服务器端口（完成大部分工作）到相关实例的安全组。Jconsole 连接但似乎挂起并且从不显示任何信息。

We are running our java with something like the following:

我们正在使用以下内容运行我们的 java：

java -server -jar foo.jar other parameters here > java.log 2>&1

We have tried:

我们已经尝试过：

• Telnets to the ports connectbut no information is displayed.
• We can run jconsoleon the instance itself using remote-X11 over ssh and it connects and shows information. So the JRE isexporting it locally.
• Opening all ports in the security group. Weeee.
• Using tcpdumpto make sure the traffic is not going to other ports.
• Simulating it locally. We can always connect to our local JREs or those running elsewhere on our network using the same application parameters.

• Telnet 到端口连接但不显示任何信息。
• 我们可以jconsole通过 ssh 使用 remote-X11 在实例本身上运行，它会连接并显示信息。因此JRE是本地出口的。
• 打开安全组中的所有端口。呜呜呜
• 使用tcpdump以确保交通不会到其他端口。
• 在本地模拟它。我们始终可以使用相同的应用程序参数连接到本地 JRE 或运行在网络其他地方的 JRE。

java -versionoutputs:

java -version输出：

OpenJDK Runtime Environment (IcedTea6 1.11.5) (amazon-53.1.11.5.47.amzn1-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

As an aside, we are using my Simple JMXpackage which allows us to set boththe RMI registry and server ports which are typically semi-randomly chosen by the RMI registry. You can also force this with something like the following JMX URI:

顺便说一句，我们使用的是我的简单的JMX包，它允许我们设置两个RMI注册表和服务器端口，一般是通过RMI注册表半随机选择的。您还可以使用类似以下 JMX URI 的内容强制执行此操作：

service:jmx:rmi://localhost:" + serverPort + "/jndi/rmi://:" + registryPort + "/jmxrmi"

These days we use the same port for both the server and the registry. In the past we have used Xas the registry-port and X+1for the server-port to make the security-group rules easy. You connect to the registry-port in jconsoleor whatever JMX client you are using.

这些天我们对服务器和注册表使用相同的端口。过去，我们使用X注册表端口和X+1服务器端口来简化安全组规则。您连接到注册表端口jconsole或您正在使用的任何 JMX 客户端。

回答by Gray

We are having problem connecting to our Java applications running in Amazon's EC2 cluster.

我们在连接到在 Amazon 的 EC2 集群中运行的 Java 应用程序时遇到问题。

It turns out that the problem was a combination of two missing settings. The first forces the JRE to prefer ipv4 and notv6. This was necessary (I guess) since we are trying to connect to it via a v4 address:

事实证明，问题是两个缺失设置的组合。第一个强制 JRE 更喜欢 ipv4 而不是v6。这是必要的（我猜），因为我们试图通过 v4 地址连接到它：

-Djava.net.preferIPv4Stack=true

The real blocker was the fact that JMX works by first contacting the RMI port which responds with the hostnameand port for the JMX client to connect. With no additional settings it will use the local IP of the box which is a 10.X.X.Xvirtual address which a remote client cannot route to. We needed to add the following setting which is the externalhostname or IP of the server -- in this case it is the elastic hostname of the server.

真正的障碍是 JMX 的工作原理是首先联系 RMI 端口，该端口响应主机名和 JMX 客户端连接的端口。无需额外设置，它将使用盒子的本地 IP，这是10.X.X.X远程客户端无法路由到的虚拟地址。我们需要添加以下设置，即服务器的外部主机名或 IP——在本例中，它是服务器的弹性主机名。

-Djava.rmi.server.hostname=ec2-107-X-X-X.compute-1.amazonaws.com

The trick, if you are trying to automate your EC2 instances (and why the hell would you not), is how to find this address at runtime. To do that you need to put something like the following in our application boot script:

如果您正在尝试自动化您的 EC2 实例（以及您为什么不这样做），诀窍是如何在运行时找到此地址。为此，您需要在我们的应用程序启动脚本中添加如下内容：

# get our _external_ hostname
RMI_HOST=`wget -q -O - http://169.254.169.254/latest/meta-data/public-hostname`
...
java -server \
    -Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=$RMI_HOST \
    -jar foo.jar other parameters here > java.log 2>&1

The mysterious 169.254.169.254IP in the wgetcommand above provides information that the EC2 instance can request about itself. I'm disappointed that this does notinclude tags which are only available in an authenticated call.

169.254.169.254上述wget命令中的神秘IP提供了 EC2 实例可以请求的有关自身的信息。我很失望，但这并不包括仅在一个验证呼叫可用标签。

I initially was using the extern ipv4 address but it looks like the JDK tries to make a connection to the server-port when it starts up. If it uses the external IP then this was slowing our application boot time until that timed out. The public-hostname resolves locally to the 10-net address and to the public-ipv4 externally. So the application now is starting fast and JMX clients still work. Woo hoo!

我最初使用的是 extern ipv4 地址，但看起来 JDK 在启动时尝试与服务器端口建立连接。如果它使用外部 IP，那么这会减慢我们的应用程序启动时间，直到超时。public-hostname 在本地解析为 10-net 地址，在外部解析为 public-ipv4。所以应用程序现在启动速度很快，JMX 客户端仍然可以工作。呜呼！

Hope this helps someone else. Cost me 3 hours today.

希望这对其他人有帮助。今天花了我3个小时。

To force your JMX server to start the server andthe RMI registry on designated ports so you can block them in the EC2 Security Groups, see this answer:

要强制您的 JMX 服务器在指定端口上启动服务器和RMI 注册表，以便您可以在 EC2 安全组中阻止它们，请参阅以下答案：

How to close rmiregistry running on particular port?

如何关闭在特定端口上运行的 rmiregistry？

Edit:

编辑：

We just had this problem re-occur. It seems that the Java JMX code is doing some hostname lookups on the hostname of the box and using them to try to connect and verify the JMX connection.

我们刚刚又出现了这个问题。似乎 Java JMX 代码正在对框的主机名进行一些主机名查找，并使用它们来尝试连接和验证 JMX 连接。

The issue seems to be a requirement that the local hostname of the box should resolve to the local-ip of the box. For example, if your /etc/sysconfig/networkhas HOSTNAME=server1.foobar.comthen if you do a DNS lookup on server1.foobar.com, you should get to the 10-NET virtual address. We were generating our own /etc/hostsfile and the hostname of the local host was missing from the file. This caused our applications to either pause on startup or not startup at all.

问题似乎是要求盒子的本地主机名应该解析为盒子的本地 IP。例如，如果您/etc/sysconfig/network有HOSTNAME=server1.foobar.com那么如果您在 上进行 DNS 查找server1.foobar.com，您应该会到达 10-NET 虚拟地址。我们正在生成自己的/etc/hosts文件，但文件中缺少本地主机的主机名。这导致我们的应用程序在启动时暂停或根本不启动。

Lastly

最后

One way to simplify your JMX creation is to use my SimpleJMX package.

简化 JMX 创建的一种方法是使用我的SimpleJMX 包。

回答by mmindenhall

Per the second answer Why does JMX connection to Amazon EC2 fail?, the difficulty here is that by default the RMI port is selected at random, and clients need access to both the JMX and RMI ports. If you're running jdk7u4 or later, the RMI port can be specified via an app property. Starting my server with the following JMX settings worked for me:

根据第二个答案为什么 JMX 连接到 Amazon EC2 失败？，这里的难点在于，默认情况下 RMI 端口是随机选择的，客户端需要同时访问 JMX 和 RMI 端口。如果您运行的是 jdk7u4 或更高版本，则可以通过 app 属性指定 RMI 端口。使用以下 JMX 设置启动我的服务器对我有用：

Without authentication:

无需身份验证：

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=9999 
-Dcom.sun.management.jmxremote.rmi.port=9998 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=false 
-Djava.rmi.server.hostname=<public EC2 hostname>

With authentication:

带身份验证：

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=9999 
-Dcom.sun.management.jmxremote.rmi.port=9998 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=true 
-Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password
-Djava.rmi.server.hostname=<public EC2 hostname>

I also opened ports 9998-9999 in the EC2 security group for my instance.

我还在 EC2 安全组中为我的实例打开了端口 9998-9999。

回答by Timofey

A bit different approach by using ssh tunnels

使用 ssh 隧道的有点不同的方法

1. (On the Remote machine) Pass the following flags to the JVM

   -Dcom.sun.management.jmxremote.port=1099
   -Djava.net.preferIPv4Stack=true
   -Dcom.sun.management.jmxremote.ssl=false
   -Dcom.sun.management.jmxremote.authenticate=false
   -Djava.rmi.server.hostname=127.0.0.1
   

2. (On the Remote machine) Check which ports java started to use

   $ netstat -tulpn | grep java
   tcp      0      0 0.0.0.0:37484         0.0.0.0:*               LISTEN      2904/java
   tcp      0      0 0.0.0.0:1099          0.0.0.0:*               LISTEN      2904/java
   tcp      0      0 0.0.0.0:45828         0.0.0.0:*               LISTEN      2904/java
   

3. (On the local machine) Make ssh tunnels for all the ports

   ssh -N -L 1099:127.0.0.1:1099 ubuntu@<ec2_ip>
   ssh -N -L 37484:127.0.0.1:37484 ubuntu@<ec2_ip>
   ssh -N -L 45828:127.0.0.1:45828 ubuntu@<ec2_ip>`
   

4. (On the local machine) Connect by Java Mission Control to localhost:1099

1. （在远程机器上）将以下标志传递给 JVM

   -Dcom.sun.management.jmxremote.port=1099
   -Djava.net.preferIPv4Stack=true
   -Dcom.sun.management.jmxremote.ssl=false
   -Dcom.sun.management.jmxremote.authenticate=false
   -Djava.rmi.server.hostname=127.0.0.1
   

2. （在远程机器上）检查 java 开始使用哪些端口

   $ netstat -tulpn | grep java
   tcp      0      0 0.0.0.0:37484         0.0.0.0:*               LISTEN      2904/java
   tcp      0      0 0.0.0.0:1099          0.0.0.0:*               LISTEN      2904/java
   tcp      0      0 0.0.0.0:45828         0.0.0.0:*               LISTEN      2904/java
   

3. （在本地机器上）为所有端口建立 ssh 隧道

   ssh -N -L 1099:127.0.0.1:1099 ubuntu@<ec2_ip>
   ssh -N -L 37484:127.0.0.1:37484 ubuntu@<ec2_ip>
   ssh -N -L 45828:127.0.0.1:45828 ubuntu@<ec2_ip>`
   

4. （在本地机器上）通过 Java Mission Control 连接到 localhost:1099

回答by Eric Pugh

The answer given by Gray worked for me, however I find that I have to open TCP ports 0 to 65535 or I don't get in. I thinkthat you can connect on the main JMX port, and then get another one assigned. I got that from this blog postthat has always worked well for me.

Gray 给出的答案对我有用，但是我发现我必须打开 TCP 端口 0 到 65535 否则我无法进入。我认为您可以连接到主 JMX 端口，然后再分配一个。我从这篇博文中得到了这个，它一直对我有用。

回答by anuj pradhan

We are using AWS Elastic Container Service for running our spring boot services. The below config allowed us to connect to our docker containers.

我们正在使用 AWS Elastic Container Service 来运行我们的 Spring Boot 服务。下面的配置允许我们连接到我们的 docker 容器。

Without Authentication:

无身份验证：

-Dcom.sun.management.jmxremote \
    -Dcom.sun.management.jmxremote.port=9090 \
    -Dcom.sun.management.jmxremote.rmi.port=9090 \
    -Dcom.sun.management.jmxremote.authenticate=false \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Djava.rmi.server.hostname=$(/usr/bin/curl -s --connect-timeout 2 \
                    http://169.254.169.254/latest/meta-data/public-ipv4)

I found it crisp and also doesn't require any other servicer side init script.

我发现它很清晰，也不需要任何其他服务端初始化脚本。