Java Tomcat 的集群/会话复制无法正确复制

Tomcat's Clustering / Session Replication not replicating properly

提问by nvioli

I'm setting up clustering/replication on Tomcat 7 on my local machine, to evaluate it for use with my environment/codebase.

我正在本地机器上的 Tomcat 7 上设置集群/复制，以评估它是否与我的环境/代码库一起使用。

Setup

设置

I have two identical tomcat servers in sibling directories running on different ports. I have httpd listening on two other ports and connecting to the two tomcat instances as VirtualHosts. I can access and interact with both environments on the configured ports; everything is working as expected.

我在不同端口上运行的同级目录中有两个相同的 tomcat 服务器。我有 httpd 侦听其他两个端口并作为 VirtualHosts 连接到两个 tomcat 实例。我可以在配置的端口上访问两个环境并与之交互；一切都按预期工作。

The tomcat servers have clustering enabled like this, in server.xml:

tomcat 服务器在 server.xml 中启用了这样的集群：

   <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
             channelSendOptions="8">

      <Manager className="org.apache.catalina.ha.session.DeltaManager"
               expireSessionsOnShutdown="false"
               notifyListenersOnReplication="true"/>

      <Channel className="org.apache.catalina.tribes.group.GroupChannel">
        <Membership className="org.apache.catalina.tribes.membership.McastService"
                    address="228.0.0.4"
                    port="45564"
                    frequency="500"
                    dropTime="3000"/>
        <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                  address="auto"
                  port="4001"
                  autoBind="100"
                  selectorTimeout="5000"
                  maxThreads="6"/>

        <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
          <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
        </Sender>
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
      </Channel>

      <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
             filter=""/>
      <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

      <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                tempDir="/tmp/war-temp/"
                deployDir="/tmp/war-deploy/"
                watchDir="/tmp/war-listen/"
                watchEnabled="false"/>

      <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
      <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
   </Cluster>

and I added the distributable tag to the very beginning of web.xml:

我在 web.xml 的开头添加了 distributable 标签：

<web-app xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                      http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
  version="3.0">
  <distributable />

  (lots more...)

</web-app>

What's working

什么在起作用

When the servers start, they log

当服务器启动时，它们会记录

Sep 16, 2013 1:44:23 PM org.apache.catalina.ha.tcp.SimpleTcpCluster startInternal
INFO: Cluster is about to start
Sep 16, 2013 1:44:23 PM org.apache.catalina.tribes.transport.ReceiverBase getBind
FINE: Starting replication listener on address:10.0.0.100
Sep 16, 2013 1:44:23 PM org.apache.catalina.tribes.transport.ReceiverBase bind
INFO: Receiver Server Socket bound to:/10.0.0.100:4001
Sep 16, 2013 1:44:23 PM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast soTimeout to 500
Sep 16, 2013 1:44:23 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4
Sep 16, 2013 1:44:24 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:4
Sep 16, 2013 1:44:24 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:8
Sep 16, 2013 1:44:25 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:8

When the second server starts up, the first one logs

当第二个服务器启动时，第一个记录

Sep 16, 2013 2:17:30 PM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector messageReceived
FINE: Received a failure detector packet:ClusterData[src=org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4000,{10, 0, 0, 100},4000, alive=112208, securePort=-1, UDP Port=-1, id={118 6 107 -67 88 98 72 95 -73 41 4 -108 58 -5 -127 -41 }, payload={}, command={}, domain={}, ]; id={25 110 120 -2 -25 6 78 -97 -84 -34 2 -11 49 -62 -8 -56 }; sent=2013-09-16 14:17:30.139]
Sep 16, 2013 2:17:30 PM org.apache.catalina.tribes.transport.nio.NioReplicationTask remoteEof
FINE: Channel closed on the remote end, disconnecting
Sep 16, 2013 2:17:30 PM org.apache.catalina.tribes.membership.McastServiceImpl memberDataReceived
FINE: Mcast add member org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4001,{10, 0, 0, 100},4001, alive=1010, securePort=-1, UDP Port=-1, id={82 -45 -109 -56 -110 -5 78 -10 -103 61 -40 -59 -36 -79 104 120 }, payload={}, command={}, domain={}, ]
Sep 16, 2013 2:17:30 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4001,{10, 0, 0, 100},4001, alive=1011, securePort=-1, UDP Port=-1, id={82 -45 -109 -56 -110 -5 78 -10 -103 61 -40 -59 -36 -79 104 120 }, payload={}, command={}, domain={}, ]

and when one is shutdown, the other one logs

当一个关闭时，另一个记录

Sep 16, 2013 2:28:05 PM org.apache.catalina.tribes.membership.McastServiceImpl memberDataReceived
FINE: Member has shutdown:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4001,{10, 0, 0, 100},4001, alive=422279, securePort=-1, UDP Port=-1, id={54 43 17 -9 13 -11 72 -63 -107 -78 -8 65 -21 -77 115 88 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]
Sep 16, 2013 2:28:05 PM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector memberDisappeared
INFO: Verification complete. Member disappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4001,{10, 0, 0, 100},4001, alive=422279, securePort=-1, UDP Port=-1, id={54 43 17 -9 13 -11 72 -63 -107 -78 -8 65 -21 -77 115 88 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]]
Sep 16, 2013 2:28:05 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberDisappeared
INFO: Received member disappeared:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 100}:4001,{10, 0, 0, 100},4001, alive=422279, securePort=-1, UDP Port=-1, id={54 43 17 -9 13 -11 72 -63 -107 -78 -8 65 -21 -77 115 88 }, payload={}, command={66 65 66 89 45 65 76 69 88 ...(9)}, domain={}, ]

so I know they're aware of each other.

所以我知道他们知道彼此。

Finally, when I use the Cluster/Operations MBean in jconsole to try to set property "foo" to "bar", jconsole reports "method successfully invoked", and the server logs

最后，当我在 jconsole 中使用 Cluster/Operations MBean 尝试将属性“foo”设置为“bar”时，jconsole 报告“方法已成功调用”，并且服务器记录

Sep 16, 2013 2:30:18 PM org.apache.catalina.ha.tcp.SimpleTcpCluster setProperty
WARNING: Dynamic setProperty(foo,value) has been disabled, please use explicit properties for the element you are trying to identify

I'm not too worried about that error; mostly included to demonstrate that setProperty creates a log statement.

我不太担心那个错误；主要用于演示 setProperty 创建日志语句。

What's not working

什么不起作用

As far as I can tell, no session information is being replicated in my app.

据我所知，我的应用程序中没有复制任何会话信息。

The tomcat manager only lists sessions started on the server it's monitoring, and not the other one in the cluster.

tomcat 管理器只列出在它所监控的服务器上启动的会话，而不是集群中的另一个。

I'm under the impression that whenever the app calls HttpSession.setAttribute, that attribute should be replicated to the other cluster nodes, and I would expect that some record of that would be logged. My app includes this line:

我的印象是，每当应用程序调用 HttpSession.setAttribute 时，该属性都应该复制到其他集群节点，我希望会记录一些记录。我的应用程序包括这一行：

   public static void saveBillingInfo(IPageContext pageContext, BillingInfo billingInfo)
   {    
      pageContext.getSession().setAttribute("billingInfo", billingInfo);
      //etc...
   }

where BillingInfo is a Serializable class containing only one field, a HashMap of information about the billing info.

其中 BillingInfo 是一个 Serializable 类，仅包含一个字段，即有关计费信息的信息的 HashMap。

No log statements are written when this or any other line processes, and I don't see any evidence that session information is actually being shared.

当此行或任何其他行处理时，不会写入任何日志语句，并且我没有看到任何证据表明会话信息实际上正在共享。

Any suggestions or further questions are welcome.

欢迎任何建议或进一步的问题。

采纳答案by Jason

We had this identical issue. Although not documented anywhere, what solved it for me was to simply move the <Manager> tag from server.xml to the global context.xml (bringing it out of the <Server>...<Cluster>... group and into the <Context> group). As soon as we did this, everything "magically" began working. This only applied to Tomcat 7...Tomcat 6 worked perfectly with the setup you describe above (and as the documentation describes).

我们有这个相同的问题。虽然没有在任何地方记录，但对我来说解决它的是简单地将 <Manager> 标记从 server.xml 移动到全局 context.xml（将它从 <Server>...<Cluster>... 组中取出并放入<Context> 组）。一旦我们这样做了，一切都“神奇地”开始工作。这仅适用于 Tomcat 7...Tomcat 6 与您在上面描述的设置（以及文档描述）完美配合。

<Context>
    <Manager className="org.apache.catalina.ha.session.DeltaManager"
             expireSessionsOnShutdown="false"
             notifyListenersOnReplication="true" />
</Context>

Now just remove the <Manager> tag from your Cluster group in server.xml and you're done.

现在只需从 server.xml 中的集群组中删除 <Manager> 标记即可。

回答by Fabian Mejia

on server.xml just add:

在 server.xml 上添加：

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
             channelSendOptions="6"/>  

and, in your context.xml:

并且，在您的 context.xml 中：

<Manager className="org.apache.catalina.ha.session.DeltaManager"
         expireSessionsOnShutdown="false"
         notifyListenersOnReplication="true" />

it worked for me, i'm using Tomcat 7 and Tomcat 8

它对我有用，我正在使用 Tomcat 7 和 Tomcat 8

回答by David I

Our solution was to add the <distributable/>tag to the WEB-INF/web.xmlfile under the <web-app>element. Not sure why this worked for us an not for the OP.

我们的解决方案是将<distributable/>标签添加到元素WEB-INF/web.xml下的文件中<web-app>。不知道为什么这对我们有用而不对 OP 有用。

By the way, Jason's solution worked for us too.

顺便说一下，Jason 的解决方案也适用于我们。

回答by user6649882

According to Tomcat docs you need to do one of these two (NOT both):

根据 Tomcat 文档，您需要执行以下两项操作之一（不是同时执行）：

1. Specify in web.xml <distributable/>to enable clustering and use the default <Manager>specified in the server.xml or...

2. Add a <Manager>at the application level inside context.xml

1. 在 web.xml 中指定<distributable/>以启用集群并使用<Manager>server.xml 中指定的默认值或...

2. <Manager>在 context.xml 中的应用程序级别添加一个

回答by Thanh Nguyen

If you want to enable Clustering for specific deployed application, you can using below example

如果要为特定部署的应用程序启用集群，可以使用以下示例

server.xml

服务器.xml

  <Host name="localhost"  appBase="webapps"
        unpackWARs="true" autoDeploy="true">

      <Channel className="org.apache.catalina.tribes.group.GroupChannel">
        <Membership className="org.apache.catalina.tribes.membership.McastService"
                    address="228.0.0.4"
                    port="45564"
                    frequency="500"
                    dropTime="3000"/>
        <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                  address="auto"
                  port="4000"
                  autoBind="100"
                  selectorTimeout="5000"
                  maxThreads="6"/>

        <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
          <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
        </Sender>
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
      </Channel>

      <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
             filter=""/>
      <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

      <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                tempDir="/tmp/war-temp/"
                deployDir="/tmp/war-deploy/"
                watchDir="/tmp/war-listen/"
                watchEnabled="false"/>

      <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
      <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>

      <Manager className="org.apache.catalina.ha.session.DeltaManager"
       expireSessionsOnShutdown="false"
       notifyListenersOnReplication="true"/>

    <Resource auth="Container" driverClassName="oracle.jdbc.OracleDriver" factory="com.atomikos.tomcat.EnhancedTomcatAtomikosBeanFactory" minPoolSize="4" maxPoolSize="50" name="jdbc/myoracle/myconect" password="sdfhsfghsgfsfg" type="com.atomikos.jdbc.nonxa.AtomikosNonXADataSourceBean" uniqueResourceName="uniresource" url="jdbc:oracle:thin:@10.10.10.10.:1521:abc" user="user"/>
    </Context>