java 数千个主题的高性能 DefaultMessageListenerContainer 配置
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/10948943/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
High Performance DefaultMessageListenerContainer config for thousands of topics
提问by Laures
for my current project i need to consume messages from many destinations (from hundreds up to 20 or 30k) all destinations are topics. currently (for initial load tests) all messages are created localy on the same server, in a thread pool.
对于我当前的项目，我需要使用来自许多目的地（从数百个到 20 或 30k）的消息，所有目的地都是主题。当前（对于初始负载测试）所有消息都在同一服务器的本地线程池中创建。
my current spring config uses an embedded activemq in a network of brokers (for clustering) and DefaultMessageListenerContainers (DMLCs) with a common TaskExecutor. while the number of destinations is very high, the throughput of each destination is relatively low.
我当前的 spring 配置在代理网络（用于集群）和 DefaultMessageListenerContainers (DMLCs) 网络中使用嵌入式 activemq，并带有一个公共 TaskExecutor。虽然目的地数量非常多，但每个目的地的吞吐量相对较低。
My only requirement is that all messages are consumed as soon as possible.
我唯一的要求是尽快消耗所有消息。
My Config:
我的配置：
<bean id="connectionfactory" class="org.springframework.jms.connection.CachingConnectionFactory" destroy-method="destroy">
<property name="targetConnectionFactory">
<ref bean="amqConnectionFactory" />
</property>
</bean>
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost:61616?async=false&jms.dispatchAsync=false" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
<bean id="listenerThreadPoolTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="70" />
<property name="maxPoolSize" value="70" />
<property name="daemon" value="true" />
<property name="keepAliveSeconds" value="60" />
</bean>
<!-- Message Listener Container Template for Topics -->
<bean id="topiccontainertemplate" class="org.springframework.jms.listener.DefaultMessageListenerContainer" scope="prototype"
destroy-method="destroy">
<property name="autoStartup" value="false" />
<property name="connectionFactory" ref="connectionfactory" />
<property name="pubSubDomain" value="true" />
<property name="cacheLevelName" value="CACHE_CONSUMER" />
<property name="destinationName" value="default" />
<property name="maxMessagesPerTask" value="1" />
<property name="receiveTimeout" value="1" />
<property name="taskExecutor" ref="listenerThreadPoolTaskExecutor" />
</bean>
My code uses the application context as a DMLC-Factory and sets the final configuration of the container:
我的代码使用应用程序上下文作为 DMLC-Factory 并设置容器的最终配置：
AbstractMessageListenerContainer container = context.getBean("simpletopiccontainertemplate", AbstractMessageListenerContainer.class);
container.setDestinationName(localEntity.getId().getDestination());
container.setMessageListener(mylistener);
container.start();
while we don't lose messages in this configuration the transition time for an individual message can be very long.
虽然在此配置中我们不会丢失消息，但单个消息的转换时间可能很长。
Q1:Is there a more efficient way to listen to a big number of destinations?
Q1：有没有更有效的方式来收听大量目的地？
Q2:Are there possible improvements for my listener configuration?
Q2：我的监听器配置有可能改进吗？
Q3:Beside the DMLC i also tried the SimpleMessageListenerContainer but i couldn't get it to work. is there something wrong with my config?
Q3：除了 DMLC，我还尝试了 SimpleMessageListenerContainer，但我无法让它工作。我的配置有问题吗？
<bean id="simpletopiccontainertemplate" class="org.springframework.jms.listener.SimpleMessageListenerContainer" scope="prototype"
destroy-method="destroy">
<property name="autoStartup" value="false" />
<property name="connectionFactory" ref="connectionfactory" />
<property name="destinationName" value="default" />
<property name="concurrency" value="1" />
<property name="pubSubDomain" value="true" />
<property name="taskExecutor" ref="listenerThreadPoolTaskExecutor" />
</bean>
采纳答案by Laures
I moved away from the DefaultMessageListenerContainerto the SimpleMessageListenerContainer. The simple container uses push messaging. This way (and with appropriate config the embedded broker pushes messages into the listener instead of polling the destination.
我从 搬到DefaultMessageListenerContainer了SimpleMessageListenerContainer。简单容器使用推送消息。这样（并且通过适当的配置，嵌入式代理将消息推送到侦听器而不是轮询目的地。
回答by Tomasz Nurkiewicz
Do I understand correctly that you are manually creating 20-30 thousand of SimpleMessageListenerContainers, each listening on a different topic? I'm surprised that it even works since every listener by default creates one thread and 20-30 thousands of threads running concurrently in one JVM is impressive (and scary).
我是否正确理解您正在手动创建 20-30 千个SimpleMessageListenerContainers，每个听一个不同的主题？我很惊讶它甚至可以工作，因为默认情况下每个侦听器都会创建一个线程，而在一个 JVM 中同时运行的 20-30 万个线程令人印象深刻（而且很可怕）。
I see two ways to improve your performance:
我看到有两种方法可以提高你的表现：
read messages manually, e.g. using
JmsTemplate.receive()convenience method. Just remember to use very low timeout. By iteratively checking each and every topic in fewer threads (10? 100?) you will experience some latency, but less threads means less time wasted for context switching.Honestly I don't think this solution will scale and give you better performance, but try, examining different numbers of threads and timeouts.
Cluster! I am afraid that listening on 20-30k topics with one application will never work (fast enough). However seems like you can easily scale-out this task. Throw in ten or maybe hundred servers and let each listen for two thousand or two hundred topic accordingly. This will scale linearily.
手动读取消息，例如使用
JmsTemplate.receive()便捷方法。请记住使用非常低的超时。通过在更少的线程（10？100？）中迭代检查每个主题，您会遇到一些延迟，但更少的线程意味着更少的时间浪费在上下文切换上。老实说，我认为此解决方案不会扩展并为您提供更好的性能，但请尝试检查不同数量的线程和超时。
簇！我担心用一个应用程序听 20-30k 主题永远不会奏效（足够快）。但是，您似乎可以轻松扩展此任务。投入十个或一百个服务器，让每个服务器相应地收听两千或两百个主题。这将线性缩放。
