Skip to content

Instantly share code, notes, and snippets.

@jmesnil
Created October 4, 2012 14:33
Show Gist options
  • Save jmesnil/3833904 to your computer and use it in GitHub Desktop.
Save jmesnil/3833904 to your computer and use it in GitHub Desktop.
How to run HornetQ 2.3 clusters with AS7 & JGroups

build AS7

run AS7 domain with full-ha hosts

$ cd build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT

modify domain/configuration/host.xml so that the other-server-group is using the full-ha profile

64c64
<         <server name="server-one" group="main-server-group">
---
>         <server name="server-one" group="other-server-group">
73c73
<         <server name="server-two" group="main-server-group" auto-start="true">
---
>         <server name="server-two" group="other-server-group" auto-start="true">

modify domain/configuration/domain.xml to replace use of socket-binding by JGroups for {broadcast|cluster}-group

1171,1172c1171,1173
<                             <socket-binding>messaging-group</socket-binding>
---
>                             <jgroups-stack>udp</jgroups-stack>
>                             <jgroups-channel>hq-cluster</jgroups-channel>
1178c1179,1180
<                             <socket-binding>messaging-group</socket-binding>
---
>                             <jgroups-stack>udp</jgroups-stack>
>                             <jgroups-channel>hq-cluster</jgroups-channel>

start the domain (with HornetQ cluster password)

$ ./bin/domain.sh -Djboss.messaging.cluster.password=mypwd
...
[Host Controller] 16:28:27,321 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server-one
[Host Controller] 16:28:27,327 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server-two
...
[Server:server-two] 16:28:33,072 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111001: live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-two/data/messagingjournal,bindingsDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-two/data/messagingbindings,largeMessagesDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-two/data/messaginglargemessages,pagingDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-two/data/messagingpaging)
[Server:server-two] 16:28:33,074 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111008: Waiting to obtain live lock
[Server:server-two] 16:28:33,093 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111016: Using AIO Journal
[Server:server-one] 16:28:33,095 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111001: live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-one/data/messagingjournal,bindingsDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-one/data/messagingbindings,largeMessagesDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-one/data/messaginglargemessages,pagingDirectory=/home/jmesnil/Developer/jboss-as/build/target/jboss-as-7.2.0.Alpha1-SNAPSHOT/domain/servers/server-one/data/messagingpaging)
[Server:server-one] 16:28:33,097 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111008: Waiting to obtain live lock
[Server:server-one] 16:28:33,120 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111016: Using AIO Journal
[Server:server-two] 16:28:33,186 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111039: Waiting to obtain live lock
[Server:server-two] 16:28:33,187 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111040: Live Server Obtained live lock
[Server:server-one] 16:28:33,236 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111039: Waiting to obtain live lock
[Server:server-one] 16:28:33,236 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111040: Live Server Obtained live lock
[Server:server-one] 16:28:33,847 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-1) send buffer of socket java.net.DatagramSocket@74c11174 was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
[Server:server-two] 16:28:33,848 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-5) send buffer of socket java.net.DatagramSocket@78135095 was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
[Server:server-one] 16:28:33,848 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-1) receive buffer of socket java.net.DatagramSocket@74c11174 was set to 20MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
[Server:server-one] 16:28:33,850 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-1) send buffer of socket java.net.MulticastSocket@2f61128e was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
[Server:server-two] 16:28:33,850 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-5) receive buffer of socket java.net.DatagramSocket@78135095 was set to 20MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
[Server:server-two] 16:28:33,851 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-5) send buffer of socket java.net.MulticastSocket@3c0fa3c9 was set to 640KB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max send buffer in the OS correctly (e.g. net.core.wmem_max on Linux)
[Server:server-two] 16:28:33,851 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-5) receive buffer of socket java.net.MulticastSocket@3c0fa3c9 was set to 25MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
[Server:server-one] 16:28:33,852 WARN  [org.jgroups.protocols.UDP] (MSC service thread 1-1) receive buffer of socket java.net.MulticastSocket@2f61128e was set to 25MB, but the OS only allocated 131.07KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
[Server:server-one] 16:28:33,854 INFO  [stdout] (MSC service thread 1-1)
[Server:server-one] 16:28:33,855 INFO  [stdout] (MSC service thread 1-1) -------------------------------------------------------------------
[Server:server-one] 16:28:33,855 INFO  [stdout] (MSC service thread 1-1) GMS: address=master:server-one/hq-cluster, cluster=hq-cluster, physical address=127.0.0.1:55200
[Server:server-one] 16:28:33,856 INFO  [stdout] (MSC service thread 1-1) -------------------------------------------------------------------
[Server:server-two] 16:28:33,854 INFO  [stdout] (MSC service thread 1-5)
[Server:server-two] 16:28:33,857 INFO  [stdout] (MSC service thread 1-5) -------------------------------------------------------------------
[Server:server-two] 16:28:33,858 INFO  [stdout] (MSC service thread 1-5) GMS: address=master:server-two/hq-cluster, cluster=hq-cluster, physical address=127.0.0.1:55350
[Server:server-two] 16:28:33,858 INFO  [stdout] (MSC service thread 1-5) -------------------------------------------------------------------
[Server:server-one] 16:28:35,930 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111024: Started Netty Acceptor version 3.4.5.Final-2da5b0e 127.0.0.1:5,455 for CORE protocol
[Server:server-one] 16:28:35,932 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111024: Started Netty Acceptor version 3.4.5.Final-2da5b0e 127.0.0.1:5,445 for CORE protocol
[Server:server-one] 16:28:35,936 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111009: Server is now live
[Server:server-one] 16:28:35,936 INFO  [org.hornetq.core.server] (MSC service thread 1-1) HQ111003: HornetQ Server version 2.3.0.BETA1 (HornetQ sting, 122) [c67452b4-0e2f-11e2-825d-8996bc49e6b8]
...
[Server:server-one] 16:28:36,032 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.2.0.Alpha1-SNAPSHOT "Steropes" started in 8570ms - Started 152 of 301 services (148 services are passive or on-demand)
[Server:server-two] 16:28:36,500 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111024: Started Netty Acceptor version 3.4.5.Final-2da5b0e 127.0.0.1:5,595 for CORE protocol
[Server:server-two] 16:28:36,502 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111024: Started Netty Acceptor version 3.4.5.Final-2da5b0e 127.0.0.1:5,605 for CORE protocol
[Server:server-two] 16:28:36,503 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111009: Server is now live
[Server:server-two] 16:28:36,503 INFO  [org.hornetq.core.server] (MSC service thread 1-5) HQ111003: HornetQ Server version 2.3.0.BETA1 (HornetQ sting, 122) [c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55]
...
[Server:server-two] 16:28:36,607 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.2.0.Alpha1-SNAPSHOT "Steropes" started in 9146ms - Started 152 of 301 services (148 services are passive or on-demand)
[Server:server-one] 16:28:36,646 INFO  [org.hornetq.core.server] (Thread-4 (HornetQ-server-HornetQServerImpl::serverUUID=c67452b4-0e2f-11e2-825d-8996bc49e6b8-132636541)) HQ111032: Bridge ClusterConnectionBridge@d117f23 [name=sf.my-cluster.c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55, queue=QueueImpl[name=sf.my-cluster.c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=c67452b4-0e2f-11e2-825d-8996bc49e6b8]]@77b1d988 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@d117f23 [name=sf.my-cluster.c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55, queue=QueueImpl[name=sf.my-cluster.c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55, postOffice=PostOfficeImpl           [server=HornetQServerImpl::serverUUID=c67452b4-0e2f-11e2-825d-8996bc49e6b8]]@77b1d988 targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5595&host=kunlun-local], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@2068171695[nodeUUID=c67452b4-0e2f-11e2-825d-8996bc49e6b8, connector=TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=kunlun-local, address=jms, server=HornetQServerImpl::serverUUID=c67452b4-0e2f-11e2-825d-8996bc49e6b8])) [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5595&host=kunlun-local], discoveryGroupConfiguration=null]] is connected
[Server:server-two] 16:28:36,687 INFO  [org.hornetq.core.server] (Thread-8 (HornetQ-server-HornetQServerImpl::serverUUID=c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55-1698703569)) HQ111032: Bridge ClusterConnectionBridge@1013a7b1 [name=sf.my-cluster.c67452b4-0e2f-11e2-825d-8996bc49e6b8, queue=QueueImpl[name=sf.my-cluster.c67452b4-0e2f-11e2-825d-8996bc49e6b8, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55]]@7a665db5 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@1013a7b1 [name=sf.my-cluster.c67452b4-0e2f-11e2-825d-8996bc49e6b8, queue=QueueImpl[name=sf.my-cluster.c67452b4-0e2f-11e2-825d-8996bc49e6b8, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55]]@7a665db5 targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=kunlun-local], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@210999737[nodeUUID=c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55, connector=TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5595&host=kunlun-local, address=jms, server=HornetQServerImpl::serverUUID=c66d4da3-0e2f-11e2-bcd0-d7ab96d52f55])) [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=kunlun-local], discoveryGroupConfiguration=null]] is connected
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment