Tomcat Cluster Farm Deployer issue
How do I update for 3 web servers?
Do the same as when doing with 2 nodes.
On the deployment node, add one more member in StaticMembershipInterceptor:
<Member className="org.apache.catalina.tribes.membership.StaticMember"
port="4002"
securePort="-1"
host="192.168.5.208"
domain="staging-cluster"
uniqueId="{0,1,2,3,4,5,6,7,8,9}"/>
On the node 3, create a cluster like belows:
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster">
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="192.168.5.208"
port="4002"
selectorTimeout="100"
maxThreads="6"/>
</Channel>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/opt/tomcat6/tempdir/"
deployDir="/opt/tomcat6/webapps/"
watchDir="/opt/tomcat6/watchdir/"
watchEnabled="false"/>
</Cluster>
And copy a *.war file to watchdir directory on the deployment node, the below lines will appears in catalina.out:
INFO: check cluster wars at /usr/share/tomcat6/watchdir
Aug 17, 2011 2:07:53 PM org.apache.catalina.tribes.group.interceptors.TcpFailureDetector performBasicCheck
WARNING: Member added, even though we werent notified:org.apache.catalina.tribes.membership.MemberImpl[tcp://192.168.5.208:40
02,192.168.5.208,4002, alive=0,id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }, payload={}, command={}, domain={115 116 97 103 105 110
103 45 99 ...(15)}, ]
Aug 17, 2011 2:07:53 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member added:org.apache.catalina.tribes.membership.MemberImpl[tcp://192.168.5.208:4002,192.168.5.208,4002,
alive=0,id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }, payload={}, command={}, domain={115 116 97 103 105 110 103 45 99 ...(15)}, ]
Aug 17, 2011 2:08:12 PM org.apache.catalina.ha.deploy.WarWatcher check
INFO: check cluster wars at /usr/share/tomcat6/watchdir
Aug 17, 2011 2:08:12 PM org.apache.catalina.ha.deploy.FarmWarDeployer fileModified
INFO: Installing webapp[/cas] from /usr/share/tomcat6/webapps/cas.war
Aug 17, 2011 2:08:12 PM org.apache.catalina.ha.deploy.FarmWarDeployer remove
INFO: Cluster wide remove of web app /cas
Aug 17, 2011 2:08:13 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive cas.war
Check the webapps folder on the node 3:
drwxr-xr-x 8 root root 4096 Aug 17 14:08 cas
-rw-r--r-- 1 root root 19962865 Aug 17 14:08 cas.war
...
and the catalina.out:
Aug 17, 2011 2:08:13 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive cas.war
EDIT
Invalid or unreadable WAR file : invalid CEN header (bad signature)
It looks like the WAR file is corrupted when deploying to 2 other nodes. How often does it happen?
Isn't it possible making FarmDeployer work with multicast from tomcat cluster docs which works perfectly for me apart from FarmDeployer?
I figured it out! The reason is you haven't added a multicast route for your network interface, so the cluster message cannot be send/receive between Tomcat nodes:
lsof -i :4000
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
java 9968 tomcat 37u IPv6 121232775 TCP localhost:terabase (LISTEN)
Do it by following steps:
-
Make sure that your network interface is enabled for multicast:
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 -
Add a multicast route:
route add -host 228.0.0.4 dev bond1
(you should change bond1 to the network interface that receives the cluster message)
Testing with ping -c 4 228.0.0.4 didn't work for me. So, I downloaded multicast test tool, run it and I see the belows response:
./bin/multicast-test.sh -local 192.168.5.149 -group 228.0.0.4:45564
PATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/ossec/bin
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
Oracle Coherence Version 3.3.1/389
Grid Edition: Development mode
Copyright (c) 2000-2007 Oracle. All rights reserved.
Starting test on ip=/192.168.5.149, group=/228.0.0.4:45564, ttl=4
Configuring multicast socket...
Starting listener...
Fri Aug 19 10:19:14 ICT 2011: Sent packet 1.
Fri Aug 19 10:19:14 ICT 2011: Received test packet 1 from self (sent 9ms ago).
Fri Aug 19 10:19:14 ICT 2011: Received 69 bytes from an unknown multicast application at 192.168.5.199: ???
Fri Aug 19 10:19:14 ICT 2011: Received 69 bytes from an unknown multicast application at 192.168.5.149: ???
Or you can check with lsof, netstat, ...
lsof -i :4000
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
java 11084 tomcat 37u IPv6 121236631 TCP localhost:terabase (LISTEN)
java 11084 tomcat 59u IPv6 121236830 TCP localhost:terabase->192.168.5.199:43519 (ESTABLISHED)
java 11084 tomcat 66u IPv6 121236835 TCP localhost:55538->192.168.5.199:terabase (ESTABLISHED)
java 11084 tomcat 67u IPv6 121236836 TCP localhost:55539->192.168.5.199:terabase (ESTABLISHED)
You are ready for configure Tomcat clustering with multicast.
Enable clustering configuration same as mentioned in the doc, but change <Receiver address from "auto" to specified interface:
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="192.168.5.149"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
Do the same for all the nodes in the cluster to see it works.
EDIT 2
IMPROVEMENT: I happily Used the Multicast configuration changing auto to ip in Receiver and I saw myapp.war on another web server when I placed the war file in watch directory on deployer web server. The Problem is, war is only extracting on the deployer web server(creating a direcotry) but not on other servers.
It means that the connection between deployment node and the other nodes is established. Did you still get the same error "Invalid or unreadable WAR file : invalid CEN header (bad signature)"? Everytime?
java.util.zip.ZipException: invalid END header (bad central directory offset)
According to the bug 45851, you should upgrade your Tomcat version (at least 6.0.19) and try again. I'm using version 6.0.29 and as you can see, it works fine.
It's needing tomcat restart on other web servers then it copies and extracts and the site is available with other-web-severs-ip/myapp but not with domain name(404 error). Where do you point the site domain name, I pointed the site domain name in the DNS to LB and created a virtual host in all the tomcat web servers with same domain name
No. It seems that you haven't cleared thoroughly. No need to create virtual host in all Tomcat nodes.
Assuming that you have Apache HTTP server and Tomcat on node 1, with IP 192.168.1.101. Two other Tomcat nodes at 1.102, 1.103. jvmRoute corresponding are: tomcat1, tomcat2, tomcat3.
The mod_jk.conf on Apache HTTP server:
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/conf/workers.properties
JkShmFile /var/log/httpd/jk.shm
JkLogFile /var/log/httpd/mod_jk.log
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkMount /myapp/* cluster
The workers.properties is something like this:
worker.list = cluster
worker.tomcat1.port=8009
worker.tomcat1.host=localhost
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=1
worker.tomcat2.port=8009
worker.tomcat2.host=192.168.1.102
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=1
worker.tomcat3.port=8009
worker.tomcat3.host=192.168.1.103
worker.tomcat3.type=ajp13
worker.tomcat3.lbfactor=1
worker.cluster.type=lb
worker.cluster.balance_workers=tomcat1,tomcat2,tomcat3
worker.cluster.sticky_session=1
Try again.