mod_jk produces Error smh errno=13
I am trying to use mod_jk
for load balancing.
Loadbalancer: 172.17.8.189 (HTTPD+mod_jk)
Node1: 172.17.8.193 (apache tomcat instance 1)
Node2: 172.17.8.196 (apache tomcat instance 2)
I checked my tomcat instances and both of them are listening at 8009. However, I keep getting these errors.
[Tue Jun 18 17:49:14 2013][18057:47118810657648] [error] init_jk::mod_jk.c (3348): Initializing shm:/etc/httpd/logs/jk.shm.18057 errno=13. Load balancing workers will not function properly.
[Tue Jun 18 17:49:14 2013][18057:47118810657648] [info] init_jk::mod_jk.c (3365): mod_jk/1.2.37 initialized
Debug log is
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_set_time_fmt::jk_util.c (461): Pre-processed log time stamp format is '[%a %b %d %H:%M:%S %Y]'
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_open::jk_uri_worker_map.c (865): rule map size is 2
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_add::jk_uri_worker_map.c (815): wildchar rule '/jasperserver-pro/*=loadbalancer' source 'JkMount' was added
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_add::jk_uri_worker_map.c (824): exact rule '/jkstatus/=status' source 'JkMount' was added
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_dump::jk_uri_worker_map.c (176): uri map dump after map open: id=0, index=0 file='(null)' reject_unsafe=0 reload=60 modified=0 checked=0
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_dump::jk_uri_worker_map.c (182): generation 0: size=0 nosize=0 capacity=0
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_dump::jk_uri_worker_map.c (182): generation 1: size=2 nosize=0 capacity=4
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_dump::jk_uri_worker_map.c (195): NEXT (1) map #0: uri=/jasperserver-pro/* worker=loadbalancer context=/jasperserver-pro/* source=JkMount type=Wildchar len=19
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] uri_worker_map_dump::jk_uri_worker_map.c (195): NEXT (1) map #1: uri=/jkstatus/ worker=status context=/jkstatus/ source=JkMount type=Exact len=10
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] init_jk::mod_jk.c (3303): Setting default connection pool max size to 1
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.list' with value 'loadbalancer,status' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node1.port' with value '8009' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node1.host' with value '172.17.8.193' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node1.type' with value 'ajp13' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node1.lbfactor' with value '1' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node2.port' with value '8009' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node2.host' with value '172.17.8.196' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node2.type' with value 'ajp13' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.node2.lbfactor' with value '1' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.loadbalancer.type' with value 'lb' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.loadbalancer.balance_workers' with value 'node1,node2' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.loadbalancer.sticky_session' with value '1' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_read_property::jk_map.c (482): Adding property 'worker.status.type' with value 'status' to map.
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_map_resolve_references::jk_map.c (764): Checking for references with prefix worker. with wildcard (recursion 1)
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_shm_calculate_size::jk_shm.c (129): worker loadbalancer of type lb has 2 members
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [debug] jk_shm_calculate_size::jk_shm.c (136): shared memory will contain 0 ajp workers and 1 lb workers with 2 members
[Tue Jun 18 18:14:00 2013][18778:47687756850032] [error] init_jk::mod_jk.c (3348): Initializing shm:/etc/httpd/logs/jk.shm.18778 errno=13. Load balancing workers will not function properly.
As the result the request arent being properly rerouted.
My worker.properties
# Define list of workers that will be used
# for mapping requests
worker.list=loadbalancer,status
# Define Node1
# modify the host as your host IP or DNS name.
worker.node1.port=8009
worker.node1.host=172.17.8.193
worker.node1.type=ajp13
worker.node1.lbfactor=1
# Define Node2
# modify the host as your host IP or DNS name.
worker.node2.port=8009
worker.node2.host=172.17.8.196
worker.node2.type=ajp13
worker.node2.lbfactor=1
# Load-balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=1
#worker.list=loadbalancer
# Status worker for managing load balancer
worker.status.type=status
My mode-jk.conf
# Load mod_jk module
# Specify the filename of the mod_jk lib
LoadModule jk_module modules/mod_jk.so
# Where to find workers.properties
JkWorkersFile conf/workers.properties
# Where to put jk logs
JkLogFile logs/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel info
# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
# JkOptions indicates to send SSK KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat
JkRequestLogFormat "%w %V %T"
# Mount your applications
JkMount /myapp/* loadbalancer
# You can use external file for mount points.
# It will be checked for updates each 60 seconds.
# The format of the file is: /url=worker
# /examples/*=loadbalancer
#JkMountFile conf/uriworkermap.properties
# Add shared memory.
# This directive is present with 1.2.10 and
# later versions of mod_jk, and is needed for
# for load balancing to work properly
JkShmFile logs/jk.shm
#JkShmFile /var/log/httpd/mod_jk.shm
# Add jkstatus for managing runtime data
<Location /jkstatus/>
JkMount status
Order deny,allow
Deny from all
Allow from localhost
</Location>
Server.xml
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- Prevent memory leaks due to use of particular java/javax APIs-->
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<!-- JMX Support for the Tomcat server. Documentation at /docs/non-existent.html -->
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more "Connectors" that share
a single "Container" Note: A "Service" is not itself a "Container",
so you may not define subcomponents such as "Valves" at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can define one or more named thread pools-->
<!--
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="8081" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL HTTP/1.1 Connector on port 8443
This connector uses the JSSE configuration, when using APR, the
connector should be using the OpenSSL style configuration
described in the APR documentation -->
<!--
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!-- An Engine represents the entry point (within Catalina) that processes
every request. The Engine implementation for Tomcat stand alone
analyzes the HTTP headers included with the request, and passes them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing via AJP ie :
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost" jvmRoute="node2">
<!--For clustering, please take a look at documentation at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation) -->
<!--
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564" frequency="500"
dropTime="3000"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto" port="4000" autoBind="100"
selectorTimeout="5000" maxThreads="6"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
<!-- The request dumper valve dumps useful debugging information about
the request and response data received and sent by Tomcat.
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.valves.RequestDumperValve"/>
-->
<!-- This Realm uses the UserDatabase configured in the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are immediately
available for use by the Realm. -->
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<!-- Define the default virtual host
Note: XML Schema validation will not work with Xerces 2.2.
-->
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/>
-->
</Host>
</Engine>
</Service>
</Server>
Would appreciate any feedback.
Are you using selinux? If you are, you might want to check on it being an selinux context problem.
The correct answer is online:
http://www.laliluna.de/articles/2008/09/14/apache_tomcat_mod_jk_selinux.html
The part you need is near the bottom, which I will copy and paste here in case that page goes away:
# mkdir /var/run/mod_jk
# semanage fcontext -a -t httpd_var_run_t "/var/run/mod_jk(/.*)?"
This apparently configures SELinux to allow mod_jk to work. It worked for me anyway.
Your OS might prevent access to the shared memory file. Try adding
# Set this to a location allowed to the 'Apache' user
JkShmFile /var/log/httpd/mod_jk.shm
in your mode-jk.conf
(I am using the filename spelling from your post but it might be mod_jk.conf or similar)