# Clustering with JOnAS

This guide describes how to configure Apache, Tomcat and JOnAS to install a cluster.

This configuration uses the Apache/Tomcat plug-in mod_jk. This plug-in allows to use Apache HTTP server in front of one or several Tomcat JSP/Servlet engines, and to forward some of the HTTP requests (generally the ones concerning the dynamic pages, i.e. JSP and Servlet requests) to Tomcat instances.

It also uses the In-Memory-Session-Replication technique based on the group communication protocol JavaGroups to provide failover at servlet/JSP level.

Here after is described one presented architecture, the steps to run an application in a cluster of JOnAS following the architecture previously presented, and other possible configurations.

- Architecture
- Step 1 : Products Installation

- Installing Apache
- Installing Tomcat
- Installing JOnAS

- Step 2 : Configuring each branch

- Server J1
- Server T1
- Running the application

- Step 3 : Load Balancing with mod_JK

- Configuring the JK Module
- Configuring JOnAS
- Running your Web Application

- Step 4 : Session Replication

- Running your application

- Other Possible Configurations

- Load Balancing
- High Availability
- In a coming version

- References

## Architecture

The architecture presented in this guide is : Apache as the front-end HTTP server, Tomcat as the JOnAS web container, the JOnAS EJB Container, and a shared database. The mod_jk plug-in will provide Load Balancing / High Availability at the Servlet / JSP level. In the described architecture, Tomcat runs in a different JVM than the EJB container, but it is also possible to join the two ones. On each EJB container node runs a JNDI repository dedicated to the JOnAS server. They are independent of each other (but generally contain the same information since the same beans and resources are deployed on each server). The database is shared by the JOnAS servers.

The versions considered here are : Apache 1.3, Tomcat 4.0.x for JOnAS 2.6.x, and Tomcat 4.1.x for JOnAS 3.0.

The architecture presented in this document is :

This architecture provides :

- Load balancing : Requests may be dispatched over a set of servers in order to distribute the load. This improves the "scalability" by allowing to process more requests concurrently.
- High Availability (HA) : having several servers able to fulfill a resquest makes possible to ensure that if a server dies, the request may be sent to an available server (so the load balancing algorithme may just ensure that the server to which the request will be sent is available). Thus "Service Availability" may be achieved.
- Failover at Servlet / JSP Level : This feature insures that if one JSP/servlet server goes down, then another server is able to transparently take over, i.e. the request will be switched to another server, without service disruption. This means that the user will not have to start over. This way ?Continuity? may be achieved.

However, failover at EJB level is not provided yet. This means that if an EJB server goes down, the corresponding branch (Server Tomcat + Server JOnAS) of the system is down, i.e. if the server J1 goes down, the upper branch is down and every request sent to T1 will fail. The mechanism to provide failover at EJB level is under development and will be available in a coming version of JOnAS.

## Products Installation

First of all, you need to install the different products on each server. On the server A, you have to install Apache 1.3. On the servers T1, T2, J1 and J2, you have to install JOnAS 2.6.x or 3.0. On the server T1 and T2, depending on the version of JOnAS chosen, you need to install Tomcat 4.0.x (for JOnAS 2.6.x) or Tomcat 4.1.x (for JOnAS 3.0)

### Installing Apache

1. Download Apache HTTP server source code at Apache site
2. Untar the source code
`tar fvxz apache_1.3.26.tar.gz`
3. Compile and Install
`cd apache_1.3.26`
`./configure --enable-module=include --enable-module=so --prefix=/usr/local/apache`
`make install`

You may also install a binary version. See on the Apache site.

### Installing Tomcat

1. Download Tomcat at Apache site
2. Install Tomcat by unzipping the downloaded file and put it in a specific directory. For Tomcat 4.0.x and 4.1.x, this directory is referred as CATALINA_HOME (an environment variable that you have to position).
If you want to share a single copy of Tomcat 4 among multiple users on the same server, you must configure a CATALINA_BASE environment variable (in addition to CATALINA_HOME) that points to a directory that is unique to your instance. Set both variables in your environment and export them.
3. On Windows, set the JAVA_HOME variable in your environment to your JDK installation directory.

### Installing JOnAS

See Installing JOnAS from scratch.

## Configuring each branch

The aim of the chapter is to configure Tomcat and JOnAS to run the following architecture :

### Server J1

This server contains the EJB container and the registry. The two main files to configure for this architecture are jndi.properties and jonas.properties.
Hereafter is an example of what your jndi.properties should look like (if you are using Jeremie) :

`java.naming.factory.initial org.objectweb.jeremie.libs.services.registry.jndi.JRMIInitialContextFactory`
`java.naming.provider.url jrmi://serverJ1:12340`
`java.naming.factory.url.pkgs org.objectweb.jonas.naming`

In the file jonas.properties, you have to provide the services started in this JOnAS server. In the case of the ServerJ1, the services need are :

`registry,jmx,security,jtm,dbm,ejb,ear`

### Server T1

This server contains the web container. It makes a remote call to the registry of the server J1. To do so, the jndi.properties file should be similar to the one of the server J1 i.e. :

`java.naming.factory.initial org.objectweb.jeremie.libs.services.registry.jndi.JRMIInitialContextFactory`
`java.naming.provider.url jrmi://serverJ1:12340`
`java.naming.factory.url.pkgs org.objectweb.jonas.naming`

In addition, the jonas.properties file should indicate that the registry is in a separate JVM :

`jonas.service.registry.mode remote`

The file jonas.properties should only contain the following services :

`registry,web,ear`

### Running the application

The two servers should have different names and you have to start the JOnAS server on serverJ1 (containing the registry) before the JOnAS server on serverT1.

On serverJ1 do : `jonas start -n <jonas-name-serverJ1>`

Then on serverT1 do : `jonas start -n <jonas-name-serverT1>`

## Step 3 : Load balancing with mod_jk

The aim of this chapter is to configure Apache, Tomcat and JOnAS to run the following architecture :

### Configuring the JK Module

#### JK module principles

Mod_jk is a plug-in that handles the communication between Apache and Tomcat.
Mod_jk uses the concept of worker . A worker is a Tomcat instance that is running to serve servlet requests coming from the web server. Each worker is identified to the web server by the host where it is located, the port where it listen and the communication protocol used to exchange messages. In our configuration there is one worker for each Tomcat instance and one worker that will handle the load balancing (this is a particular worker with no host and no port number). All those workers are defined in a file worker.properties.

Note : this module can also be used for site partitioning.

#### install Mod_jk

The easiest is to download the binary on the Tomcat Site or on this site. Put this file in the directory libexec (for unix) or modules (for windows or Mandrake) of the installation directory of Apache.

#### Configure Apache

- httpd.conf
Create a file tomcat_jk.conf that has to be included in $APACHE_HOME/conf/httpd.conf
This file should load the module mod_jk :

`LoadModule jk_module modules/mod_jk.so (for windows)`
`LoadModule jk_module libexec/mod_jk.so (for Unix)`
`AddModule mod_jk.c`

And configure mod_jk :

`# Location of the worker file`
`JkWorkersFile "/etc/httpd/conf/jk/workers.properties"`
`# Location of the log file`
`JkLogFile "/etc/httpd/jk/logs/mod_jk.log"`
`# Log level : debug, info, error or emerg`
`JkLogLevel emerg`
`# Assign specific URL to Tomcat workers`
`JkMount /admin loadbalancer`
`JkMount /admin/* loadbalancer`
`JkMount /examples loadbalancer`
`JkMount /examples/* loadbalancer`

- worker.properties
This file should first contain the list of workers :
`worker.list=<a comma separated list of worker names>`
then the properties of each worker :
`worker.<worker name>.<property>=<property value>`

Here is an example of what a worker.properties file should look like :

`# List the workers name`
`worker.list=worker1,worker2,loadbalancer`
`# ----------------`
`# First worker`
`# ----------------`
`worker.worker1.port=8009`
`worker.worker1.host=server1`
`worker.worker1.type=ajp13`
`# Load balance factor`
`worker.worker1.lbfactor=1`
`# ----------------`
`# Second worker`
`# ----------------`
`worker.worker2.port=8009`
`worker.worker2.host=server2`
`worker.worker2.type=ajp13`
`worker.worker2.lbfactor=1`
`# ----------------------`
`# Load Balancer worker`
`# ----------------------`
`worker.loadbalancer.type=lb`
`worker.loadbalancer.balanced_workers=worker1,worker2`

#### Configure Tomcat

The following configuration steps should be done for each Tomcat server.

1. Configure Tomcat for the connector AJP13
In the file conf/server.xml of your Tomcat installation directory, add (if not already there) :

`<!-- Define an AJP 1.3 Connector on port 8009 -->`
`<Connector className="org.apache.ajp.tomcat4.Ajp13Connector"`
`port="8009" minProcessors="5" maxProcessors="75"`
`acceptCount="10" debug="20"/>`

2. Define the jvmRoute
In the file conf/server.xml of your Tomcat installation directory, a unique route to the Catalina engine should be added :
Replace the line :
`<Engine name="Standalone" defaultHost="localhost" debug="0">`
with :
`<Engine jvmRoute="worker1" name="Standalone" defaultHost="localhost" debug="0">`

Note : the jvmRoute name should be the same as the name of the associated worker defined in worker.properties. This will ensure the Session affinity.

### Configuring JOnAS

In the JOnAS specific deployment descriptor you should add the tag `shared` for the involved entity beans and set it to `true`. When this flag is set to true, multiple instance of the same entity bean in different JOnAS servers can access a common database concurrently.

Here is an example of deployment descriptor with the flag `shared`:

`<jonas-ejb-jar>`
`<jonas-entity>`
`<ejb-name>Id_1</ejb-name>`
`<jndi-name>clusterId_1</jndi-name>`
`<shared>true</shared>`
`<jdbc-mapping>`
`<jndi-name>jdbc_1</jndi-name>`
`<jdbc-table-name>clusterIdentityEC</jdbc-table-name>`
`<cmp-field-jdbc-mapping>`
`<field-name>name</field-name>`
`<jdbc-field-name>c_name</jdbc-field-name>`
`</cmp-field-jdbc-mapping>`
`<cmp-field-jdbc-mapping>`
`<field-name>number</field-name>`
`<jdbc-field-name>c_number</jdbc-field-name>`
`</cmp-field-jdbc-mapping>`
`<finder-method-jdbc-mapping>`
`<jonas-method>`
`<method-name>findByNumber</method-name>`
`</jonas-method>`
`<jdbc-where-clause>where c_number = ?</jdbc-where-clause>`
`</finder-method-jdbc-mapping>`
`<finder-method-jdbc-mapping>`
`<jonas-method>`
`<method-name>findAll</method-name>`
`</jonas-method>`
`<jdbc-where-clause></jdbc-where-clause>`
`</finder-method-jdbc-mapping>`
`</jdbc-mapping>`
`</jonas-entity>`
`</jonas-ejb-jar>`

### Running your Web Application

You are now ready to run your web application:

1. start the jonas servers on ServerJ1 and ServerJ2 : `jonas start` in the directory of your application
2. start the jonas servers on ServerT1 and ServerT2 : `jonas start` in the directory of your application
3. restart Apache : `httpd restart`
4. Use your browser to access the welcome page, usually index.html

## Session Replication

This is the last step to configure a cluster of JOnAS with load balancing, High Availability and failover at JSP/servlet level. This step adds the session replication to the previous architecture:

Session replication is the term that is used when your current service state is being replicated across multiple application instances. Session replication occurs when we replicate the information that is stored in your HttpSession from, in our case, one servlet engine instance to another. This could be data like the items that you have in your shopping cart. It could also be the information that your are entering on an insurance application. Whatever is being stored in your session has to be replicated in order for the service to fail over without a disruption.

The solution chosen to achieve Session replication is called in-memory-session-replication. It uses a group communication protocol written entirely in Java called JavaGroups. JavaGroups is a communication protocol based on the idea of virtual synchrony and probabilistic broadcasting.

Hereafter are the steps to achieve Session replication with JOnAS.

- Download :

- Javagroups Library
- Tomcat Session Replication Library

- On the servers T1 and T2, copy the cluster binaries (javagroups.jar and tomcat-javagroups.jar) into `<TOMCAT-HOME>/server/lib`
- On the servers T1 and T2, run ant install in `<JONAS_ROOT>`
- On the servers T1 and T2, open the `<TOMCAT_HOME>/conf/server.xml` file and configure your `<context>` as described :
`<Context path="/example" docBase="example" debug="99"`
`reloadable="true" crossContext="true"`
`className="org.objectweb.jonas.web.catalina.JOnASStandardContext">`
`<Manager className="org.apache.catalina.session.InMemoryReplicationManager"`
`protocolStack="UDP(mcast_addr=237.0.0.1;mcast_port=45566;ip_ttl=32):PING(timeout=3000;num_initial_members=6):FD(timeout=5000):VERIFY_SUSPECT(timeout=1500):pbcast.STABLE(desired_avg_gossip=10000):pbcast.NAKACK(gc_lag=10;retransmit_timeout=3000):UNICAST(timeout=5000;min_wait_time=2000):MERGE2:FRAG:pbcast.GMS(join_timeout=5000;join_retry_timeout=2000;shun=false;print_local_addr=false)">`
`</Manager>`
`</context>`

Note 1 : Don't put any end-of-line or space character in the protocolStack definition.

Note 2 : The protocolStack has to be identically configured for all JOnAS/Tomcat instances.

### Running your Web Application

You are now ready to run your web application in the cluster :

1. start the jonas servers on ServerJ1 and ServerJ2 : `jonas start` in the directory of your application
2. start the jonas servers on ServerT1 and ServerT2 : `jonas start` in the directory of your application
3. restart Apache : `httpd restart`
4. Use your browser to access the welcome page, usually index.html

## Other Possible configurations

Using the different configuration steps presented higher, several architectures are possible. To present some of them, the following symbols will be used :

| A node (computer) that hosts one or more server
| A web container |  | An ejb container
| A JOnAS instance that hosts a web container |  | A JOnAS instance that hosts an ejb container
| A JOnAS instance that hosts a web container and an ejb container
| An Apache server with the mod_jk module

### Load Balancing

With just a Load Balancing mechanism, it is possible to just put Apache as a front end of two (or more) JOnAS Application servers :

It is also possible to run the web container and the ejb container in different JVMs :

### High Availability

Then it is possible to add the session replication mechanism to the previous architectures to obtain high available systems :

- With the web container and the ejb container within the same JVM :

- With the web container and the ejb container in different JVMs :

### In a comming version

We are currently developing a solution to enable load balancing and high availability at EJB level. This wil enable the following architecture :

## References

- Working with mod_jk
- Tomcat workers Howto
- Apache JServ Protocol version 1.3 (ajp13)
- Apache - Tomcat HOWTO
- Apache 1.3.23 + Tomcat 4.0.2 + Load Balancing
- In Memory Session Replication in Tomcat 4