Configuring HTTP and HTTPS clustering on JBoss Server

Some colleagues of mine were facing problems getting the HTTP/HTTPS clustering setup done for JBoss server. Although I have no experience on working with JBoss I decided to give it a try.

Note that my development environment is Windows. The first thing I did was get hold of JBoss 4.2.2GA installable. Why this one, because this is the one that I have!! I copied the installable twice on my machine’s D: drive, creating two JBoss homes namely D:\JBoss 4.2.2GA-1 and D:\JBoss 4.2.2GA-2. To get the clustering setup done I referred to JBoss documentation. The documentation is decent and assists in getting your setup right. Instructions regarding setting up HTTP related services can be found  under section 1.5 titled “HTTP Services”.

First things first. Let us setup the load balancer. The load balancer is not part of the JBoss installable. JBoss uses the popular Apache Web server to assist it in achieving load balancing. The Apache web server’s jk module is used to forward all requests to the JBoss servlet container. Apache web server downloadable is available here. I have used Apache 2.0.52 and 2.0.55 for our demonstration. Per JBoss, any version in the range 2.0.x is acceptable. Next get hold of the JK module binaries from the site. I have used mod_jk-1.2.28-httpd-2.0.52.so. Please select the jk module version compatible to your Apache server. Detailed instructions of version compatibility are available on the download page. for e.g. for jk 1.2.28. Copy the JK module so file in the APACHE_HOME/modules folder.

Modify the APACHE_HOME/conf/httpd.conf and add the following lines at the end of the file.

# Include mod_jk's specific configuration file
Include conf/mod-jk.conf

Create a new mod-jk.conf file and copy the file in the APACHE_HOME/conf folder. The contents of the file are as below:

# Load mod_jk module
LoadModule jk_module modules/mod_jk-1.2.28-httpd-2.0.52.so

# Where to find worker.properties
JkWorkersFile conf/workers.properties

# Where to put jk logs
JkLogFile logs/mod_jk.log

# Set the jk log level [debug/error/info]
JkLogLevel info

# Select the log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

#JkOptions indicate to send SSK KEY SIZE
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat
JkRequestLogFormat "%w %V %T"

# Mount your applications
JkMount /* loadbalancer

# Add shared memory
JkShmFile logs/jk.shm

Ensure that the file name is as per the installable copied in the modules folder. As per the instructions in the clustering guide, the mod-jk.conf file should have a Location tag in it. I have removed the same as it is not supported by my older Apache server. You can add the same if required. The JkMount directive in the file configures the URL that needs to be redirected. Currently it is configured to reroute all URLs; feel free to customize if required.

Create a new worker.properties file. Contents are as below:

# Define list of workers
worker.list=loadbalancer,status

# Define Node1
worker.node1.port=8009
worker.node1.host=localhost
worker.node1.type=ajp13
worker.node1.lbfactor=1
worker.node1.cachesize=10

# Define Node2
worker.node2.port=8109
worker.node2.host=localhost
worker.node2.type=ajp13
worker.node2.lbfactor=1
worker.node2.cachesize=10

# Load balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=1
#worker.list=loadbalancer

# Status worker for managing load balancer
worker.status.type=status

Note that the ports 8009 and 8109 are the JBoss AJP connector ports and not the HTTP ports. Copy the attached workers.properties in APACHE_HOME/conf folder. I have defined two nodes and am assuming both are located on the same machine.

The server.xml within JBOSS_HOME/server/default/deploy/jboss-web.deployer should have the following tag:

    <Connector port="8009" address="${jboss.bind.address}" protocol="AJP/1.3"
         emptySessionPath="true" enableLookups="false" redirectPort="8443" >

This defines the AJP port.

This should be enough to get the JBOSS server running in clustered mode for HTTP.

I also needed to get the HTTPS setup rolling. Apparently Apache does not provide built-in SSL support. To achieve SSL support you will need to download the OpenSSL project and install it. An alternative is to get an integrated apache-openssl download from this site (The site was not available , hence I used downloads from the following site and got the openssl download from here.

An update: The Apache site provides a Windows binary with OpenSSL built-in. So you can use that one as well.

I am assuming that you have created the certificate using the tomcat(jboss) server. For my testing purposes I have created a self signed certificate using the Java utility keytool. The syntax for certificate creation is as below:

keytool -genkey -alias <aliasName> -keystore <keystore name>

More clarity is available at this site.

Go to the server.xml file located within the <JBoss_home>\server\default\deploy\jboss-web.deployer folder. Search for a connector tag with the following description:

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS"
               keystoreFile="D:\jboss-4.2.2GA-1\server\default\deploy\jboss-web.deployer\testing.keystore"
               keystorePass="testing"/>

Add the new attributes keystoreFile and keystorePass in the connector tag. Do the same procedure for the other server. Change the port of this server. Add a new AJP connector both the server.xml files.

<Connector port="8019" address="${jboss.bind.address}" protocol="AJP/1.3"
         emptySessionPath="true" enableLookups="false" scheme="https" secure="true" redirectPort="8443" />
</sourcecode>
Note that two new attributes scheme and secure have been added in the AJP connector declaration. Ensure that the port number in use do not conflict with other port numbers.

The Jboss servers are now ready to receive SSL requests. Now to set up the apache server for taking care of load balancing. If you have installed Apache using the URL provided above, the conf folder will have two files httpd.conf and ssl.conf. Open the conf files are check the ServerRoot and DocumentRoot paths.

Make sure that following lines in the httpd.conf file are uncommented.

LoadModule ssl_module modules/mod_ssl.so

<IfModule mod_ssl.c>
    Include conf/ssl.conf
</IfModule>

Add the following lines at the end of the httpd.conf file

<VirtualHost *:80>
 JkMount /* loadbalancer
</VirtualHost>

# Include mod_jk's specific configuration file
Include conf/mod-jk.conf

mod-jk.conf remains unchanged.

Unzip the OpenSSL.zip on the machine. Copy the libeay32.dll and ssleay32.dll in Windows\system32 folder of the machine.

The Tomcat keystore and Apache SSL certificates and keys are incompatible. They need to be converted into compatible certificate and key. For details around the conversion process refer the url.

Now you should be having the pem certificate and private key. Copy the files in a suitable folder and make relevant entries for them in the ssl.conf:

SSLCertificateFile /root/SSL_export/exported-pem.crt
SSLCertificateKeyFile /root/SSL_export/exported.key

The intermediate certificate is not required in case of self signed certificates.

Add the following statement within the VirtualHost tag of the ssl.conf file

JkMount /* loadbalancerSSL

In ssl.conf file remove <IfDefine SSL> and </IfDefine> tags ensure that the ServerName, DocumentRoot are pointing to the correct folders. The workers.properties file is configured to handle 4 nodes, two for HTTP requests and two for HTTPS requests.

Here is the updated workers.properties file.

# Define list of workers
worker.list=loadbalancer,loadbalancerSSL,status
#worker.list=loadbalancer,status

# Define Node1
worker.node1.port=8009
worker.node1.host=localhost
worker.node1.type=ajp13
worker.node1.lbfactor=1
worker.node1.cachesize=10

# Define Node2
worker.node2.port=8109
worker.node2.host=localhost
worker.node2.type=ajp13
worker.node2.lbfactor=1
worker.node2.cachesize=10

# Define Node3
worker.node3.port=8019
worker.node3.host=localhost
worker.node3.type=ajp13
worker.node3.lbfactor=1
worker.node3.cachesize=10

# Define Node2
worker.node4.port=8119
worker.node4.host=localhost
worker.node4.type=ajp13
worker.node4.lbfactor=1
worker.node4.cachesize=10

# Load balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=1
#worker.list=loadbalancer

# Load balancing behaviour
worker.loadbalancerSSL.type=lb
worker.loadbalancerSSL.balance_workers=node3,node4
worker.loadbalancerSSL.sticky_session=1
#worker.list=loadbalancer

# Status worker for managing load balancer
worker.status.type=status

The above configuration should be enough to get the JBoss running in a clustered environment for HTTP as well as HTTPS requests.

This post does not cover the portion for sticky session configuration.

Advertisements

10 thoughts on “Configuring HTTP and HTTPS clustering on JBoss Server

  1. Thank you.
    Just one question. If the SSL certificate is signed by certificate authority, we can use one domain name for all Jboss nodes and Apache server?

    1. Interesting question. The answer lies in selecting the appropriate licensed certificate from CA. They have different licensing policies based on your configuration. For Verisign you can refer this URL. The link should provide you with the relevant answers.

      In case the question is how to set up the certificate i.e. on the web server or the application server, here’s a reference example.

  2. Thanks for the first link, it really helped. As I understand the question is just in the licence, technically everithing will work event if we generate by keytool only one SSL certificate for the selected domain, sign it by authority and after that copy it to other nodes and make a converted copy with intermediate certificate for Apache. Am I right? As to the licences it’s not quite clear how many of them should we have in configuration of 2 nodes and load-balancer (Apache). In this case Apache SSL certificate should be licenced too?

    1. Yes, getting multiple nodes protected depends on your certificate license. I will need to confirm this but one option is to set up Apache server with the certificate and use JK –> AJP module for pass thru communication to the JBoss nodes. Refer the second link. They have installed it in the same manner.

      I checked up my setup. The SSL certificates are installed in my JBoss server not at the Apache end. I believe that you can also set up certificates at Apache server and JBoss. So it is essentially your choice. From a security perspective, I think having them at the JBoss Application server end should be enough.

  3. Hi, this appears to be load balancing jboss instances with apache over ajp. Have I missed something ? Where is the jboss clustering, and how does session state etc. get replicated between the two instances ? Thanks.

    1. Yes, this is load balancing JBoss instances. We did not need to handle session replication for that particular requirement. May be this will help.

      1. Yes, This is really very very fine post for jboss clustering with http & https protocols.
        Thank you for your soft supportive help and guide.

  4. I have a comma separated string as value of an action column which I would like to set in a string array parameter of an action method if the condition is met. However, I am just getting the first string with $1…how can I get comma separated string as string array?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s