scaling drupal step two - sticky load balancing with apache mod_proxy
posted october 21st, 2007 by john
if you've setup your drupal deployment with a separate database and web (drupal) server (see scaling drupal step one - a dedicated data server), a good next step, is to cluster your web servers. drupal generates a considerable load on the web server and can quickly become resource constrained there. having multiple web servers also increases the the redundancy of your deployment. as usual, my examples are for apache2, mysql5 and drupal5 on debian etch. see the scalability overview for related articles.
one way to do this is to use a dedicated web server running apache2 and mod_proxy / mod_proxy_balancer to load balance your drupal servers.
deployment overview
this table summaries the characteristics of this deployment choice| scalability: | fair |
| redundancy: | fair |
| ease of setup: | fair |
servers
in this example, i use:
| web server | drupal-lb1.mydomain.com | 192.168.1.24 |
| web server | drupal-lb2.mydomain.com | 192.168.1.25 |
| data server | drupal-data-server1.mydomain.com | 192.168.1.26
|
| load balancer | apache-balance-1.mydomain.com | 192.168.1.34 |
network diagram
load balancer setup: install and enable apache and proxy_balancer
create a dedicated server for load balancing. install apache2 (apt-get install apache2) and then
install mod proxy_balancer and proxy_http with dependencies
# a2enmod proxy_balancer
# a2enmod proxy_httpenable mod_proxy in mods-available/proxy.conf. note that i'm leaving ProxyRequests off since we're only using the ProxyPass and ProxyPassReverse directives. this keeps the server secure from spammers trying to use your proxy to send email.
<IfModule mod_proxy.c>
# set ProxyRequests off since we're only using the ProxyPass and ProxyPassReverse
# directives. this keeps the server secure from
# spammers trying to use your proxy to send email.
ProxyRequests Off
<Proxy *>
AddDefaultCharset off
Order deny,allow
Allow from all
#Allow from .example.com
</Proxy>
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
ProxyVia On
</IfModule>configure mod_proxy and mod_proxy_balancer
mod_proxy and mod_proxy balancer serve as a very functional load balancer. however mod_proxy_balancer makes slightly unfortunate assumptions about the format of the cookie that you'll use for sticky session handling. one way to work around this is to create your own session cookie (very easy with apache). the examples below describe how to do this
first create a virtual host or use the default (/etc/apache2/sites-available/default) and add this configuration to it:
<Location /balancer-manager>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 192.168
</Location>
<Proxy balancer://mycluster>
# cluster member 1
BalancerMember http://drupal-lb1.mydomain.com:80 route=lb1
# cluster member 2
BalancerMember http://drupal-lb2.mydomain.com:80 route=lb2
</Proxy>
ProxyPass /balancer-manager !
ProxyPass / balancer://mycluster/ lbmethod=byrequests stickysession=BALANCEID
ProxyPassReverse / http://drupal-lb1.mydomain.com/
ProxyPassReverse / http://drupal-lb2.mydomain.com/- i'm allowing access to the balancer manager (the web UI) from any IP matching 192.168.*.*
- i'm load balancing between 2 servers (drupal-lb1.mydomain.com, drupal-lb2.mydomain.com) on port 80
- i'm defining two routes for these servers called lb1 and lb2
- i'm excluding (!) the balancer-manager directory fro the ProxyPass to allow access to the manager ui on the load balancing server
- i'm expecting a cookie called BALANCEID to be available to manage sticky sessions
- this is a simplistic load balancing configuration. apache has many options to control timeouts, server loading, failover etc. too much to cover but read more in the apache documentation
configure the web (drupal) servers to write a session cookie
on each of the web (drupal) servers, add this code to your vhost configuration:RewriteEngine On
RewriteRule .* - [CO=BALANCEID:balancer.lb1:.mydomain.com]making sure to specify the correct route e.g. lb1 on drupal-lb1.mydomain.com etc.
you also probably want to setup your cookie domain properly in drupal, i.e. modify drupal/sites/default/settings.php as follows:
# $cookie_domain = 'example.com';
$cookie_domain = 'mydomain.com';important urls
useful urls for testing are:- balancer manager ui: http://apache-balance-1.mydomain.com/balancer-manager
- direct access to drupal on lb1: http://drupal-lb1.mydomain.com/drupal/
- direct access to drupal on lb1: http://drupal-lb2.mydomain.com/drupal/
- access to drupal through the load balancer: http://apache-balance-1.mydomain.com/drupal/
the balancer manager
the mod_proxy_balancer ui enables point-and-click update of balancer members.
the balancer manager allows you to dynamically change the balance factor or a particular member, change it's route or put it in the off line mode.
debugging
to debug your configuration it's useful to turn up apache's debugging level on your apache load balancer by adding this to your vhost configuration:LogLevel debugthis will produce some very useful debugging output (/var/log/apache2/error.log) from the proxying and balancing code.
firefox's cookie viewer tools->options->privicy->show cookies is also useful to view and manipulate your cookies.
if you plan to experiment with bringing servers up and down to test them being added and removed from the cluster you should consider setting the "connection pool worker retry timeout" to a value lower than the default 60s. you could set them to e.g. 10s by changing your configuration to the one below. a 10s timeout allows for quicker test cycles.
BalancerMember http://drupal-lb1.scream.squaretrade.com:80 route=lb1 retry=10
BalancerMember http://drupal-lb2.scream.squaretrade.com:80 route=lb2 retry=10next steps
one single-point-of-failure in this deployment is the apache load balancer. consider clustering your load balancer with scaling drupal step three - using heartbeat to implement a redundant load balancer
references and documentation
- apache's mod_proxy_balancer documentation
- apache's mod_proxy documentation
- mark round's great article on managing your own sticky sessions with mod_proxy_balance
- getting started with heartbeat
tech blog
if you found this article useful, and you are interested in other articles on linux, drupal, scaling, performance and LAMP applications, consider subscribing to my technical blog. - john's blog
- 53632 reads
Good Morning, good
Good Morning, good afternoon, good evening
I need your expert advice? I am trying to setup Apache load balancing using Tomcat on the backend to run a servlet application. The problem is I can’t work out how to maintain stickysession using Tomcat. I can get stickysession working when using Apache on the backend (rewrite_mod) but not with Tomcat?
Any guidance you can provide to help me resolve this problem would be great help.
Software:
Apache 2.2.11
Tomcat 6.0.18
JDK 1.5.0.17
The servlet application uses a Login password control screen and every time you try to login in you are bounced to the other instance. I need Tomcat to maintain stickysession with apache to ensure the user can login.
The plan is to run this all on one server the Apache acting as load balancer for Both tomcat instances diverting the load evenly.
Apache Httpd.conf
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule rewrite_module modules/mod_rewrite.so
ProxyRequests off
ProxyPreserveHost on
Order deny,allow
Allow from all
ProxyPass /balancer-manager !
ProxyPass /TomcatA http://localhost:8080
ProxyPass /TomcatB http://localhost:8090
ProxyPass / balancer://mycluster/ stickysession=BALANCEID nofailover=off
#ProxyPass / balancer://mycluster/ stickysession=BALANCEDID
#ProxyPass / balancer://mycluster/ stickysession=JSESSIONID|jsessionid
ProxyPassReverse / http://localhost:8080/
ProxyPassReverse / http://localhost:8090/
BalancerMember http://localhost:8080 route=TomcatA retry=60
BalancerMember http://localhost:8090 route=tomcatB retry=60
#BalancerMember http://192.168.144.20:8080 route=jvm1 disablereuse=On
#BalancerMember http://192.168.144.20:8090 route=jvm2 disablereuse=On
ProxySet lbmethod=byrequests
SetHandler balancer-manager
Order deny,allow
Allow from all
Tomcat Instances TomcatA & TomcatB (server.xml)
TomcatA
TomcatB
Excellent series of Drupal
Excellent series of Drupal tutorials...
Thanks John!
so greate
so greate
Thanks! Very useful post.
Thanks! Very useful post. Apache mod_proxy_balancer provided to be delicate thing :)
Could you please let me know
Could you please let me know what exactly you are trying to accomplish by this rule:
RewriteRule .* - [CO=BALANCEID:balancer.lb1:.mydomain.com]
It writes a session cookie
It writes a session cookie (BALANCEID) to identify the balanced server in question.
This post is great,
This post is great, thanks.
But when I configuration my 2 webservers to 2 forums (use modules forum and Upload file), the problem is which server will store the file when I upload (1 or 2, but needs BOTH) ?
Ha, take a look at step one.
Ha, take a look at step one. there is a long discussion of this issue there, with solutions.
Why would the first step be
Why would the first step be setting up load balancing? Why not database master/slave replication to split the database load?
mike, thanks for stopping
mike, thanks for stopping by.
you're actually on step two, of the overall plan. please see the overview.
regarding master/slave replication, this would typically be fairly late in most LAMP scaling plans (although this is clearly application dependent). you'll typically hit cpu exhaustion from apache/php well before you get anywhere near overloading a well tuned mysql database.
i cover database replication and a general discussion on scaling your database tier in step four. if you are seeing very early scalability issues with your database, it's likely that you have a poorly optimized system e.g. un-indexed queries hitting large tables. take a look at step four for a general discussion of this.
Check out pound:
Check out pound: http://www.apsis.ch/pound/
post new comment