scaling drupal step one - a dedicated data server
if you've already installed drupal on a single node (see easy-peasy-lemon-squeezy drupal installation on linux), a good first step to scaling a drupal install is to create a dedicated data server. by dedicated data server i mean a server that hosts both the database and a fileshare for node attachments etc. this splits the database server load from the web server, and lays the groundwork for a clustered web server deployment. here's how you can do it. as usual, my examples are for apache2, mysql5 and drupal5 on debian etch. see the scalability overview for related articles.
deployment overview
this table summaries the characteristics of this deployment choice:| scalability: | poor |
| redundancy: | poor |
| ease of setup: | good |
servers
in this example, i use:
| web server | drupal-lb1.mydomain.com | 192.168.1.24 |
| data server | drupal-data-server1.mydomain.com | 192.168.1.26
|
update
the recipe below uses nfs, you might want to consider using rysnc as an alterative. see the discussion in step one B -- john, 11 nov 2007
if you plan to run with a single webserver for a while, you can skip the nfs / rsync malarky until step 2 -- john, 11 nov 2007
data server: setup mysql and prepare it for remote access
install mysql. i use mysql5. you'll need to enable this for remote access. edit/etc/mysql/my.cnf and change the bind address to your local server address e.g.
# bind-address = 127.0.0.1
bind-address = 192.168.1.26now allow access to your database from your web (drupal) servers. run mysql and do:
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE
TEMPORARY TABLES, LOCK TABLES
ON drupaldb.*
TO drupal@'drupal-lb1.mydomain.com' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;# /etc/init.d/mysql restartdata server: setup a shared nfs partition
moving drupal's file data (node attachments etc) to an nfs server does two things. it allows you to manage all your important data on a single server, simplifying backups etc. it also paves the way for web server clustering, where clearly it doesn't make sense to write these files onto random web servers in the cluster.install the nfs server:
apt-get install nfs-kernel-servermake a directory to share:
# mkdir -p /var/drupalSharedFiles/files
# chown www-data.www-data /var/drupalSharedFiles/files/etc/exports:
/var/drupalSharedFiles 192.168.1.1/24(rw,sync)now share it:
# exportfs -aweb (drupal) server: install drupal and point it to your data server
install drupal on your web server (see the drupal installation recipe for specifics). make sure that it can connect to your database server. you can verify database connectivity using:# mysql -e "select * from users limit 1" --user=drupal --host=drupal-data-server1.mydomain.com --password=password drupaldbif this doesn't work, go back to the instructions above and make sure you did your binding and granting properly.
now edit your sites/default/settings.php to point it to your new data server e.g.
$db_url = 'mysql://drupal:password@drupal-data-server1.mydomain.com/drupaldb';hit your drupal site and make sure that it sees the new database.
next, mount the area for shared files, add this to the /etc/fstab file:
192.168.1.26:/var/drupalSharedFiles /var/drupalSharedFiles nfs rw,hard,intr,async,users,noatime 0 0make sure that you've got the portmapper installed on the client, you'll need this to do a performant nfs mount. if you don't have it:
# apt-get install portmap# mount -afiles directory to point to your remote one, cd to your drupal directory (probably /var/www/drupal) and:
# cp -a files/.htaccess /var/drupalSharedFiles/files
# rm files/.htaccess ; rmdir files
# ln -s /var/drupalSharedFiles/files/ .http://drupal-lb1.mydomain.com/drupal/?q=admin/logs/status and making sure that it sees your nfs area as writable. you can also just upload an attachment and see what happens.
you should be all set.
next steps
got all that working? want more scalability and redundancy? consider clustering your drupal servers with step two - sticky load balancing with apache mod proxytech blog
if you found this article useful, and you are interested in other articles on linux, drupal, scaling, performance and LAMP applications, consider subscribing to my technical blog. 
delicious
digg
reddit
google
yahoo
since i wrote this article
since i wrote this article robert douglass has written a very nice article on using Lighttpd as a static file server for drupal that you should check out. his solution promises fast sharing of uploaded files and other static drupal content. i suspect that this would also be helpful if you're considering cdn hosting of your static content.
Alternate data server The
Alternate data server
The solution I am planning to use is shared iSCSI storage using OCFS2 for the file system.
Cheers
yea, i've heard very good
yea, i've heard very good things about OCFS2. i'd love to hear how it works out for you.
Great articles! Do you
Great articles!
Do you prefer to just generically use the data servers? or is it a good practice to have something such as
files1.mydomain.com
db1.mydomain.com
etc, the main load balancing server does nothing but dibby up the loads correct?
Tj, it really depends on
Tj, it really depends on your application, but if you have a lot of traffic to your file server, then you would benefit by splitting it from your database server. if you don't they'll both be competing for network, filesystem buffer, disk io and cpu resources.
Great tips! These are
Great tips! These are really handy tutorials.
One thing that I think is missing is the motivation for using an nfs mount. I can guess at why, but it's not clear what benefit that provides.
And a nitpick - I really like using example.com for the domain as opposed to "mydomain" or "yoursite" since those are real sites that have other purposes.
greggles. both good points.
greggles. both good points. thanks.
i'll use example.com in future.
and yea, you are right, the reason for the nfs mount isn't obvious here. i've clarified it above as follows: moving drupal's file data (node attachments etc) to an nfs server does two things. it allows you to manage all your important data on a single server, simplifying backups etc. it also paves the way for web server clustering, where clearly it doesn't make sense to write these files onto random web servers in the cluster.
post new comment