Highly Available NFS on Ubuntu 10.04-Lucid Lynx server

Highly Available NFS setup that I have installed Consists of two machines(nodes), Both having 2 physical HDDs and 2 network adapters. DRBD, Heartbeat,NFS and NTP are the necessary utilities. This installation has been tested on Ubuntu 10.04.

Installation

The following utilities should be installed on both the machines.

DRBD can be installed using the following command

$ sudo apt-get install drbd8-utils

Heartbeat can be installed using the following command

$ sudo apt-get install heartbeat

NFS server can be installed using the following command

$ sudo apt-get install nfs-kernel-server

NTP can be installed using the following command

$ sudo apt-get install ntp

Configuration

Common file Configurations to be done on both the nodes

/etc/drbd.conf

Edit the file /etc/drbd.conf, delete all the lines that are already present and add the following lines

resource r0 {

protocol C;

handlers {
pri-on-incon-degr "echo o > /proc/sysrq-trigger ; halt -f";
pri-lost-after-sb "echo o > /proc/sysrq-trigger ; halt -f";
local-io-error "echo o > /proc/sysrq-trigger ;halt -f";
outdate-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";
}

startup {
degr-wfc-timeout 120;
}

disk {
on-io-error detach;
}

net {
cram-hmac-alg sha1;
shared-secret "secret";
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
}

syncer {
rate 100M;
verify-alg sha1;
al-extents 257;
}

on <hostname ofnode1> {
device  /dev/drbd0;
disk    /dev/sdb1;
address <ipaddress of eth1 of node1>:7788;
meta-disk internal;
}

on <hostname of node2> {
device  /dev/drbd0;
disk    /dev/sdb1;
address <ip-address of eth1 of node2>:7788;
meta-disk internal;
}

/etc/heartbeat/ha.cf

Edit the file /etc/heartbeat/ha.cf and add the following lines

logfacility local0
keepalive 2
deadtime 30
warntime 10
initdead 120
bcast eth0
bcast eth1
node <hostname of node1>
node <hostname of node2>

/etc/heartbeat/haresources

Edit the file /etc/heartbeat/haresources and add the following line

<hostname of node1>      IPaddr::<virtual IP>/24/eth1   drbddisk::r0    Filesystem::/dev/drbd0::/srv/data::ext3 nfs-kernel-server

/etc/heartbeat/authkeys

Edit the /etc/heartbeat/authkeys file and add the following lines

auth 3
3 md5 password

Now change the permissions for the /etc/heartbeat/authkeys using the following command

$ sudo chmod 600 /etc/heartbeat/authkeys

/etc/hosts

Add appropriate entries in /etc/hosts file in case there is no DNS server in the network.

Other Configurations to be done on both the nodes

Create a new partition in the second Hard-drive named sdb1 using any appropriate tool

Initialize the meta-data disk by using the following command

$ sudo drbdadm create-md r0

Run the following commands

$ sudo chgrp haclient /sbin/drbdsetup
$ sudo chmod o-x /sbin/drbdsetup
$ sudo chmod u+s /sbin/drbdsetup
$ sudo chgrp haclient /sbin/drbdmeta
$ sudo chmod o-x /sbin/drbdmeta
$ sudo chmod u+s /sbin/drbdmeta

Start the DRBD service using the following command

$ sudo /etc/init.d/drbd start

Configurations to be done on Node1

Run the following commands

$ sudo drbdadm -- --overwrite-data-of-peer primary all
$ sudo mkfs.ext3 /dev/drbd0
$ sudo mount /dev/drbd0 /srv
$ sudo mkdir /srv/data
$ sudo mount /dev/drbd0 /srv/data
$ sudo mv /var/lib/nfs/ /srv/data/
$ sudo ln -s /srv/data/nfs/ /var/lib/nfs
$ sudo mv /etc/exports /srv/data
$ sudo ln -s /srv/data/exports /etc/exports
$ mkdir /srv/data/export

Edit the file /etc/exports and add the following line

/srv/data/export        10.10.1.10/24(rw,no_subtree_check)</pre>

Configurations to be done on Node2

Run the following commands

$ sudo rm -rf /var/lib/nfs
$ sudo ln -s /srv/data/nfs/ /var/lib/nfs
$ sudo rm /etc/exports
$ sudo ln -s /srv/data/exports /etc/exports

Reboot both the nodes.