DRBD with OCFS2 and fstab

Two-nodes active/active DRBD cluster implemented on Debian Jessie with OCFS2 on top of it, so the file system can be mounted and accessed on both nodes at the same time. Sounds like a easy-peasy task considering the amount of articles on the web (mainly copy/paste of the same content though).

So, you finish with the setup, everything is synced and shiny, you edit fstab, perform the final reboot, and… oopsie daisy, nothing is mounted. You start digging into _netdev direction, or suspecting that perhaps an order in which drbd and ocfs2 are started is to blame, or putting mount stanza into rc.local — none of this helps. You might even come up with an excuse that you will not reboot those servers often, however, the fact that you need to manually perform some post-reboot actions doesn’t sound promising at all. Particularly if it’s an unexpected reboot over a weekend. Particularly if it happened some years after the installation hence you need to find (and most importantly, keep in mind about) those notes. Particularly if you already quit this job, and there is another poor fella taking care of the servers. And finally, to make things even more complicated, you might have services that actually depend on the availability of the mounted drive after the reboot (Apache or Samba for example).

Obviously, this needs to be fixed once and for all, and I have good news for you. :) If you were vigilant enough during troubleshooting you’d notice that a) if you try to mount the drive through /etc/rc.local there will be a warning thrown at boot time (something about missing device), and b) when you mount drbd drive manually it’s not mounted instantly — there is several seconds delay before the disk is successfully attached. That brought me to the suspicion that perhaps drbd is actually not ready at the time mount in /etc/rc.local is executed, and by deliberately introducing some delay things can be improved. And voila — it really did seem to do the trick!

Here is my /etc/fstab entry:

/dev/drbd0   /var/www   ocfs2   noauto,noatime   0   0

And here is my /etc/rc.local introducing 30 seconds delay prior to mount, to give enough time for DRBD to cool down:

sleep 30
mount /dev/drbd0
exit 0

Now, I’m not sure whether this is by design, since DRBD nodes do have to communicate with each other (initial election and/or sync), and that contributes to the delay in creating /dev/drbd0, OR, my environment is generally slow (everything is virtualized on not-so-super-fast SATA drives), but it works.

Tags: , , , , ,

2 Responses to “DRBD with OCFS2 and fstab”

  1. Adam Palmer says:

    Try ‘nofail,auto,_netdev’ as fstab options.

  2. r00t says:

    Thanks for the hint Adam. I’m pretty sure I’ve tried it with no luck tho…

Leave a Reply