<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><DIV>We are talking about application servers.</DIV><DIV><BR class="khtml-block-placeholder"></DIV>One of the toughest things about clustering in general and GFS in particular is the failure scenarios.<DIV><BR class="khtml-block-placeholder"></DIV><DIV>When you have any sort of cluster issue, if your root is on a shared GFS, that GFS freezes in various ways until fencing happens. The problem with this is that certain binaries that are on the same GFS may need to be used to recover. How do you execute fence_apc to fence a failed node when it is on a GFS that is hung waiting on that same fencing operation?</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>There are ways around this involving RAM disks and the like, but eventually we just settled on having a minimal flash disk that would get us onto our SAN (but not clustered). Only after we were on a non-clustered-FS on our SAN would we then start up our clustered filesystem. This gave us the ability to move our nodes around easily. This is an often overlooked benefit of a shared root that putting your root FS on SAN gives you as well. There's nothing like booting up a dead node on spare hardware. This also gives you a solid way to debug a damaged root system. With shared-root it's all or nothing. It's not so with this configuration. You also have separate syslog files and other things that are one more special case on a shared root. It's also easy to set up nodes with slightly different configurations (shared-root makes this another special case). As for the danger of drive failure, a read-only IDE flash disk (Google for Transcend) is simple, easy, and dead solid.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>After consolidating your shared configuration files into /etc/shared and placing appropriate symlinks into that directory, it is a simple matter of rsync / csync / tsync / cron+scp to keep them synchronized.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>It is tempting to want to have a shared root to minimize management requirements. It is tempting to want to play games with ramfs and the like to provide a support system that will function when that shared root is hung due to clustering issues. It is tempting to think that having a shared GFS root is really useful.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>However, if you value reliability and practicality, it's much easier to script up an occasional Rsync than it is to do so many acrobatics for such little gain. For a cluster (and its apps) to be reliable at all, it needs to be able to function, recover, and generally have a stable operating environment. Putting GFS under the userspace that drives it is asking for trouble.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><DIV><DIV>On Jan 31, 2007, at 1:34 PM, <A href="mailto:isplist@logicore.net">isplist@logicore.net</A> wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I'm thinking for application servers/cluster only, not workstation users.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On Wed, 31 Jan 2007 11:10:55 -0800, Tom Mornini wrote:</DIV> <BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">We boot from flash drives, then pivot root to SAN storage.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I agree with no drives in servers, but shared root is a</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">whole different ball game if you mean everyone using a</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">single filesystem for root.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">-- Tom Mornini, CTO</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">-- Engine Yard, Ruby on Rails Hosting</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">-- Reliability, Ease of Use, Scalability</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">-- (866) 518-YARD (9273)</DIV></BLOCKQUOTE></BLOCKQUOTE></DIV></DIV><BR><DIV> <SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><DIV>-- </DIV><DIV>Jayson Vantuyl</DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px;"><SPAN class="Apple-style-span" style="font-size: 10px; "><SPAN class="Apple-style-span" style="font-size: 10px; ">Systems Architect</SPAN></SPAN></SPAN></FONT></DIV><DIV><B style="font-weight: bold; "><A href="http://www.engineyard.com"><SPAN class="Apple-style-span" style="font-weight: normal;; color: rgb(0, 0, 238); -khtml-text-decorations-in-effect: underline; "><SPAN class="Apple-style-span" style="color: rgb(0, 0, 238); -khtml-text-decorations-in-effect: underline; ">Engine Yard</SPAN></SPAN></A></B></DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px;"><SPAN class="Apple-style-span" style="font-size: 10px; "><SPAN class="Apple-style-span" style="font-size: 10px; "><A href="mailto:jvantuyl@engineyard.com">jvantuyl@engineyard.com</A></SPAN></SPAN></SPAN></FONT></DIV></SPAN><BR class="Apple-interchange-newline"></SPAN></SPAN> </DIV><BR></BODY></HTML>