<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><DIV><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Ok, might as well ask this... since I can't seem to find anything on it. How<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">about just a central storage that can be split up into many small segments so<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">that blades can boot over the network, then joint the GFS cluster?</DIV></BLOCKQUOTE>We use an IDE flash disk in each server. It's just too easy to put a readonly bootstrap image on the flash and boot up off of that. With affordable 256MB flash disks you can even have a powerful repair environment there in case things get broken.</DIV><DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I mean, all I want to do is to remove the drives since they really aren't<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">being used. All of the work is being done on the GFS cluster once a machine is<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">up and running. It barely does anything with it's drive other than the OS of<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">course, even logging is all remote.</DIV></BLOCKQUOTE>Don't remove the drives, use IDE flash drives instead. I think you can also use USB thumb drives if your BIOS supported it. A 256MB flash for $26.30 is hard to beat. Order directly from the manufacturer at:</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><A href="http://www.transcendusa.com/Products/ModDetail.asp?ModNo=26&LangNo=0">http://www.transcendusa.com/Products/ModDetail.asp?ModNo=26&LangNo=0</A></DIV><DIV><BR><DIV>We put the boot loader, kernel, and a simple maintenance environment on the flash. We still boot our root off of the SAN. Interestingly, our SAN supports partitioning. What we do here is have partitions for each node (automatically mounted using a LABEL= mount). After that boots up, we run CLVM with our GFSes on top of it. Quite handy (and CLVM isn't really necessary for your case).</DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Isn't there a simpler way of getting this done without having to get into<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">whole new technologies? All of the blades have PXE boot capabilities, there<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">must be some simple way of doing this?</DIV></BLOCKQUOTE>I'd avoid this. I've tried the PXE boot thing before and the PXE only becomes one more single point of failure / maintenance. There's nothing like rebooting your cluster only to find that the PXE server has a failed disk. :(</DIV><DIV><BR class="khtml-block-placeholder"></DIV>Basically with a SAN set up as follows:<DIV><BR class="khtml-block-placeholder"></DIV><DIV>/dev/san0p1 (FS for node 0, labeled node0)</DIV><DIV>/dev/san0p2 (FS for node 1, labeled node1)</DIV><DIV>...</DIV><DIV>/dev/san1 (CLVM / GFS / other stuff)</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Your boot flash doesn't need much more than a very tiny Linux system (busybox is your friend), a file containing the node id (in this case /node_id) and a /linuxrc containing:</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>#!/bin/sh</DIV><DIV>NODEID=`cat /node_id`</DIV><DIV># SET UP SAN HERE IF NECESSARY</DIV><DIV>mount /proc # Necessary because LABEL-mounting requires /proc/partitions</DIV><DIV>mount -o ro -L root-${NODEID} /newroot</DIV><DIV>cd /newroot</DIV><DIV>pivot_root . oldroot/</DIV><DIV>exec sbin/init</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Considering that your flash hardly ever changes, and you can script creating of the flash image and node partitions, this quickly becomes very low maintenance. If you want them to be identical, grab the MAC address off of the first NIC and generate the label with that...</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>A shared root is a nice idea. However you just end up creating a fragile custom environment that is hostile to lots of software and creates new single points of failure and contention (making it neither high-performance nor high-availability).</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV><DIV> <SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><DIV>-- </DIV><DIV>Jayson Vantuyl</DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px;"><SPAN class="Apple-style-span" style="font-size: 10px; "><SPAN class="Apple-style-span" style="font-size: 10px; ">Systems Architect</SPAN></SPAN></SPAN></FONT></DIV><DIV><B style="font-weight: bold; "><A href="http://www.engineyard.com"><SPAN class="Apple-style-span" style="font-weight: normal;; color: rgb(0, 0, 238); -khtml-text-decorations-in-effect: underline; "><SPAN class="Apple-style-span" style="color: rgb(0, 0, 238); -khtml-text-decorations-in-effect: underline; ">Engine Yard</SPAN></SPAN></A></B></DIV><DIV><FONT class="Apple-style-span" size="2"><SPAN class="Apple-style-span" style="font-size: 10px;"><SPAN class="Apple-style-span" style="font-size: 10px; "><SPAN class="Apple-style-span" style="font-size: 10px; "><A href="mailto:jvantuyl@engineyard.com">jvantuyl@engineyard.com</A></SPAN></SPAN></SPAN></FONT></DIV></SPAN><BR class="Apple-interchange-newline"></SPAN></SPAN></DIV><BR></DIV></BODY></HTML>