[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[K12OSN] LTSP+OpenMosix: Rev. 3.3

For those attempting the K12LTSP+OpenMosix installation (and there are many
of you, I've been replying to this issue all night):

Rev. 3.2 of the How-To is wrong in indicating that you can use the
ltsp_initrd_3.0.3-i386.tgz files for readying the client kernels with buildk
in the manner that I indicated.  It will not work.  You must use 3.0.1.  The
attached document corrects this error and contains links to the 3.0.1 kit.

I can tell you all that the interest in this is really growing from the
number of e-mails I have been receiving.  I believe that you will see
success with this latest document and my apologies for the errors in
previous versions.  I never imagined my first how-to would generate this
kind of response...

Now, FWIW, ltsp_initrd_3.0.3-i386.tgz looks promising in it's features, I
just haven't been able to get it to work for me yet...

James Jensen
Title: LTSP+OpenMosix: A Mini How-To

LTSP+OpenMosix: A Mini How-To

Revision 3.3

What is this?

This document will provide details (via an example set-up) of combining the OpenMosix clustering system with the K12LTSP 2.0 distribution, an innovative GNU/Linux distribution based on RedHat 7.2 & LTSP 3.0 (a set of Open Source software packages from the Linux Terminal Server Project that makes it all happen). OpenMosix (http://openmosix.sourceforge.net/) is a completely Open Source fork of the MOSIX clustering system.

OpenMosix, like the original MOSIX, is a clustering system that in the most basic of terms combines the cpu resources of the client systems into what could be compared to an SMP processing system. Processes get "moved" through load balancing to the the nodes (Linux Terminal Client computers) in the cluster that can be most effective in running them. Fully automated. No special programming required as in other clustering systems. The hope or theory is that since the K12LTSP client is simply executing XWindows and nothing more that OpenMosix can be implemented to utilize the idle processor cycles on these clients to maximize the efficiency of the entire system--relieving some of the stress that the server is under.

This document was derived as a result of Richard Camps very useful How-To, "Mosix Cluster with Diskless Nodes", which detailed adding MOSIX clustering functionality to an installation of the K12LTSP 2.0 distribution. (Without his article I would still be fighting to get this stuff working!). However, it should be noted that I alone am responsible for this particular document and all of the mistakes in it are mine! :-)

Mandatory Disclaimer: This article is provided as an example of how I got this working on my test setup. It is not the final word on the subject and it should be noted--use this information at your own risk. No warranty or guarantee is expressed or implied. I am not responsible for any damage resulting from your implementation of this information. Again, use this How-To at your own risk.

Having said that, let's get going!

Why OpenMosix instead of the original MOSIX?

  1. As I stated previously, Richard Camp posted an excellent How-To on installing the original MOSIX system with the K12LTSP 2.0.x distribution. It pointed me in the right direction and I'm truly grateful for his efforts. Without which, who knows if I'd have ever gotten anywhere with the concept of combining LTSP & MOSIX. Unfortunately in my case, the MOSIX install script that came from the MOSIX download always failed when the 2.4.17 kernel compile would throw errors during the modules_install phase. MOSIX simply would not patch & install automatically with the installation script. I began investigating a manual MOSIX kernel patch & userland tools installation and came across the OpenMosix project in the process. I found OpenMosix very simple to implement.

  2. MOSIX is either no longer Open Source or it's status as Open Source software is uncertain at this time. Without clarity on this issue, I would think that anyone implementing MOSIX should first obtain permission to do so from the copyright holder, Amnon Barak.

    As of this writing, I don't have all of the details of MOSIX's current licensing status and I did not find any statement regarding the issue on the MOSIX site but you may wish to review the following for more information on the matter--

    About OpenMosix Page:

    Interview with Moshe Bar (founder of OpenMosix & a former project manager for MOSIX):

    The OpenMosix FAQ:

The uncertainty surrounding MOSIX licensing and the ease of installation makes OpenMosix an ideal alternative for our purposes.

MOSIX Installation background Information, do not proceed without it!!!

Please! Do yourself a favor, obtain, study, and digest Richard Camps original MOSIX+LTSP How-To "Mosix Cluster with Diskless Nodes". Richard's How-To contains information that will not be covered here but may prove useful in getting your OpenMosix cluster running.

You can find his original article at http://www.k12os.org/print.php?sid=14

And, without further ado, away we go...

My Test Set-up used in this example:

        Pentium II w/ 6GB IDE drive, 96MB RAM, Intel 10/100 NIC. K12LTSP 2.0 installed.
        Various systems, booting from floppy: 486, Pentium, Pentium MMX, Pentium II.
        3com 10Mbit or Intel 10/100Mbit NICs. 16-64MB RAM
        Simple 4-port 10baseT hub. (How's that for bandwidth? ;-)

Nothing spectacular here just an odd assortment of stuff, but it's been a useful test setup for me. And, even more importantly, it actually works! :-)

K12LTSP 2.0.x Distribution:

LTSP Files (particularly "ltsp_initrd_kit-3.0.1-i386.tgz"):

OpenMosix Files:

Files needed in this example (Note: this example employs the latest openmosix software at this writing, openmosix5. Obtain the following OpenMosix files from SourceForge):

  1. The K12LTSP 2.0.x Distribution set
  2. ltsp_initrd_kit-3.0.1-i386.tgz (from the LTSP File List page. Currently available in the LTSP 3.0 ISO image under the directory /3.0/tgzs/ or here. The md5sum for this file from the CD is: bc5e68e1ee75a6ecf304c629ec7f0687.)
  3. openmosix-kernel-2.4.16-openmosix5.i686.rpm (Server Kernel)
  4. openmosix-tools-0.1.3-6.i386.rpm (Userland Tools for OpenMosix)
  5. openmosix-kernel-2.4.16-openmosix5.src.rpm (Source files for Client Kernel)

    OR (instead of the openmosix-kernel-2.4.16-openmosix5.src.rpm)

  6. openMosix2.4.16-4.gz (Patch for Kernel 2.4.16 if not using the above source rpm)
  7. Kernel 2.4.16 from The Linux Kernel Archives (if not using the above source rpm)

mcedit is the text editor that will be used throughout this example. If you wish to utilize it, you can install it from the RPMS section on Disc 1 of the K12LTSP 2.0.x distribution via: rpm -ivh mc-4.5.51-36.i386.rpm, or substitute the editor of your choice.

The step-by-step:

(Note: items following the ">" are command line items you need to run.)

  1. Download the files listed above.
  2. Install the K12LTSP 2.0.x distribution. When prompted in the Package Group Selection portion of the installation add "Software Development" as one of the selections (helps to have these packages installed for adding OpenMosix).
  3. Copy the downloaded OpenMosix & LTSP files to your LTSP server.
  4. Navigate to the folder containing the downloads for the project: > cd /your-download-folder/
  5. Install the OpenMosix kernel to the server via the command: > rpm -ivh openmosix-kernel-2.4.16-openmosix5.i686.rpm. This adds the kernel to the grub bootloader as well, or a link "vmlinuz-openmosix" that can be used for lilo (just add it to lilo.conf).
  6. Install the Userland tools "openmosix-tools-0.1.3-6.i386.rpm" via the command: > rpm -ivh openmosix-tools-0.1.3-6.i386.rpm
  7. Install the OpenMosix kernel source "openmosix-kernel-2.4.16-openmosix5.src.rpm" via the command: > rpm -ivh openmosix-kernel-2.4.16-openmosix5.src.rpm
  8. Download "openMosix2.4.16-4.gz" and a copy of linux-2.4.16.tar.bz2 from The Linux Kernel Archives
  9. > cp /your-download-folder/linux-2.4.16.tar.bz2 /usr/src
    or, if you installed the openmosix-kernel-2.4.16-openmosix5.src.rpm then run:
    > cp /usr/src/redhat/SOURCES/linux-2.4.16.tar.bz2 /usr/src/
  10. > cd /usr/src
  11. > tar xjvf linux-2.4.16.tar.bz2
  12. > mv /usr/src/linux /usr/src/linux-2.4.16-om5
  13. > cd /usr/src/linux-2.4.16-om5
  14. > mcedit /usr/src/linux-2.4.16-om5/Makefile
  15. Set "EXTRAVERSION = -om5" (without the quotation marks)
  16. F10 to save & quit.
  17. > cp/your-download-folder/openMosix2.4.16-4.gz /usr/src/linux-2.4.16-om5/
    or, if you installed the openmosix-kernel-2.4.16-openmosix5.src.rpm then run:
    > cp /usr/src/redhat/SOURCES/openMosix2.4.16-4.gz /usr/src/linux-2.4.16-om5/
  18. > gunzip /usr/src/linux-2.4.16-om5/openMosix2.4.16-4.gz
  19. > patch -Np1 < openMosix2.4.16-4
  20. > make mrproper
  21. > cp /opt/ltsp/kernel-configs/config.2.4.9-ltsp-5 /usr/src/linux-2.4.16-om5/.config
  22. > make oldconfig (or menuconfig if you prefer)
  23. Enable the OpenMosix options
    a)Yes to process migration support, Kernel Diagnostics, Direct File-System Access, OpenMosix File-System.
    b) No to Kernel Debugger.
    c) Defaults for remaining items.
  24. > make dep
  25. > make bzImage
  26. > make modules
  27. > make modules_install
  28. > cp -R /lib/modules/2.4.16-om5 /opt/ltsp/i386/lib/modules/
  29. If you haven't already done so, download "ltsp_initrd_kit-3.0.1-i386.tgz" from the LTSP Project File List, if available or obtain it from the ltsp_cd.iso, or here. (ltsp_initrd_3.0.3-i386.tgz will not work for this example!)
  30. > cp /your-download-folder/ltsp_initrd_kit-3.0.1-i386.tgz /usr/src/
  31. > cd /usr/src
  32. > tar xzvf ltsp_initrd_kit-3.0.1-i386.tgz
  33. > cd /usr/src/ltsp_initrd_kit
  34. > mcedit buildk
  35. Scroll to the bottom and comment out the "prepare_kernel" lines with a preceeding "#" (without the quotation marks)
  36. Add the following line "prepare_kernel /usr/src/linux-2.4.16-om5 2.4.16-om5" (without quotation marks)
  37. F10 to save & quit
  38. > ./buildk
  39. > cd /tftpboot/lts/
  40. > ls
  41. Your new OpenMosix client kernel should appear as "vmlinuz-2.4.16-om5"
  42. If you wish, create a symbolic link to the OpenMosix kernel for easier upgrading in the future (you won't have to edit the client filename line in dhcpd.conf later)
  43. > ln -s vmlinuz-2.4.16-om5 vmlinuz-openmosix
  44. > cd /etc
  45. > mcedit dhcpd.conf
  46. Edit the "filename..." line using one of the two lines below depending on whether or not you created the symbolic link:
             a) filename "/lts/vmlinuz-openmosix";
             b) filename "/lts/vmlinuz-2.4.16-om5";
  47. Add the following line (all on one line) in dhcpd.conf (above the trick from Peter comment):
    option host-name = concat( "ws" , binary-to-ascii( 10, 8, "", substring( reverse( 1, leased-address), 0, 1)));
  48. F10 to save & exit
  49. > service dhcpd restart
  50. Copy & edit other files as indicated in Richard's How-To "Mosix Cluster with Diskless Nodes". Specifically, we are concerned with the following:
  51. Create & save the mosix.map file:
  52. > cd /etc
  53. > mcedit mosix.map
  54. Enter the following text for mosix.map:
    1 1
    2 253
  55. F10 to Save & Exit
  56. > cp /etc/mosix.map /opt/ltsp/i386/etc/ (Note: exact copies of mosix.map should be maintained in both locations.)
  57. Copy the userland tools & other files for the LTSP clients:
  58. > cp /sbin/setpe /opt/ltsp/i386/sbin/
  59. > cp /bin/mosrun /opt/ltsp/i386/bin/
  60. > cp /usr/bin/mosmon /opt/ltsp/i386/usr/bin/
  61. > cp /usr/bin/mosctl /opt/ltsp/i386/usr/bin/
  62. > cp /usr/bin/migrate /opt/ltsp/i386/usr/bin/
  63. ***NOTE***
    See Appendix A below for a list of other Userland tools that you may wish to copy for Linux Terminal Client node use. Simply copy them from their existing locations to the corresponding /opt/ltsp/i386/ locations as you just did for the previous files. You may need to make some or all of them available to the Linux Terminal nodes to effectively utilize the cluster.

  64. > cp /bin/touch /opt/ltsp/i386/bin/
  65. > rm /opt/ltsp/i386/etc/hosts
  66. > cp /etc/hosts /opt/ltsp/i386/etc/
  67. > cp /etc/rc.d/init.d/openmosix /opt/ltsp/i386/etc/rc.openmosix
  68. > mkdir /opt/ltsp/i386/mfs
  69. > cd /opt/ltsp/i386/etc/
  70. > mcedit fstab
  71. Add the following line to fstab:
    none /mfs mfs dfsa=1 0 0
  72. F10 to Save & Exit
  73. > mcedit /opt/ltsp/i386/etc/rc.local
  74. Add the following lines to the end of rc.local:
    # OpenMosix startup section
    # we don't want any terminal processes to migrate
    echo 1 > /proc/mosix/admin/lstay
    # start mosix
    /etc/rc.openmosix start
    # mount mfs filesystem. doesn't work when done earlier
    mount /mfs
    # End OpenMosix startup section
  75. F10 to Save and exit
  76. Once completed, and you verify that everything boots as it should, use the command > mosmon -t to check your work, hopefully you will see all of your OpenMosix nodes in action!

Some other quick notes once you have everything running properly and you're experienced with K12LTSP:

  1. If you don't need a GUI on your LT/OpenMosix nodes edit the dhcpd.conf & lts.conf files to set the system(s) to "RUNLEVEL = 3".
  2. The MOSIX site lists a couple of ways to test your cluster with awk that will work just as well with OpenMosix. See the following:

This document (Rev. 3.3) corrects errors found in the previous version, particularly the fact that you cannot use ltsp_initrd_kit-3.0.3-i386.tgz in the installation as described above, you need 3.0.1. (If you have success with 3.0.3 please let me know the details.) Hopefully, with these corrections, this How-To gives us a complete step-by-step process for easily implementing OpenMosix on a K12LTSP server. Richard Camp is in the process of producing more complete documentation, scripts & other items that he will make available at his website when ready.

Thanks to everyone out there that develops & supports Open Source software. Particular thanks to the folks at the Linux Terminal Server Project, the K-12Linux Project, and the openMosix Project for making this possible. And, finally special thanks goes out to Richard Camp, B. Smith, and T. Rhodes for their valuable feedback as this document has developed.

I hope with this revision that you will have success in implementing OpenMosix with LTSP. Amazing to me is the fact that while I can do this install "in my sleep" it has been somewhat difficult to get it right on "paper". My apologies to those who may have been become frustrated in previous attempts at implementing this due to my errors. This really does work! Don't give up!

Happy clustering!

James Jensen

Rev. 3.3
Friday, April 19, 2002

Appendix A

OpenMosix Userland Tools Chart

Summary of each file is based on it's manpage entry.

Userland File Summary
/bin/mosrun Run a command with particular node-allocation preferences.
/bin/mps Report multicomputer process status.
/etc/mosix.map OpenMosix node map file.
/etc/rc.d/init.d/openmosix OpenMosix init script.
/sbin/setpe MOSIX node configuration.
/usr/bin/cpujob mosrun frontend script: Runs a command advising MOSIX that it is a computation intensive program.
/usr/bin/fastdecay mosrun frontend script: runs a command advising MOSIX to apply fast statistics-decaying.
/usr/bin/iojob mosrun frontend script: runs a command, advising MOSIX that it is an I/O intensive program.
/usr/bin/migrate Request migration of a particular process on MOSIX.
/usr/bin/mosctl MOSIX system administrator's tools.
/usr/bin/mosmon MOSIX load monitor
/usr/bin/mtop Display multicomputer top CPU processes.
/usr/bin/nodecay mosrun frontend script: runs a command, advising MOSIX to not decay the program's statistics.
/usr/bin/nomig mosrun frontend script: runs a command with a node-lock.
/usr/bin/runhome Runs a command locked at the home node.
/usr/bin/runon mosrun frontend script: runs a command locked on a particular MOSIX node.
/usr/bin/slowdecay mosrun frontend script: runs a command, advising MOSIX to apply slow statistics-decaying.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]