[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [K12OSN] Open-mosix question

Valid questions all.  There are so many factors involved, from hardware
combinations to what processes are running, to network bandwidth.  I don't
have the answers for you, but I will say that it won't hurt to try, IMO.  If
you find that something is not to your liking with it, you can always switch
back to the standard K12LTSP distribution kernels.

The key here is testing this in a larger scale environment and somehow
gauging the response.  I'm working towards that but I don't have the
resources put it together at this time.  I know some of you guys hate
attachments on the list but there were a couple of big errors in the last
update of my how-to.  I have fixed those errors and tested them on ground up
installations twice today.  No problems.  So in any event if you do attempt
it you won't be calling me names because you can't find certain files listed
in the previous version!  LOL!  (Of course in all seriousness the disclaimer
in the doc remains totally in effect...)

What is your setup like?  How many clients?  What kind of hardware, network
capacity, etc?

I know that there are at least a few folks out there giving it a go, perhaps
in larger environs.  Hopefully we can get some feedback from them to
formulate some answers to your questions...



----- Original Message -----
From: "Daniel Bodanske" <danielbo clickta com>
To: <k12osn redhat com>
Sent: Wednesday, April 17, 2002 11:38 AM
Subject: [K12OSN] Open-mosix question

I've been watching the open-mosix discussion, and, excuse my ignorance,
but is anyone using this in a normal LTSP environment, should you be,
and, if so, does it make any real difference in this kind of situation.
 If it's worth setting up then I'll do it: if not, then I won't.

K12OSN mailing list
K12OSN redhat com
For more info see <http://www.k12os.org>
Title: LTSP+OpenMosix: A Mini How-To

LTSP+OpenMosix: A Mini How-To

What is this?

This document will provide details (via an example set-up) of combining the OpenMosix clustering system with the K12LTSP 2.0 distribution, an innovative GNU/Linux distribution based on RedHat 7.2 & LTSP 3.0 (a set of Open Source software packages from the Linux Terminal Server Project that makes it all happen). OpenMosix (http://openmosix.sourceforge.net/) is a completely Open Source fork of the MOSIX clustering system.

OpenMosix, like the original MOSIX, is a clustering system that in the most basic of terms combines the cpu resources of the client systems into what could be compared to an SMP processing system. Processes get "moved" through load balancing to the the nodes (Linux Terminal Client computers) in the cluster that can be most effective in running them. Fully automated. No special programming required as in other clustering systems. The hope or theory is that since the K12LTSP client is simply executing XWindows and nothing more that OpenMosix can be implemented to utilize the idle processor cycles on these clients to maximize the efficiency of the entire system--relieving some of the stress that the server is under.

This document was derived as a result of Richard Camps very useful How-To, "Mosix Cluster with Diskless Nodes", which detailed adding MOSIX clustering functionality to an installation of the K12LTSP 2.0 distribution. (Without his article I would still be fighting to get this stuff working!). However, it should be noted that I alone am responsible for this particular document and all of the mistakes in it are mine! :-)

Mandatory Disclaimer: This article is provided as an example of how I got this working on my test setup. It is not the final word on the subject and it should be noted--use this information at your own risk. No warranty or guarantee is expressed or implied. I am not responsible for any damage resulting from your implementation of this information. Again, use this How-To at your own risk.

Having said that, let's get going!

Why OpenMosix instead of the original MOSIX?

  1. As I stated previously, Richard Camp posted an excellent How-To on installing the original MOSIX system with the K12LTSP 2.0.x distribution. It pointed me in the right direction and I'm truly grateful for his efforts. Without which, who knows if I'd have ever gotten anywhere with the concept of combining LTSP & MOSIX. Unfortunately in my case, the MOSIX install script that came from the MOSIX download always failed when the 2.4.17 kernel compile would throw errors during the modules_install phase. MOSIX simply would not patch & install automatically with the installation script. I began investigating a manual MOSIX kernel patch & userland tools installation and came across the OpenMosix project in the process. I found OpenMosix very simple to implement.

  2. MOSIX is either no longer Open Source or it's status as Open Source software is uncertain at this time. Without clarity on this issue, I would think that anyone implementing MOSIX should first obtain permission to do so from the copyright holder, Amnon Barak.

    As of this writing, I don't have all of the details of MOSIX's current licensing status and I did not find any statement regarding the issue on the MOSIX site but you may wish to review the following for more information on the matter--

    About OpenMosix Page:

    Interview with Moshe Bar (founder of OpenMosix & a former project manager for MOSIX):

    The OpenMosix FAQ:

The uncertainty surrounding MOSIX licensing and the ease of installation makes OpenMosix an ideal alternative for our purposes.

MOSIX Installation background Information, do not proceed without it!!!

Please! Do yourself a favor, obtain, study, and digest Richard Camps original MOSIX+LTSP How-To "Mosix Cluster with Diskless Nodes". Richard's How-To contains information that will not be covered here but may prove useful in getting your OpenMosix cluster running.

You can find his original article at http://www.k12os.org/print.php?sid=14

And, without further ado, away we go...

My Test Set-up used in this example:

        Pentium II w/ 6GB IDE drive, 96MB RAM, Intel 10/100 NIC. K12LTSP 2.0 installed.
        Various systems, booting from floppy: 486, Pentium, Pentium MMX, Pentium II.
        3com 10Mbit or Intel 10/100Mbit NICs. 16-64MB RAM
        Simple 4-port 10baseT hub. (How's that for bandwidth? ;-)

Nothing spectacular here just an odd assortment of stuff, but it's been a useful test setup for me. And, even more importantly, it actually works! :-)

K12LTSP 2.0.x Distribution:

LTSP Files (particularly "ltsp_initrd_kit-3.0.x-i386.tgz"):

OpenMosix Files:

Files needed in this example (Note: this example employs the latest openmosix software at this writing, openmosix5. Obtain the following OpenMosix files from SourceForge):

  1. The K12LTSP 2.0.x Distribution set
  2. initrd_kit-3.0.x-i386.tgz (from the LTSP File List page)
  3. openmosix-kernel-2.4.16-openmosix5.i686.rpm (Server Kernel)
  4. openmosix-tools-0.1.3-6.i386.rpm (Userland Tools for OpenMosix)
  5. openmosix-kernel-2.4.16-openmosix5.src.rpm (Source files for Client Kernel)

    OR (instead of the openmosix-kernel-2.4.16-openmosix5.src.rpm)

  6. openMosix2.4.16-4.gz (Patch for Kernel 2.4.16 if not using the above source rpm)
  7. Kernel 2.4.16 from The Linux Kernel Archives (if not using the above source rpm)

mcedit is the text editor that will be used throughout this example. If you wish to utilize it, you can install it from the RPMS section on Disc 1 of the K12LTSP 2.0.x distribution via: rpm -ivh mc-4.5.51-36.i386.rpm, or substitute the editor of your choice.

The step-by-step:

(Note: items following the ">" are command line items you need to run.)

  1. Download the files listed above.
  2. Install the K12LTSP 2.0.x distribution. When prompted in the Package Group Selection portion of the installation add "Software Development" as one of the selections (helps to have these packages installed for adding OpenMosix).
  3. Copy the downloaded OpenMosix & LTSP files to your LTSP server.
  4. Navigate to the folder containing the downloads for the project: > cd /your-download-folder/
  5. Install the OpenMosix kernel to the server via the command: > rpm -ivh openmosix-kernel-2.4.16-openmosix5.i686.rpm. This adds the kernel to the grub bootloader as well, or a link "vmlinuz-openmosix" that can be used for lilo (just add it to lilo.conf).
  6. Install the Userland tools "openmosix-tools-0.1.3-6.i386.rpm" via the command: > rpm -ivh openmosix-tools-0.1.3-6.i386.rpm
  7. Install the OpenMosix kernel source "openmosix-kernel-2.4.16-openmosix5.src.rpm" via the command: > rpm -ivh openmosix-kernel-2.4.16-openmosix5.src.rpm
  8. Download "openMosix2.4.16-4.gz" and a copy of linux-2.4.16.tar.bz2 from The Linux Kernel Archives
  9. > cp /your-download-folder/linux-2.4.16.tar.bz2 /usr/src
    or, if you installed the openmosix-kernel-2.4.16-openmosix5.src.rpm then run:
    > cp /usr/src/redhat/SOURCES/kernel-2.4.16.tar.bz2 /usr/src/
  10. > cd /usr/src
  11. > tar xjvf linux-2.4.16.tar.bz2
  12. > mv /usr/src/linux /usr/src/linux-2.4.16-om5
  13. > cd /usr/src/linux-2.4.16-om5
  14. > mcedit /usr/src/linux-2.4.16-om5/Makefile
  15. Set "EXTRAVERSION = -om5" (without the quotation marks)
  16. F10 to save & quit.
  17. > cp/your-download-folder/openMosix2.4.16-4.gz /usr/src/linux-2.4.16-om-5/
    or, if you installed the openmosix-kernel-2.4.16-openmosix5.src.rpm then run:
    > cp /usr/src/redhat/SOURCES/openMosix2.4.16-4.gz /usr/src/linux-2.4.16-om5/
  18. > gunzip /usr/src/linux-2.4.16-om5/openMosix2.4.16-4.gz
  19. > patch -Np1 < openMosix2.4.16-4
  20. > make mrproper
  21. > cp /opt/ltsp/kernel-configs/config.2.4.9-ltsp-5 /usr/src/linux-2.4.16-om5/.config
  22. > make oldconfig (or menuconfig if you prefer)
  23. Enable the OpenMosix options
    a)Yes to process migration support, Kernel Diagnostics, Direct File-System Access, OpenMosix File-System.
    b) No to Kernel Debugger.
    c) Defaults for remaining items.
  24. > make dep
  25. > make bzImage
  26. > make modules
  27. > make modules_install
  28. > cp -R /lib/modules/2.4.16-om5 /opt/ltsp/i386/lib/modules/
  29. If you haven't already done so, download "ltsp_initrd_kit-3.0.3-i386.tgz" from the LTSP Project File List
  30. > cp /your-download-folder/ltsp_initrd_kit-3.0.3-i386.tgz /usr/src/
  31. > cd /usr/src
  32. > tar xzvf ltsp_initrd_kit-3.0.3-i386.tgz
  33. > cd /usr/src/ltsp_initrd_kit
  34. > mcedit buildk
  35. Scroll to the bottom and comment out the "prepare_kernel" lines with a preceeding "#" (without the quotation marks)
  36. Add the following line "prepare_kernel /usr/src/linux-2.4.16-om5 2.4.16-om5" (without quotation marks)
  37. F10 to save & quit
  38. > ./buildk
  39. > cd /tftpboot/lts/
  40. > ls
  41. Your new OpenMosix client kernel should appear as "vmlinuz-2.4.16-om5"
  42. If you wish, create a symbolic link to the OpenMosix kernel for easier upgrading in the future (you won't have to edit the client filename line in dhcpd.conf later)
  43. > ln -s vmlinuz-2.4.16-om5 vmlinuz-openmosix
  44. > cd /etc
  45. > mcedit dhcpd.conf
  46. Edit the "filename..." line using one of the two lines below depending on whether or not you created the symbolic link:
             a) filename "/lts/vmlinuz-openmosix";
             b) filename "/lts/vmlinuz-2.4.16-om5";
  47. Add the following line (all on one line) in dhcpd.conf (above the trick from Peter comment):
    option host-name = concat( "ws" , binary-to-ascii( 10, 8, "", substring( reverse( 1, leased-address), 0, 1)));
  48. F10 to save & exit
  49. > service dhcpd restart
  50. Copy & edit other files as indicated in Richard's How-To "Mosix Cluster with Diskless Nodes". Specifically, we are concerned with the following:
  51. Create & save the mosix.map file:
  52. > cd /etc
  53. > mcedit mosix.map
  54. Enter the following text for mosix.map:
    1 1
    2 253
  55. F10 to Save & Exit
  56. > cp /etc/mosix.map /opt/ltsp/i386/etc/ (Note: exact copies of mosix.map should be maintained in both locations.)
  57. Copy the userland tools & other files for the LTSP clients:
  58. > cp /sbin/setpe /opt/ltsp/i386/sbin/
  59. > cp /bin/mosrun /opt/ltsp/i386/bin/
  60. > cp /usr/bin/mosmon /opt/ltsp/i386/usr/bin/
  61. > cp /usr/bin/mosctl /opt/ltsp/i386/usr/bin/
  62. > cp /usr/bin/migrate /opt/ltsp/i386/usr/bin/
  63. ***NOTE***
    See Appendix A below for a list of other Userland tools that you may wish to copy for Linux Terminal Client node use. Simply copy them from their existing locations to the corresponding /opt/ltsp/i386/ locations as you just did for the previous files. You may need to make some or all of them available to the Linux Terminal nodes to effectively utilize the cluster.

  64. > cp /bin/touch /opt/ltsp/i386/bin/
  65. > rm /opt/ltsp/i386/etc/hosts
  66. > cp /etc/hosts /opt/ltsp/i386/etc/
  67. > cp /etc/rc.d/init.d/openmosix /opt/ltsp/i386/etc/rc.openmosix
  68. > mkdir /opt/ltsp/i386/mfs
  69. > cd /opt/ltsp/i386/etc/
  70. > mcedit fstab
  71. Add the following line to fstab:
    none /mfs mfs dfsa=1 0 0
  72. F10 to Save & Exit
  73. > mcedit /opt/ltsp/i386/etc/rc.local
  74. Add the following lines to the end of rc.local:
    # OpenMosix startup section
    # we don't want any terminal processes to migrate
    echo 1 > /proc/mosix/admin/lstay
    # start mosix
    /etc/rc.openmosix start
    # mount mfs filesystem. doesn't work when done earlier
    mount /mfs
    # End OpenMosix startup section
  75. F10 to Save and exit
  76. Once completed, and you verify that everything boots as it should, use the command > mosmon -t to check your work, hopefully you will see all of your OpenMosix nodes in action!

Some other quick notes once you have everything running properly and your experienced with K12LTSP:

  1. If you don't need a GUI on your LT/OpenMosix nodes edit the dhcpd.conf & lts.conf files to set the system(s) to "RUNLEVEL = 3".
  2. The MOSIX site lists a couple of ways to test your cluster with awk that will work just as well with OpenMosix. See the following:

Well, that's about it. I hope that this was helpful & useful. This document (Rev. 3.1) corrects errors found in the previous version. I ran through a complete K12LTSP+OpenMosix installation twice with this version of the document to verify it's accuracy. Thanks to B. Smith for his feedback. This (now) gives us a complete step-by-step How-To for easily implementing OpenMosix on a K12LTSP server. Richard Camp is in the process of producing more complete documentation, scripts & other items that he will make available at his website when ready.

Thanks, and happy clustering!

James Jensen

Rev. 3.2
Thursday, April 18, 2002

Appendix A

OpenMosix Userland Tools Chart

Summary of each file is based on it's manpage entry.

Userland File Summary
/bin/mosrun Run a command with particular node-allocation preferences.
/bin/mps Report multicomputer process status.
/etc/mosix.map OpenMosix node map file.
/etc/rc.d/init.d/openmosix OpenMosix init script.
/sbin/setpe MOSIX node configuration.
/usr/bin/cpujob mosrun frontend script: Runs a command advising MOSIX that it is a computation intensive program.
/usr/bin/fastdecay mosrun frontend script: runs a command advising MOSIX to apply fast statistics-decaying.
/usr/bin/iojob mosrun frontend script: runs a command, advising MOSIX that it is an I/O intensive program.
/usr/bin/migrate Request migration of a particular process on MOSIX.
/usr/bin/mosctl MOSIX system administrator's tools.
/usr/bin/mosmon MOSIX load monitor
/usr/bin/mtop Display multicomputer top CPU processes.
/usr/bin/nodecay mosrun frontend script: runs a command, advising MOSIX to not decay the program's statistics.
/usr/bin/nomig mosrun frontend script: runs a command with a node-lock.
/usr/bin/runhome Runs a command locked at the home node.
/usr/bin/runon mosrun frontend script: runs a command locked on a particular MOSIX node.
/usr/bin/slowdecay mosrun frontend script: runs a command, advising MOSIX to apply slow statistics-decaying.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]