[Linux-cluster] AoE does not discover

[Bowie Bailey]

Tracy R Reed wrote:
> Hello all!
> 
> Anyone here use AoE? I cannot seem to find an AoE specific mailing
> list. Not sure I want to bother the Coraid folks since I am not using
> Coraid equipment. Is there a better place for this query? I am so
> close to getting my cluster mounted with root (and then Xen domains)
> on AoE but my initrd is having problems getting AoE set up. I am
> using kernel vmlinuz-2.6.16.13-xen.
> 
> AoE is compiled in statically.
> 
> /sys/module/aoe/parameters/ contains a file called aoe_iflist
> 
> AoE is mentioned in the dmesg when it boots.
> 
> /sys/module/aoe/parameters/aoe_iflist contains the active
> ethernet interface. If I echo interface names into /dev/aoe/interfaces
> they appear in /sys/module/aoe/parameters/aoe_iflist so I know aoe is
> loaded and doing something.
> 
> But when I do echo 1 > /dev/aoe/discover nothing happens.
> 
> Nothing in the dmesg etc. Not sure where to go from here.
> I have vblade running on another machine which is directly connected
> to the diskless cpu node. No hub or switch involved, just cross
> connected ethernet. If I run tethereal on the vblade machine looking
> for some sort of AoE related traffic I see nothing coming across the
> wire. I do see dhcp requests coming from that interface when the box
> boots due to PXE trying to find a server on that interface etc. So I
> know the physical ethernet link is good.
> 
> There really isn't much written about AoE. It's so simple that it
> seems like there shouldn't be much to write about.

I had a similar problem when I first set up my cluster, but I don't
remember the exact fix.  Reading over my config notes, there are only
two AoE specific things mentioned.

1)  Modify /etc/lvm/lvm.conf for AoE drives
    Add the line:
        types = [ "aoe", 16 ]

2)  Start services:
    modprobe aoe

And the startup section of my init file for the AoE services looks
like this:

    start()
    {
        echo -n "Starting aoe:"
        ifconfig eth0 up
        sleep 3         # network interface comes up

        modprobe aoe
        sleep 3         # time for AoE discovery and udev

        success
        echo
        return 0
    }

Hopefully some of this is helpful.

-- 
Bowie