install debian directly onto an AoE root filesystem

Something that just about no one out there seems to be doing (yet) is trying to install Debian directly onto network block devices. The Debian installer doesnt support it (yet), grub doesnt support it (usually), and its just generally not an easy thing to do.

Now, there are quite a few ways around this problem. You can install to a ‘real’ computer and migrate the installation to a network block device. You can use debootstrap in place of the actual Debian installation system. You can use a combination of these two methods, NFS root filesystems, TFTP hacks, etc. All of these solutions are lacking in my opinion. I want to run the ‘real’ debian installer against a network block device and boot my physical hardware using only the built in PXE booting capability of the BIOS.

Taking all these issues as a personal challenge, I’ve outlined below how to go about using the regular old Debian Lenny installer directly against an AoE block device.

First of all, youll need the full installer disc 1 for Lenny. We need one of the .deb’s thats on that CD. The netinst or businesscard installers will not work with the steps below. (NOTE: If you want to be really creative, I suppose you could download the .deb from within the installer system via wget or even get it before hand and put it on a thumb drive or something…)

1. Boot the lenny install cd just to the point that /cdrom is mounted. For me, that was the “Configure the network” prompt.

2. Switch to a virtual terminal by hitting alt-f2.

3. Unpack the .deb that contains the aoe kernel module

udpkg --unpack /cdrom/pool/main/l/linux-2.6/linux-image-2.6.26-2-486_2.6.26-15_i386.deb

4. Now load the kernel module we just unpacked

insmod /lib/modules/2.6.26-2-486/kernel/drivers/block/aoe/aoe.ko

5. Now we create a symlink to the AoE block device. Its important that we use a symlink at this point in the process since it will cause the installer to set up a lot of paths in the correct fashion.

ln -s /dev/etherd/eX.X /dev/sda

6. Hit alt-f1 and continue with installation until the point that grub fails to install, then switch back to terminal 2.

7. Now chroot into the mostly installed system

chroot /target

8. Install the aoetools package

apt-get -y install aoetools

9. Listed below are two scripts that youll need to get into your chroot environment. They are used to build a new custom initrd image. I find it easiest to put them on a webserver somewhere and wget them.

wget http://www.pixelchaos.net/stuff/aoe/hooks/aoetools -O /etc/initramfs-tools/hooks/aoetools
chmod 755 /etc/initramfs-tools/hooks/aoetools
wget http://www.pixelchaos.net/stuff/aoe/scripts/local-top/aoetools -O /etc/initramfs-tools/scripts/local-top/aoetools
chmod 755 /etc/initramfs-tools/scripts/local-top/aoetools

10. Now we regenerate the initrd image

update-initramfs -u -k all

11. Now that all the above is done, we can manually install grub – which the debian installer was unable to do.

apt-get -y install grub
mkdir /boot/grub
cp -r /usr/lib/grub/i386-pc/. /boot/grub

12. Now we can remove our old symlink and create some hard links instead. This is required to allow grub to modify the MBR of our block device.

rm /dev/sda
ln /dev/etherd/eX.X /dev/sda
ln /dev/etherd/eX.Xp1 /dev/sda1
ln /dev/etherd/eX.Xp5 /dev/sda5

13. Now update grubs device.map

echo "(hd0) /dev/sda" > /boot/grub/device.map

14. Now run “grub” and issue the following commands

device (hd0) /dev/sda
root (hd0,0)
setup (hd0)

15. Now issue the “update-grub” command which will generate /boot/grub/menu.lst for the first time

16. Remember that grub has been using the hard link to /dev/sda this whole time. Change the root=/dev/sda1 line in /boot/grub/menu.lst to root=/dev/etherd/eX.Xp1, then issue the “update-grub” command again to commit the change to the boot stanzas.

17. For the sake of completeness, lets change grubs device.map back to its original contents.

echo "(hd0) /dev/etherd/eX.X" > /boot/grub/device.map

18. Get out of the chroot environment by issuing the “exit” command.

19. Go back to installer and choose “Continue without boot loader” to finish the installation and reboot the computer.

Thats it! Assuming you have properly set up a gPXE boot environment everything should “just work”. Setting up gPXE is outside the scope of this document but some good notes are available on the etherboot web site (see the links below).

Please note that there is one caveat to this whole process. Grub does not support installation onto AoE block devices. There very well may be some issues the next time you try to upgrade your kernel if some post-inst script calls grub-install against your root device. Keep that in mind.

aoe-hooks script

#!/bin/sh

set  -e

PREREQ=""

prereqs()
{
echo "$PREREQ"
}

case $1 in
prereqs)
prereqs
exit 0
;;
esac

. /usr/share/initramfs-tools/hook-functions

[ -x /sbin/aoe-discover ] && copy_exec /sbin/aoe-discover /sbin
manual_add_modules aoe

aoe-local-top script

#!/bin/sh

set -e

PREREQ="udev"

prereqs()
{
echo "$PREREQ"
}

case $1 in
prereqs)
prereqs
exit 0
;;
esac

case $ROOT in
/dev/etherd/e*)
INTERFACES=`sed -ne '/eth.*:/{s/:.*$//;p;}' < /proc/net/dev`
for i in $INTERFACES; do
echo Bringing up interface $i for AoE
ifconfig $i up
done
# Make sure udev has processed all events from adding the NIC
# modules before loading aoe
[ -x /sbin/udevsettle ] && /sbin/udevsettle --timeout=30
modprobe aoe
# Wait until aoe device files have been generated.
[ -x /sbin/udevsettle ] && /sbin/udevsettle --timeout=30
aoe-discover
;;
esac

references
http://maht0x0r.blogspot.com/2009/03/installing-debian-5-for-migration-to.html
http://www.etherboot.org/wiki/sanboot/debian_and_ubuntu
http://mike.neir.org/wiki/articles/AoE_Root_Fedora

4 thoughts on “install debian directly onto an AoE root filesystem”

  1. I fixed the links in the above article… they were missing /stuff/ from the path.

    Thanks!

  2. Nice! I just set up an ubuntu 12.04 via AoE and stumbled upon this site. Everything works fine except 2nd script: “/sbin/udevsettle […]” has to be “/sbin/udevadm settle […]” now…

Leave a Reply

Your email address will not be published. Required fields are marked *