NetBSD, Xen PVM, PVGrub and Ganeti

Booting a NetBSD domU with an in-tree kernel using PVGrub, in a Ganeti cluster

PVGrub ?

PVGrub is a bootloader for Xen ParaVirtualized domains that permits to boot securely a kernel that is stored inside the container ("in tree", whereas the kernel in commonly stored on the dom0, "out of tree").
PVGrub is an alternative to pygrub, which is supposedly insecure (or, less secure than pvgrub).

PVGrub is not distributed by Debian. You can compile it by fetching the Xen sources (can be apt-get source xen) and running

make stubdom DESTDIR=$(PWD)/destdir
cp destdir/usr/lib/xen/boot/pv-grub-x86_* /somewhere/

Ganeti and PVGrub

Instances run in a Ganeti cluster can be configured to use PVGrub.
This isn't easy to find how to do so in the official documentation though.

gnt-cluster modify -H xen-pvm:kernel_args='(hd0)/pvgrub.cfg'
gnt-cluster modify -H xen-pvm:kernel_path='/usr/lib/xen/boot/pv-grub-x86_64.gz'
gnt-cluster modify -H xen-pvm:initrd_path=''
gnt-cluster modify -H xen-pvm:bootloader_path=''
gnt-cluster modify -H xen-pvm:bootloader_args=''
gnt-cluster modify -H xen-pvm:root_path=''

NetBSD and PVGrub

The slice where PVGrub should look for the kernel seems to need to meet these requirements:

Sample pvgrub.cfg for NetBSD:

default         0
timeout         1

title NetBSD 6.0
  root (hd0)
  kernel /netbsd root=xbd0

Example of a NetBSD instance creation with the help of an existing NetBSD domain

Coquine is an existing instance running NetBSD. Coquinei386 will be created and configured to boot with pvgrub.

cluster# gnt-instance add -t plain -B memory=1G,vcpus=8 -o debootstrap+default --no-install -s 40G -n headmaster coquinei386
cluster# gnt-instance activate-disks coquinei386 | awk -F: '{ print $3}'
/dev/xenvg/ce77cb71-ce17-4a9c-a0e5-eeed22bb6929.disk0
cluster# xm block-attach coquine phy:/dev/xenvg/ce77cb71-ce17-4a9c-a0e5-eeed22bb6929.disk0 /dev/xvdb w
coquine# disklabel -I -e xbd1
(change fsize and bsize. For example:)

4 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:  83886080         0     4.2BSD   2048 16384     0  # (Cyl.      0 -  40959)
 d:  83886080         0     unused      0     0        # (Cyl.      0 -  40959)

coquine# newfs -O2 xbd1a
coquine# mount /dev/xbd1a /mnt
coquine# for f in /path/to/sets/*tgz; do tar xzpf $f -C /mnt/; done
coquine# cat > /mnt/etc/fstab <<EOF                                                   
/dev/xbd0a      /       ffs     rw,log      0       0    
EOF
coquine# cat > /mnt/pvgrub.cfg <<EOF
default         0
timeout         1

title NetBSD 6.0
  root (hd0)
  kernel /netbsd root=xbd0
EOF
coquine# umount /mnt
cluster# xm block-detach coquine 51728
cluster# gnt-instance start coquinei386