Booting a NetBSD domU with an in-tree kernel using PVGrub, in a Ganeti cluster
PVGrub ?
PVGrub is a bootloader for Xen ParaVirtualized domains that permits to boot securely a kernel that is stored inside the container ("in tree", whereas the kernel in commonly stored on the dom0, "out of tree").
PVGrub is an alternative to pygrub, which is supposedly insecure (or, less secure than pvgrub).
PVGrub is not distributed by Debian. You can compile it by fetching the Xen sources (can be apt-get source xen) and running
make stubdom DESTDIR=$(PWD)/destdir cp destdir/usr/lib/xen/boot/pv-grub-x86_* /somewhere/
Ganeti and PVGrub
Instances run in a Ganeti cluster can be configured to use PVGrub.
This isn't easy to find how to do so in the official documentation though.
gnt-cluster modify -H xen-pvm:kernel_args='(hd0)/pvgrub.cfg' gnt-cluster modify -H xen-pvm:kernel_path='/usr/lib/xen/boot/pv-grub-x86_64.gz' gnt-cluster modify -H xen-pvm:initrd_path='' gnt-cluster modify -H xen-pvm:bootloader_path='' gnt-cluster modify -H xen-pvm:bootloader_args='' gnt-cluster modify -H xen-pvm:root_path=''
NetBSD and PVGrub
The slice where PVGrub should look for the kernel seems to need to meet these requirements:
- start at 0
- fsize and bsize should be specified in the disklabel
- filesystem should be FFSv2 (newfs -O2)
Sample pvgrub.cfg for NetBSD:
default 0 timeout 1 title NetBSD 6.0 root (hd0) kernel /netbsd root=xbd0
Example of a NetBSD instance creation with the help of an existing NetBSD domain
Coquine is an existing instance running NetBSD. Coquinei386 will be created and configured to boot with pvgrub.
cluster# gnt-instance add -t plain -B memory=1G,vcpus=8 -o debootstrap+default --no-install -s 40G -n headmaster coquinei386 cluster# gnt-instance activate-disks coquinei386 | awk -F: '{ print $3}' /dev/xenvg/ce77cb71-ce17-4a9c-a0e5-eeed22bb6929.disk0 cluster# xm block-attach coquine phy:/dev/xenvg/ce77cb71-ce17-4a9c-a0e5-eeed22bb6929.disk0 /dev/xvdb w coquine# disklabel -I -e xbd1 (change fsize and bsize. For example:) 4 partitions: # size offset fstype [fsize bsize cpg/sgs] a: 83886080 0 4.2BSD 2048 16384 0 # (Cyl. 0 - 40959) d: 83886080 0 unused 0 0 # (Cyl. 0 - 40959) coquine# newfs -O2 xbd1a coquine# mount /dev/xbd1a /mnt coquine# for f in /path/to/sets/*tgz; do tar xzpf $f -C /mnt/; done coquine# cat > /mnt/etc/fstab <<EOF /dev/xbd0a / ffs rw,log 0 0 EOF coquine# cat > /mnt/pvgrub.cfg <<EOF default 0 timeout 1 title NetBSD 6.0 root (hd0) kernel /netbsd root=xbd0 EOF coquine# umount /mnt cluster# xm block-detach coquine 51728 cluster# gnt-instance start coquinei386