Skip to the note
zecrazytux.net sysadmin  |  note index

~ / sysadmin / netbsd / ganeti pvgrub

sysadmin

Booting a NetBSD guest with PVGrub on Ganeti

Sysadmin NetBSD Xen Ganeti

A note on getting NetBSD to boot as a paravirtualised guest on a Ganeti cluster, with the guest kernel living inside the guest rather than on the host.

Why a bootloader inside the guest

A paravirtualised guest normally has its kernel loaded by the host. Booting from inside the guest instead means each domain keeps its own kernel and can be upgraded independently. The older loaders that read the guest filesystem from the host have a poor security reputation; PVGrub is the tidier alternative.

Building PVGrub

The mainstream distribution may not ship this loader, so it is compiled from the hypervisor sources. The build produces a small gzip compressed boot image that the cluster can point at.

make stubdom DESTDIR="$PWD/destdir"
cp destdir/usr/lib/xen/boot/pv-grub-x86_64.gz /usr/lib/xen/boot/

Telling the cluster to use it

A cluster wide setting makes every later paravirtualised instance boot through the loader, taking its kernel from the guest itself.

gnt-cluster modify -H xen-pvm:kernel_path='/usr/lib/xen/boot/pv-grub-x86_64.gz'
gnt-cluster modify -H xen-pvm:kernel_args='(hd0)/pvgrub.cfg'
gnt-cluster modify -H xen-pvm:initrd_path=''
gnt-cluster modify -H xen-pvm:bootloader_path=''

The slice the loader expects

  • The partition holding the kernel should start at offset zero.
  • The disklabel needs explicit fragment and block sizes.
  • The filesystem should be the newer fast file system format.

A configuration file inside the guest

default 0
timeout 1

title NetBSD
  root (hd0)
  kernel /netbsd root=xbd0

Creating an instance by hand

One practical route is to attach the new instance's disk to an existing NetBSD domain, partition and populate it there, write the boot configuration, detach it, and only then start the new instance. This keeps the tooling on a system that is already understood.

# on the cluster head
gnt-instance add -t plain -B memory=1G,vcpus=4 --no-install -s 40G -n node1 guest1
# attach the new disk to an existing guest, partition it, and copy the sets
newfs -O 2 xbd1a
mount /dev/xbd1a /mnt
# write /mnt/etc/fstab and /mnt/pvgrub.cfg, then umount and detach
gnt-instance start guest1

References

Background on the hypervisor and the cluster manager is listed on the resources page.