--- title: "Dual-Boot NetBSD And Archlinux" date: 2024-07-18T20:32:52+02:00 slug: 2024-07-18-uefi-netbsd-archlinux-dual-boot type: posts draft: false summary: | This is the method I use to dual-boot Archlinux and NetBSD in my laptop. categories: - NetBSD - Linux tags: - NetBSD - Linux - UEFI - encryption - installation --- This is the method I use to dual-boot Archlinux and NetBSD in my laptop. I've written this with a virtual machine, to reproduce and test every step. But, at least in theory, it should work in any UEFI AMD64 machine. # Motivation I've been forcing myself to use NetBSD. When you are familiar with a tool it's very hard to get the work done with a different one. Working with a new and unfamiliar tool makes you slower, feels less ergonomic and frustrating. This is why people keep stuck with proprietary bloatware like Adobe Photoshop and alike. Not because the lack of open/libre alternatives, but because these alternatives are different and unfamiliar. So, to learn a tool, you have to force yourself to use it. In your spare time, if your profession is merciless with mistakes and delays. In your working time, if your profession allows it. Working with a new tool is painful and frustrating. So certain discipline is required. But once the initial pain goes away, stuff becomes easier with time. And the new tool becomes a little bit more familiar every day. Then, you can judge this tools by its own merits. With experience you can tell where this tool is stronger, and where is weaker. With NetBSD I can use [Darktable](https://www.darktable.org/) to edit and process my photos. Basic stuff like reading my e-mail and browse the web. I can use my favorite [password manager](https://keepassxc.org/) and [synchronize](https://syncthing.net/) the keyring with my phone. But I can't use my Wacom tablet in NetBSD. While, I can do some photo editing with it, NetBSD is not really meant to do artwork with it. Probably, NetBSD has been heavily tested with server workloads, but not so much with desktop ones. And hardware support is not comparable with Linux. So, If you want to draw stuff with your computer, I wouldn't recommend NetBSD. So, in order to keep using NetBSD and have the ability to fallback to a more familiar OS when needed, I've decided to dual-boot Archlinux and NetBSD. This is a tutorial that I've written to myself to do this. # Arch Linux Installation This blog post have been written with a virtual machine to keep track of the steps and test them. In this kind of environment, is better to use the serial port for terminal, rather than the virtual screen. Once the arch installer image boots, you may use the serial port. ``` # systemctl start serial-getty@ttyS0.service ``` This tutorial assumes two drives in a machine, which is the setup of my laptop. It could be done with a single drive though. Let's see what disks we have available: ``` # fdisk -l Disk /dev/vda: 32 GiB, 34359738368 bytes, 67108864 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/vdb: 20 GiB, 21474836480 bytes, 41943040 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/loop0: 795.73 MiB, 834379776 bytes, 1629648 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes ``` Here I have two drives with 32GiB and 20GiB. The bigger one will have the EFI partition and Archlinux. The smaller one will have NetBSD installed. The EFI partition is usually the first partition of the first drive. It's a FAT partition that contains binaries that loads your OS. It may contain configuration and data files too. In this partition we are going to put the boot-loaders for Archlinux and NetBSD. Also, here we are going to install [rEFInd](https://www.rodsbooks.com/refind/), a tool that helps us to manage and boot different OSes. In theory, we could use the boot menu from the machine firmware. Or GRUB, or something similar. So rEFInd is not mandatory, but it will give us a nicer graphical boot menu to select what OS we want to use. To proceed with Archlinux installation we may need to connect to WiFi. Remember, these are the steps for my laptop. Your setup could be different. ``` # iwctl station wlan0 scan # iwctl station wlan0 get-networks # iwctl --passphrase passphrase station wlan0 connect SSID ``` Now we create the partitions for the first drive. ``` # fdisk /dev/vda ``` Use `g` to create new GPT partition. Use `n` to create a new partition. `t` and `1` to change a partition to EFI type. And `w` to save and exit. New partitions should look like this. ``` # fdisk -l /dev/vda Disk /dev/vda: 32 GiB, 34359738368 bytes, 67108864 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 88E94B49-29C5-4B0D-B2D4-E9EB94CC52C7 Device Start End Sectors Size Type /dev/vda1 2048 8390655 8388608 4G EFI System /dev/vda2 8390656 16779263 8388608 4G Linux filesystem /dev/vda3 16779264 67106815 50327552 24G Linux filesystem ``` In this case I've created an EFI partition of 4GiB. A Linux `/boot` partition of 4GiB. And 24GiB for `/` and other filesystems of Archlinux. 4GiB is a lot of space for an EFI partition. You shouldn't need more than that. Now we create FAT filesystems for the `/boot` and EFI partitions. ``` # mkfs.vfat -n EFI /dev/vda1 mkfs.fat 4.2 (2021-01-31) # mkfs.vfat -n BOOT /dev/vda2 mkfs.fat 4.2 (2021-01-31) # blkid /dev/vda{1,2} /dev/vda1: LABEL_FATBOOT="EFI" LABEL="EFI" UUID="0982-62D9" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="5ca2bc8a-7624-4b82-8097-b887256ece2d" /dev/vda2: LABEL_FATBOOT="BOOT" LABEL="BOOT" UUID="0A17-5347" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="25e6e4e2-b4eb-4741-b998-b9d9c0a8bc5c" ``` Then we encrypt the Linux partition. In this setup we will use LVM on top of LUKS. LUKS will encrypt the partition. And LVM will make logical volumes out of that partition. ``` # cryptsetup luksFormat /dev/vda3 # cryptsetup open /dev/vda3 cryptlvm ``` Usually, I use 4 random words [a la diceware](https://diceware.dmuth.org/) for the passphrase. That's not a very strong passphrase. But I believe it's good enough for off-line stuff, like a encrypted drive. For internet stuff, where malicious actors do have access to credentials sometimes, I use a password manager. And 24 to 32 random characters as the passwords. With 2FA when possible. But for the drive encryption, I don't recommend a password too hard to remember. And, unlike some advice on The Internet, I do recommend writing the passphrase in a piece of paper, and store it with other important documents. If you are not a state-level target, that's good enough. If you need a strong password, 7 diceware words should be enough for everybody. Now, we create the LVM group with the logical volumes. ``` # pvcreate /dev/mapper/cryptlvm # vgcreate arch /dev/mapper/cryptlvm # lvcreate -L 4G arch -n swap # lvcreate -L 16G arch -n root # lvcreate -l 100%FREE arch -n home ``` Then, we create the filesystems for it. ``` # mkfs.ext4 -L ROOT /dev/arch/root # mkfs.ext4 -L HOME /dev/arch/home # mkswap /dev/arch/swap # swapon /dev/arch/swap ``` Then, mount everything under new rootfs. As the [Archlinux installation guide](https://wiki.archlinux.org/title/Installation_guide#) suggest, we are going to use `/mnt` as the new root filesystem. ``` # mount /dev/arch/root /mnt # mkdir -p /mnt/home # mkdir -p /mnt/boot # mount /dev/arch/home /mnt/home # mount /dev/vda2 /mnt/boot # mkdir -p /mnt/boot/efi # mount /dev/vda1 /mnt/boot/efi ``` Here, we are mounting the EFI filesystem in `/boot/efi`, so it can be manipulated from Archlinux. This is not mandatory. But I usually do it, so I can edit, backup, delete stuff easily. Now, we install the system. ``` # pacstrap -K /mnt base base-devel linux linux-firmware iwd dhcpcd vim htop tmux grub efibootmgr os-prober lvm2 ``` A lot of these packages are not mandatory. This is what a new system for me usually looks like. You could replace `grub` for other bootloader, since we are using rEFInd and we are not gonna use a lot of GRUB features. At minimum `base`, `linux` and `lvm2` are necessary for this setup. Now, let's create a new `fstab` for our system. ``` # genfstab -U /mnt >> /mnt/etc/fstab ``` The last configs for our new system will require to chroot into the new system. Archlinux have a convenient script called `arch-chroot` that do other stuff, like mounting special filesystem into the new root. ``` # arch-chroot /mnt # ln -sf /usr/share/zoneinfo/Europe/Madrid /etc/localtime # hwclock --systohc # vim /etc/locale.gen # Uncomment your locale # locale-gen # echo LANG=en_US.UTF-8 >> /etc/locale.conf # echo arch > /etc/hostname # vim /etc/mkinitcpio.conf # Add hooks for LUKS and lvm2 ``` The last part is quite important. If you do it wrong, your new system won't boot This is how it should look like. ``` # grep ^HOOKS /etc/mkinitcpio.conf HOOKS=(base udev autodetect microcode modconf kms keyboard keymap consolefont block encrypt lvm2 filesystems fsck) ``` Now, we generate the new ramfs, and change the root user password. ``` # mkinitcpio -P # passwd ``` Then, we install and configure the bootloader. As I said, you may want to use a smaller, leaner bootloader. We are using rEFInd, so our bootloader doesn't need to chain-load other bootloaders. ``` # grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=GRUB # blkid /dev/vda3 # Get the UUID of the encrypted drive /dev/vda3: UUID="83e10b9c-2420-4b23-b8a5-3e0a09749f52" TYPE="crypto_LUKS" PARTUUID="5342f00d-3827-4ca1-abb8-96f0660267c7" # vim /etc/default/grub # Add the kernel parameter to the encrypted drive # grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 cryptdevice=UUID=83e10b9c-2420-4b23-b8a5-3e0a09749f52:cryptlvm root=/dev/arch/root" ``` We have to add a kernel parameter to decrypt the root filesystem. So your kernel parameters should look like this. ``` # grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 cryptdevice=UUID=83e10b9c-2420-4b23-b8a5-3e0a09749f52:cryptlvm root=/dev/arch/root" ``` Of course, you replace my partition UUID with yours. You print that with `blkid`. If it's good, we proceed to make the GRUB configuration. ``` # grub-mkconfig -o /boot/grub/grub.cfg ``` At this point, you can reboot and use your new Archlinux system. No graphical user interface, of course. That's outside the scope of this guide. I usually install Mate or i3wm. Now, the fun part. We need NetBSD in this computer too. # Install NetBSD I've already written a guide to install [NetBSD with encrypted partitions](https://vsis.online/posts/2024-05-27-uefi-full-disk-encryption/). This will be almost the same, with a few differences for EFI configuration. The first thing: we need to know the available drives. ``` # sysctl hw.disknames # dkctl ld0 # dkctl ld1 ``` Of course, it puts different names for the drives. `ld0` and `ld1` are the name of my virtual drives in this machine. Yours may be different. Keep in mind those differences in names. We need to make sure we are formatting and writing in the right drive. ``` # dkctl ld0 listwedges # dkctl ld1 listwedges ``` Once you know, what drive is the right one, we need to create new partitions for it. Assuming `ld1` is the right one. ``` # gpt destroy ld1 # gpt create ld1 # gpt add -a 2m -l NetBSD -t ffs -s 8g ld1 # gpt add -a 2m -t cgd -l syscgd ld1 ``` Here stuff may get complicated. Since we have created new partitions, the numbers of `dk*` will change. So you have to list the wedges again and make sure that you are going to format the right ones. ``` # dkctl ld0 listwedges /dev/rld0: 3 wedges: dk2: 5ca2bc8a-7624-4b82-8097-b887256ece2d, 8388608 blocks at 2048, type: msdos dk3: 25e6e4e2-b4eb-4741-b998-b9d9c0a8bc5c, 8388608 blocks at 8390656, type: ext2 fs dk4: 5342f00d-3827-4ca1-abb8-96f0660267c7, 50327552 blocks at 16779264, type: ex t2fs # dkctl ld1 listwedges /dev/rld1: 2 wedges: dk0: NetBSD, 33554432 blocks at 4096, type: ffs dk1: syscgd, 8380416 blocks at 33558528, type: cgd ``` See? `dk0` is now the NetBSD root filesystem. And `dk1` will be the NetBSD encrypted drive with protected filesystems. Now, let's add an EFI entry for NetBSD. ``` # mount /dev/dk2 /mnt # mkdir -p /mnt/EFI/NetBSD/ # cp -v /usr/mdec/*.efi /mnt/EFI/NetBSD /usr/mdec/bootia32.efi -> /mnt/EFI/boot/bootia32.efi /usr/mdec/bootx64.efi -> /mnt/EFI/boot/bootx64.efi ``` And we will create an EFI config file for NetBSD ``` vi /mnt/boot.cfg ``` ``` menu=Boot normally:root NAME=NetBSD;boot menu=Boot single user:root NAME=NetBSD;boo -s menu=Disable ACPI:root NAME=NetBSD;boot -2 menu=Disable ACPI and SMP:root NAME=NetBSD;boot -12 menu=Drop to boot prompt:prompt default=1 timeout=5 clear=1 ``` The important part is `root NAME=NetBSD`. We are telling the NetBSD bootloader that the root filesystem is in a partition labeled `NetBSD`. Now we format the partition and mount it under `/targetroot`. ``` # newfs -O 2 dk0 # mount /dev/dk0 /targetroot ``` Then, proceed with the new encrypted CGD device. ``` # mkdir -p /targetroot/etc/cgd/ # cgdconfig -g -V disklabel -o /targetroot/etc/cgd/syscgd aes-xts 512 # cgdconfig -V re-enter cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd # disklabel -Ii cgd0 # echo 'cgd0 NAME=syscgd /etc/cgd/syscgd' > /targetroot/etc/cgd/cgd.conf ``` These steps are better explained in [my last blog entry](https://vsis.online/posts/2024-05-27-uefi-full-disk-encryption/), and [the documentation](https://www.netbsd.org/docs/guide/en/chap-cgd.html). After the `disklabel -Ii cgd0` part you should have the partitions for the protected CGD device. I usually use `cgd0a` for `/var`, `cgd0b` for swap, `cgd0e` for `/usr`, and `cgd0f` for `/home`. Let's test our new CGD device. Unconfigure (close) the CGD device, then configure it again. Then, print the disklabel. ``` # cgdconfig -u cgd0 # cgdconfig cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd # disklabel cgd0 ``` If everything is OK, we create and mount our new filesystems under `/targetroot`. ``` # newfs -O 2 cgd0a # newfs -O 2 cgd0e # newfs -O 2 cgd0f # mkdir /targetroot/var /targetroot/usr /targetroot/home # mount /dev/cgd0a /targetroot/var # mount /dev/cgd0e /targetroot/usr # mount /dev/cgd0f /targetroot/home ``` With the partitions mounted, we can extract the binary sets. ``` # cd /amd64/binary/sets # tar xvzpf base.tar.xz -C /targetroot # tar xvzpf comp.tar.xz -C /targetroot # tar xvzpf etc.tar.xz -C /targetroot # tar xvzpf games.tar.xz -C /targetroot # tar xvzpf gpufw.tar.xz -C /targetroot # tar xvzpf kern-GENERIC.tar.xz -C /targetroot # tar xvzpf man.tar.xz -C /targetroot # tar xvzpf misc.tar.xz -C /targetroot # tar xvzpf modules.tar.xz -C /targetroot # tar xvzpf text.tar.xz -C /targetroot # tar xvzpf xbase.tar.xz -C /targetroot # tar xvzpf xcomp.tar.xz -C /targetroot # tar xvzpf xetc.tar.xz -C /targetroot # tar xvzpf xfont.tar.xz -C /targetroot # tar xvzpf xserver.tar.xz -C /targetroot # cd / ``` Then, chroot to new system and make the devices in `/dev`. ``` # chroot /targetroot # cd dev # ./MAKEDEV all # exit ``` Now, we edit fstab. ``` # vi /targetroot/etc/fstab ``` ``` # NetBSD /etc/fstab # See /usr/share/examples/fstab/ for more examples. NAME=NetBSD / ffs rw 1 1 kernfs /kern kernfs rw ptyfs /dev/pts ptyfs rw procfs /proc procfs rw /dev/cd0a /cdrom cd9660 ro,noauto tmpfs /var/shm tmpfs rw,-m1777,-sram%25 # Encrypted file-systems /dev/cgd0a /var ffs rw 1 2 /dev/cgd0b none swap sw /dev/cgd0e /usr ffs rw 1 2 /dev/cgd0f /home ffs rw 1 2 ``` Then we edit `rc.conf`. ``` # vi /targetroot/etc/rc.conf ``` ``` rc_configured=YES # Add local overrides below. dhcpcd=YES dhcpcd_flags="-qM wm0" hostname=ceres.local sshd=YES wscons=YES cgd=YES ``` Then we add the kernel to EFI partition. You see, this part may not be needed. Maybe because of UEFI firmware issues, the second drive was not available always in bootloader runtime. So, [I couldn't read the kernel from the second drive](https://mastodon.bsd.cafe/deck/@release_candidate/112899481162302628). I added the kernel to the EFI drive and, if you recall it, we told the bootloader to find root filesystem in `root NAME=NetBSD`. ``` cp -v /targetroot/netbsd /mnt ``` Now, let's umount the new system. ``` # umount /targetroot/home # umount /targetroot/var # umount /targetroot/usr # umount /targetroot/ # umount /mnt/ ``` Then shutdown or reboot: ``` # shutdown -p now ``` # Install rEFInd At this point, you have Archlinux and NetBSD installed in your system. you could add a new `menuentry` in GRUB to access NetBSD. Or you could use the boot menu from your machine firmware. I will use rEFInd to have a nice menu where I can select the OS. Let's boot in Archlinux and install rEFInd. ``` # pacman -S refind # refind-install ``` Then, reboot. # Conclusion I'm still working on teaching myself NetBSD. Linux ecosystem have become increasingly more bloated and bigger over time. And I'm sure that there are some scenarios where a simpler compact system would suit better. But I'm too familiar with Linux, so I have to go outside my comfort zone. NetBSD is an amazing OS. Simple, consistent, documented and straightforward. But I have to use it more, so it becomes familiar to me. Dual-boot is a good way to force yourself to use a different OS, with the fallback in case you need your familiar one. That's why a lot of people dual-boot Windows and some Linux distro.