--- title: "NetBSD - UEFI installation with Full Disk Encryption" date: 2024-05-27T22:54:45+02:00 slug: 2024-05-27-uefi-full-disk-encryption type: posts draft: false summary: | This is the method I use to install a semi-full disk encrypted NetBSD system. I may add RAID devices, LVM, multiple disks, etc. Then mount everything under `/targetroot` and extract the sets. categories: - NetBSD tags: - NetBSD - UEFI - encryption - installation --- I've been trying to teach myself NetBSD. It has been a painful experience, full of bugs and kernel panics. I've post some of them in the Fediverse. Like [here](https://mastodon.bsd.cafe/@release_candidate/112520904317829098) and [here](https://mastodon.bsd.cafe/@release_candidate/112128737628556050). One of the weakest points that I've seen in NetBSD is the installer. If you need a simple installation *it just works* ™. But as soon as you need some complex setup, like RAID mixed with encrypted partitions, or something similar, the installer is subpar. You will face some segfault from the installer, a kernel panic or another surprise. I've forced myself to use NetBSD as a daily driver in my laptop. Since it's a laptop it needs full disk encryption. A mobile computer can be stolen at anytime, specially if you live in the [Lawlessness](https://en.wikipedia.org/wiki/Latin_America). So I had to make a "special way" to install NetBSD with encryption. Now, even today I have no idea how to have actual full disk encryption with NetBSD. [CGD devices](https://www.netbsd.org/docs/guide/en/chap-cgd.html), the virtual block devices that implements encryption, require metadata in `/etc/cgd`. I've seen [a wonderful tutorial for full-disk encryption for MBR-based systems](https://www.unitedbsd.com/d/461-netbsd-full-disk-encryption-with-cgd), but not for UEFI. And my laptop doesn't really like to boot in old MBR mode. Following the [documentation on CGD drives](https://www.netbsd.org/docs/guide/en/chap-cgd.html#chap-cgd-example) and [the documentation on UEFI installations](https://wiki.netbsd.org/Installation_on_UEFI_systems/), I have a semi-full disk encryption. With plain-text root file-system, and encrypted `/home`, `/usr`, `/var` and swap. So, without further complaints, this is the way I have some disk encryption in UEFI systems with NetBSD. # Boot the installer image I wrote this post with a virtual machine to reproduce all the steps. In such environment it may be useful to use the serial port as a terminal instead the virtual monitor. So, as soon as the boot options are presented, I press option `3`. Then, I instruct the boot-loader to use `com0` as the terminal. ``` consdev com0 boot ``` # Use a shell Once the installer system is up, I avoid the installer and use a shell. Select `Utility menu`, then `Run /bin/sh`. # Disks Now that I'm in a shell I can look what disks I have. ``` # sysctl hw.disknames hw.disknames = cd0 wd0 ``` In this example `wd0` is the drive where the system should be installed. Since I want to boot with UEFI, it needs GPT partitions. ``` # gpt destroy wd0 # gpt create wd0 # gpt add -a 2m -s 128m -t efi -l EFI wd0 # gpt add -a 2m -l NetBSD -t ffs -s 8g wd0 # gpt add -a 2m -t cgd -l syscgd wd0 ``` First, I destroy some pre-existing GPT partitions. This will destroy the drive data, so be careful. These commands create three GPT partitions. One EFI partition with 128MB, one for the rootfs with 8GB and other for CGD with the rest of the disk. These are the sizes I use. Root file-system will require less than 8GB anyways, but I avoid the risk. Now this disk should have three wedges. ``` # dkctl wd0 listwedges /dev/rwd0: 3 wedges: dk0: EFI, 262144 blocks at 4096, type: msdos dk1: NetBSD, 16777216 blocks at 266240, type: ffs dk2: syscgd, 24895488 blocks at 17043456, type: cgd ``` In this example, `/dev/dk0` is the EFI partition, `/dev/dk1` is the root partition, and `/dev/dk2` is the encrypted partition. You may also run: ``` # gpt show wd0 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4062 Unused 4096 262144 1 GPT part - EFI System 266240 16777216 2 GPT part - NetBSD FFSv1/FFSv2 17043456 24895488 3 GPT part - NetBSD Cryptographic Disk 41938944 4063 Unused 41943007 32 Sec GPT table 41943039 1 Sec GPT header ``` Now, I create a file-system for boot EFI partition. Its type is FAT. ``` # newfs_msdos /dev/rdk0 ``` Then, I mount it and copy the `.efi` boot entries. ``` # mount /dev/dk0 /mnt # mkdir -p /mnt/EFI/boot # cp -v /usr/mdec/*.efi /mnt/EFI/boot /usr/mdec/bootia32.efi -> /mnt/EFI/boot/bootia32.efi /usr/mdec/bootx64.efi -> /mnt/EFI/boot/bootx64.efi # umount /mnt ``` Then, I format NetBSD root partition. The `-O 2` flag means that it will use FFv2 file-system. ``` newfs -O 2 dk1 ``` Now I can mount the future root file-system. The installer uses `/targetroot` directory to install. So, I'll do the same. ``` # mount /dev/dk1 /targetroot ``` Next, I create CGD config file. And I place it in the future `/etc/cgd/` directory. ``` # mkdir -p /targetroot/etc/cgd/ # cgdconfig -g -V disklabel -o /targetroot/etc/cgd/syscgd aes-xts 512 ``` Now it's time to create the CGD drive. ``` # cgdconfig -V re-enter cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd ``` Here, the `-V re-enter` flag is needed to validate the encrypted drive without the disklabel, which doesn't exist yet. Once the CGD drive is configured it is possible to create the disklabel for it. In this example, I use `cgd0a` for `/var`, `cgd0b` for swap, `cgd0e` for `/usr`, and finally `cgd0f` for `/home`. Partitions `c` and `d` have a special meaning in NetBSD, so I'm not using them. Because this is a example virtual machine I use very humble values of 3 and 4GB. ``` # disklabel -Ii cgd0 Enter '?' for help partition>a Filesystem type [4.2BSD]: Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: 3G Partition size ('$' for all remaining) [12156c, 24895488s, 12156M]: ^C # disklabel -Ii cgd0 Enter '?' for help partition>a Filesystem type [4.2BSD]: Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: Partition size ('$' for all remaining) [12156c, 24895488s, 12156M]: 3G a: 6291456 0 4.2BSD 0 0 0 # (Cyl. 0 - 3071) partition>b Filesystem type [unused]: swap Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: a Partition size ('$' for all remaining) [0c, 0s, 0M]: 2G b: 4194304 6291456 swap # (Cyl. 3072 - 5119) partition>e Filesystem type [unused]: 4.2BSD Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: b Partition size ('$' for all remaining) [0c, 0s, 0M]: 3G e: 6291456 10485760 4.2BSD 0 0 0 # (Cyl. 5120 - 8191) partition>f Filesystem type [unused]: 4.2BSD Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: e Partition size ('$' for all remaining) [0c, 0s, 0M]: $ f: 8118272 16777216 4.2BSD 0 0 0 # (Cyl. 8192 - 12155) partition>W Label disk [n]?y Label written partition>Q ``` Let's add CGD config file to the future root file-system. ``` # echo 'cgd0 NAME=syscgd /etc/cgd/syscgd' > /targetroot/etc/cgd/cgd.conf ``` Now it's time to check if the CGD drive is working as expected. I unconfigure and reconfigure it, then print the disklabel. ``` # cgdconfig -u cgd0 # cgdconfig cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd # disklabel cgd0 ``` The new partitions need to be formatted and mounted in their places. ``` # newfs -O 2 cgd0a # newfs -O 2 cgd0e # newfs -O 2 cgd0f # mkdir /targetroot/var /targetroot/usr /targetroot/home # mount /dev/cgd0a /targetroot/var # mount /dev/cgd0e /targetroot/usr # mount /dev/cgd0f /targetroot/home ``` At this time, the system should look like this: ``` # mount root_device on / type cd9660 (read-only, local) tmpfs on /dev type tmpfs (union, local) tmpfs on /tmp type tmpfs (local) tmpfs on /var type tmpfs (local) tmpfs on /etc type tmpfs (union, local) /dev/dk1 on /targetroot type ffs (local) /dev/cgd0a on /targetroot/var type ffs (local) /dev/cgd0e on /targetroot/usr type ffs (local) /dev/cgd0f on /targetroot/home type ffs (local) ``` # Installation of the sets The new system is composed of sets. I usually install these ones, but your requirements may be different. The tar flag `p` is very important, since these files need to preserve their owners and mode. ``` # cd /amd64/binary/sets # tar xvzpf base.tar.xz -C /targetroot # tar xvzpf comp.tar.xz -C /targetroot # tar xvzpf etc.tar.xz -C /targetroot # tar xvzpf games.tar.xz -C /targetroot # tar xvzpf gpufw.tar.xz -C /targetroot # tar xvzpf kern-GENERIC.tar.xz -C /targetroot # tar xvzpf man.tar.xz -C /targetroot # tar xvzpf misc.tar.xz -C /targetroot # tar xvzpf modules.tar.xz -C /targetroot # tar xvzpf text.tar.xz -C /targetroot # tar xvzpf xbase.tar.xz -C /targetroot # tar xvzpf xcomp.tar.xz -C /targetroot # tar xvzpf xetc.tar.xz -C /targetroot # tar xvzpf xfont.tar.xz -C /targetroot # tar xvzpf xserver.tar.xz -C /targetroot # cd / ``` Then I chroot to the new system, and make the `dev`ices. ``` # chroot /targetroot # cd dev # ./MAKEDEV all exit ``` Now I edit fstab to mount the CGD partitions. ``` # vi /targetroot/etc/fstab ``` ``` # NetBSD /etc/fstab # See /usr/share/examples/fstab/ for more examples. NAME=NetBSD / ffs rw 1 1 kernfs /kern kernfs rw ptyfs /dev/pts ptyfs rw procfs /proc procfs rw /dev/cd0a /cdrom cd9660 ro,noauto tmpfs /var/shm tmpfs rw,-m1777,-sram%25 # Encrypted file-systems /dev/cgd0a /var ffs rw 1 2 /dev/cgd0b none swap sw /dev/cgd0e /usr ffs rw 1 2 /dev/cgd0f /home ffs rw 1 2 ``` The file rc.confg also needs to be edited ``` # vi /targetroot/etc/rc.conf ``` ``` rc_configured=YES # Add local overrides below. critical_filesystems_local="OPTIONAL:/var OPTIONAL:/usr" hostname=marte.local dhcpcd=YES dhcpcd_flags="-qM wm0" sshd=YES wscons=YES cgd=YES ``` `rc_configured=YES` is important, otherwise the system will always boot in single-user mode. These are the variables I use for a new system. For example, my network device is `wm0`. And this example hostname is `marte.local`. Your network card and requirements may be different. Also `critical_filesystems_local` variable is quite important. Since our `/usr` filesystem is not mounted at the beginning, some tools may not be available. We need to wait for it to be mounted before executing binaries from `/usr`. # Unmount and reboot Umount the new system: ``` # umount /targetroot/home # umount /targetroot/var # umount /targetroot/usr # umount /targetroot/ ``` Then shutdown or reboot: ``` # shutdown -p now ``` Fun fact. Sometimes at this point I experience a kernel panic. I have no idea why. It's very unpredictable. Sometimes it happens, sometimes it doesn't. # The new system In the new system you may need to change root password: ``` # passwd ``` Install pkgin. ``` PATH="/usr/pkg/sbin:/usr/pkg/bin:$PATH" PKG_PATH="http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/10.0/All/" # export PATH # export PKG_PATH # pkg_add pkgin ``` And add a new user. ``` # useradd -m -G wheel -k /etc/skel vsis ``` And, of course, RTFM. ``` # man afterboot ``` # Conclusion This is the method I use to install a semi-full disk encrypted NetBSD system. I may add RAID devices, LVM, multiple disks, etc. Then mount everything under `/targetroot` and extract the sets. The restriction is in the root file-system. It needs to be in plain-text and in a regular partition. It seems to me that rootfs in CGD or LVM is not well supported. I may be biased by [the Arch way](https://wiki.archlinux.org/title/installation_guide) to install the system, but I find this method better than the installer.