Arch Linux on ZFS

I'm starting to work on a new talk about disk latency and decided it's time to finally get my Linux workstation up to date. I also have a nice monitor that my Mac refuses to drive properly, so I figure I'll bring the machine back as a graphical desktop.

The last time I worked on this machine, I installed Funtoo on ZFS root using bits of the guide on the Funtoo Wiki here: http://open-zfs.org/wiki/Main_Page. I changed a few things along the way. Overall, it works great. I have 4 500G drives in ZFS mirrors. I can take snapshots of the root filesystem before making changes. Creating filesystems for things like /srv/work or similar is a 'zfs create' away.

Then I went to install XFCE and got to thinking about the incredible amount of time it was going to take to build the system and said, "screw it install Arch." I'll talk about why I love Arch another time, for now I want to talk about how ZFS made it easy to transition to another distribution with the ability to change back any time.

zfs create -o mountpoint=/mnt/arch tank/arch
zfs create -o compression=lzjb tank/arch/root
zfs create tank/arch/install

OK, big deal. We have an empty directory on ZFS. tank/arch is for organizational purposes and matches how I have Funtoo installed in tank/funtoo. tank/arch/root will contain the root filesystem. tank/arch/install will be used for the Arch install chroot.

The Arch wiki has decent instructions. I won't rehash them here. I used the Arch chroot method.
https://wiki.archlinux.org/index.php/Install_from_Existing_Linux

The first step is to mount the new root filesystem under the installer chroot. A bind mount would work fine, but I chose to use ZFS.

zfs set mountpoint=/mnt/arch/install/root.x86_64/mnt tank/arch/root

Then continue with the install.

wget http://www.gtlib.gatech.edu/pub/archlinux/iso/2013.11.01/archlinux-bootstrap-2013.11.01-x86_64.tar.gz

tar -C /mnt/arch/install -xzvf /tmp/archlinux-bootstrap-2013.11.01-x86_64.tar.gz

bash /mnt/arch/install/root.x86_64/bin/arch-chroot /mnt/arch/install/root.x86_64

Before the arch-chroot script would work, I had to edit it and comment out the "track_root shm ...". In my copy that is line 75. It looks like a bug, since it isn't really required at all for installations and it's already mounted on /dev which gets bind mounted before /dev/shm. Moving on ...

pacman-key --init

pacman-key --populate archlinux

I had to edit the pacstrap script the same way as the chroot script (they share the same function) to disable the check for /dev/shm. After that, it's just a normal installation of the base system. I will add other packages after a snapshot and the first boot into the new install.

pacstrap -d /mnt base

Something weird happens here. Pacstrap works fine. The system installs and I can look inside /mnt and see all the files I expect to see. Once I exit the chroot created by arch-chroot, it becomes invisble. After unmounting /mnt, the files are back! I don't see anything in arch-chroot that might mask the mountpoint, so I suspect something in ZFS isn't interacting with chroot the way I expect. In any case, the quick solution is:

zfs umount /mnt/arch/install/root.x86_64/mnt
zfs set mountpoint=/mnt/arch/root tank/arch/root
mv /mnt/arch/install/root.x86_64/* /mnt/arch/root

I'll chase down what's going on with the chroot for another post. I tried bind mounting and it had the same problem, which is weird when things like /dev are working fine.

The rest of this is pretty boring. Set up grub, etc.. It's all in the wiki so I won't cover it here unless I find something else weird.