# Create a Custom Firecracker Image In the previous post I confirmed that I can run a Firecracker instance locally. However, it’s using a hello-world image, and I need to customize it. Today, I’m putting a custom image on a Firecraker. Basically, you have to do two things: - Make a Linux kernel. An official tutorial how to do this: https://github.com/firecracker-microvm/firecracker/blob/master/docs/rootfs-and-kernel-setup.md - Make an `ext4` filesystem image ## Create a Kernel image Here’s a quick step-by-step guide to building your own kernel that Firecracker can boot: 1. Get the Linux source code: ```bash $ git clone https://github.com/torvalds/linux.git $ cd linux.git ``` 2. Check out the Linux version you want to build (e.g. we’ll be using v5.10 here): ```bash $ git checkout v5.10 ``` 3. You will need to configure your Linux build. You can start from [the recommended config](https://raw.githubusercontent.com/firecracker-microvm/firecracker/master/resources/microvm-kernel-x86_64.config) by copying it to `.config` (under the Linux sources dir). You can make interactive config adjustments using: ```bash $ make menuconfig ``` 4. Build the uncompressed kernel image, depending on your `.config` file and may need to answer a few questions: ```bash $ make vmlinux ``` This is going to take some time, so take a moment and pour yourself some tea. 5. Upon a successful build, you can find the uncompressed kernel image under `./vmlinux`. ```bash $ cp vmlinux /tmp ``` ## Create a rootfs image A rootfs image is just a file system image, that hosts at least an init system. For instance, this guide uses an EXT4 FS image with OpenRC as an init system. Note that, whichever file system you choose to use, support for it will have to be compiled into the kernel, so it can be mounted at boot time. To build an EXT4 image: 1. Prepare a properly-sized file. We’ll use 50MiB here, but this depends on how much data you’ll want to fit inside: ```bash $ cd /tmp $ dd if=/dev/zero of=rootfs.ext4 bs=1M count=50 ``` 2. Create an empty file system on the file you created: ```bash $ mkfs.ext4 rootfs.ext4 ``` You now have an empty EXT4 image in rootfs.ext4, so let’s prepare to populate it. First, you’ll need to mount this new file system, so you can easily access its contents: ```bash $ mkdir /tmp/my-rootfs $ sudo mount rootfs.ext4 /tmp/my-rootfs ``` The minimal init system would be just an ELF binary, placed at `/sbin/init`. The final step in the Linux boot process executes `/sbin/init` and expects it to never exit. More complex init systems build on top of this, providing service configuration files, startup / shutdown scripts for various services, and many other features. For the sake of simplicity, let’s set up an Alpine-based rootfs, with OpenRC as an init system. To that end, we’ll use the official Docker image for Alpine Linux: 1. First, let’s start the Alpine container, bind-mounting the EXT4 image created earlier, to `/my-rootfs`: ```bash $ docker run -it --rm -v /tmp/my-rootfs:/my-rootfs alpine ``` 2. Then, inside the container, install the OpenRC init system, and some basic tools: ```bash # apk add openrc # apk add util-linux ``` 3. And set up userspace init (still inside the container shell): ```bash # # Set up a login terminal on the serial console (ttyS0): # ln -s agetty /etc/init.d/agetty.ttyS0 # echo ttyS0 > /etc/securetty # rc-update add agetty.ttyS0 default # # Make sure special file systems are mounted on boot: # rc-update add devfs boot # rc-update add procfs boot # rc-update add sysfs boot # # Then, copy the newly configured system to the rootfs image: # for d in bin etc lib root sbin usr; do tar c "/$d" | tar x -C /my-rootfs; done # for dir in dev proc run sys var; do mkdir /my-rootfs/${dir}; done # # All done, exit docker shell # exit ``` 4. Finally, unmount your rootfs image: ```bash $ sudo umount /tmp/my-rootfs ``` You should now have a kernel image (vmlinux) and a rootfs image (rootfs.ext4), that you can boot with Firecracker. Now, you can run Firecracker with your custom images: ```bash $ firectl --kernel=/tmp/vmlinux --root-drive=/tmp/rootfs.ext4 --kernel-opts="console=ttyS0 noapic reboot=k panic=1 pci=off nomodules rw" ``` Let’s do some automation and pre-create a `Dockerfile` and then copy the contents of the container into a filesystem image. For the testing, I’m also instal `python3` and therefore increased the image to 150MiB. Dockerfile: ```dockerfile FROM alpine:3.7 RUN apk add openrc RUN apk add util-linux RUN apk add python3 # Set up a login terminal on the serial console (ttyS0): RUN ln -s agetty /etc/init.d/agetty.ttyS0 RUN echo ttyS0 > /etc/securetty RUN rc-update add agetty.ttyS0 default # Make sure special file systems are mounted on boot: RUN rc-update add devfs boot RUN rc-update add procfs boot RUN rc-update add sysfs boot # Change root password RUN echo 'root:root' | chpasswd ``` Here’s a tiny script by Julia Evans from this post https://jvns.ca/blog/2021/01/23/firecracker--start-a-vm-in-less-than-a-second/, which creates a rootfs: ``` IMG_ID=$(docker build -q .) CONTAINER_ID=$(docker run -td $IMG_ID /bin/bash) MOUNTDIR=mnt FS=rootfs.ext4 mkdir $MOUNTDIR qemu-img create -f raw $FS 150M mkfs.ext4 $FS sudo mount $FS $MOUNTDIR sudo docker cp $CONTAINER_ID:/ $MOUNTDIR sudo umount $MOUNTDIR ``` Run a Firecracker: ```bash $ firectl --kernel=/tmp/vmlinux --root-drive=./rootfs.ext4 --kernel-opts="console=ttyS0 noapic reboot=k panic=1 pci=off nomodules rw" ... Welcome to Alpine Linux 3.7 Kernel 5.10.0 on an x86_64 (ttyS0) (none) login: root Password: Welcome to Alpine! The Alpine Wiki contains a large amount of how-to guides and general information about administrating Alpine systems. See <http://wiki.alpinelinux.org>. You can setup the system with the command: setup-alpine You may change this message by editing /etc/motd. login[612]: root login on 'ttyS0' (none):~# python3 -c "print(40 + 2)" [ 15.635148] random: python3: uninitialized urandom read (24 bytes read) 42 ``` I’m still not sure whether I like Docker setup to create rootfs, but as it works I’ll stick with it for now. I guess, it must be a different way how to set it up, and I’ll leave till next time. Hopefully, tomorrow I can find some extra time and figure out how to connect Firecracker guest to the network.