Customizing CoreOS images

For quite a while I'm impressed by the Docker and CoreOS projects and it has been quite a while on my todo list to look into it...


Since I've access to some playground with some old Workstations... I decided to start playing around with it, using PXE boot (this was already set up on that environment).


So I followed the instructions for PXE as described on the CoreOS PXE Boot page, although it kept complaining about an "invalid or corrupt kernel image", while the checksums (MD5/SHA1) were OK. Since the TFT server is running on RHEL5, I did had an old version of pxelinux, so after the downloading the latest syslinux binary from kernel.org the system booted.


After I had the system booted the system 'alerted' me on the fact that the test-environment is using an MTU of 9000 (JumboFrames)...




This could not be fixed by the cloud-config configuration over HTTP method as far as I consider... because the cloud-config is loaded by the OS and therefore it requires up-and-running network-interface (with a correct MTU set);


So I  had to modify the CoreOS initrd, to update the MTU in /usr/lib/systemd/network/99-default.link:


MTUBytes=9000

So the we need to unpack the initial ramdisk.


Unpacking the CoreOS Ramdisk


Step 1) Create a temporary location in /tmp:


# mkdir -p /tmp/coreos/{squashfs,initrd,original,custom}


Step 2) Download or copy the ramdisk /tmp/coreos/original:


# cd /tmp/coreos/original/
# wget http://storage.core-os.net/coreos/amd64-usr/alpha/coreos_production_pxe_image.cpio.gz


Step 3) unzip the ramdisk:


# gunzip coreos_production_pxe_image.cpio.gz
# cd ../initrd
# cpio -id < ../original/
coreos_production_pxe_image.cpio


Step 4) Unsquash the squash filesystem and move the original-container:


# cd ../squashfs/
# unsquashfs ../initrd/usr.squashfs
# mv ../initrd/usr.squashfs ../usr.squashfs-original


Please note... that you need at least the squashFS 4.0 tools... but you can download the source and compile the binaries (at least it works on RHEL5).


And now you can access the unpacked image via /tmp/coreos/squashfs/squashfs-root and perform modifications, but please use the path minus the usr-prefix and relative to /tmp/coreos/squashfs/squashfs-root. So summarized:


/usr/lib/systemd/network/99-default.link can be found in:
/tmp/coreos/squashfs/squashfs-root/lib/systemd/network/99-default.link


So hack around and apply modification where needed.


Packing the CoreOS Customized Ramdisk


Now we have to repack the ramdisk, so we can load it...


Step 1) Repack the squashfs 


# cd /tmp/coreos/squashfs
# mksquashfs  squashfs-root/ ../initrd/usr.squashfs -noappend -always-use-fragments


Please ensure you use squashfs tools 4.0!


Step 2) Make it all a cpio archive and zip it


# cd /tmp/coreos/initrd
# find . | cpio -o -H newc | gzip > ../custom/coreos_CUSTOM_pxe_image.cpio.gz


Now boot it and use the custom image as initrd.