In June of last year I wrote out my plans for making a tool to manage virtual machines on my homelab called waifud. I have gotten almost nothing done on it for a while due to Facts and Circumstances getting in the way. However, I have broken through the dread and gotten to a point where waifud is an actual daemon instead of a prototype of a tool written in Go.
Currently it allows you to make an instance (virtual machine), list the instances that waifud manages, delete instances, get information from libvirt for every machine that libvirt manages and manage distribution base image snapshots.
Here is the overall architecture:
config.dhallthat it expects to find in its current working directory.
I have managed to totally refactor out the requirement for waifud-metadatad at this time. I am glad this is the case, doing this will allow me to implement everything in Rust and reduce complexity in developing this tool.
Here are the downsides of the currently implemented feature set:
zfscommand and know your way around
virshenough to reset things back to how they should be). You may also need to edit the XML that the templates generate in the worst case, so being familiar with that can't hurt.
In a production environment, it would be reasonable to describe these downsides as "catastrophic". However for something that's only really started to exist in its current form less than 48 hours ago, I'd prefer to describe that as "having promise".
I still plan to make waifud under a dual pay-me-for-commercial-support/Be Gay Do Crimes license. The latter is not accepted by the OSI, so I am fairly sure that I will be able to avoid getting this software packaged by any distributions until it is more stable. Then I can relicense it as reality demands.
If you want to test this and have NixOS machines that run zfs and libvirtd, here is how you can set it up:
mkdir -p $HOME/.cache/within/mkvm/qcow2.
config.dhallin the root of the waifud repo to contain the host/hosts you want to puppet with waifud.
config.dhallto point the base URL to the IP address of the server running waifud. This is used with cloud-init to load your user-data into virtual machines. You probably want to make sure this works with plain HTTP, I don't know if all the VM images that I ship by default come with CA certificates.
cargo run --bin waifudin one terminal window.
cargo run --bin waifuctl -- --host http://[::]:23818 create --distro ubuntu-20.04 --host pneuma --zvol fast/vms --disk-size 20 --user-data /path/to/config.yamlto contain one of your hostnames, the right zfs parent dataset and your cloudconfig user data, then hope it works.
The core of this is built on Axum, a Rust web application framework that I want to base more things on in the future. Previously I have liked Rocket and Warp, however Axum seems more likely to stick around for a very long time and is actively maintained.
In its current state, here's what it looks like to create a VM running Arch Linux.
$ waifuctl --host http://[::]:23818 \ create --distro arch --host pneuma \ --zvol fast/vms --disk-size 20 \ --memory 1024 --cpus 4 \ --user-data ./var/xe-base.yaml created instance sunspot on pneuma, waiting for IP address IP address: 10.77.131.97
And then you can SSH into that instance as normal:
$ ssh [email protected] Warning: Permanently added '10.77.131.97' (ED25519) to the list of known hosts. [[email protected] ~]$ uname -av Linux sunspot 5.16.5-arch1-1 #1 SMP PREEMPT Tue, 01 Feb 2022 21:42:50 +0000 x86_64 GNU/Linux [[email protected] ~]$ cat /etc/os-release | head -n1 NAME="Arch Linux"
Then when you get bored of it, you can remove it just as easily:
$ waifuctl --host http://[::]:23818 delete sunspot
No more having to remember the baroque pacstrap flags. No more having to figure out how to add your SSH keys places. No more spending hours setting up and maintaining VMs that you may only need for a few minutes a month. No more having to come up with names for your VMs. No more remembering what you did to VMs when you later need to debug what is going on. No more of any of that. Spend less time fighting whatever you did to OpenSUSE and more time doing things that matter to you, like cuddling waifus.
Out of the box, waifud ships with templates for the following distributions:
You can query the list of templates with the following command:
$ waifuctl --host http://[::]:23818 distro list alpine-3.13 alpine-3.14 alpine-3.15 amazon-linux arch centos-7 centos-8 centos-stream-9 fedora-35 opensuse-leap-15.3 opensuse-leap-15.4 opensuse-tumbleweed rocky-linux-8 ubuntu-18.04 ubuntu-20.04
mkvmcurrently will create a custom VM image per invocation based on the module you pass it, however I may be able to figure out a decently generic base image that you can layer your own config on top of. Follow this GitHub issue for more information as I figure out better ways to do this.
mkvm created a cloudconfig seed file. This is basically an ISO file
that gets mounted to the VM and then cloud-init picks up on it and does what it
says. This is an incredibly cursed affair, but it does work.
waifud instead uses the nocloud-net data source to fetch the cloudconfig over HTTP. When you create an instance in waifud, waifuctl uploads the user-data to the server. This is then stored in SQLite and queried from when a VM boots. cloud-init picks up on this data and then executes it, just like it would when you're using a cloudconfig seed from a CD.
$ curl http://127.0.0.1:23818/api/cloudinit/e81ddefc-3fa1-4fdf-9809-ee19f06f9675/meta-data instance-id: e81ddefc-3fa1-4fdf-9809-ee19f06f9675 local-hostname: manaphy
$ curl http://127.0.0.1:23818/api/cloudinit/e81ddefc-3fa1-4fdf-9809-ee19f06f9675/user-data #cloud-config #vim:syntax=yaml users: - name: xe groups: [ wheel ] sudo: [ "ALL=(ALL) NOPASSWD:ALL" ] shell: /bin/bash ssh-authorized-keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPg9gYKVglnO2HQodSJt4z4mNrUSUiyJQ7b+J798bwD9 - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPYr9hiLtDHgd6lZDgQMkJzvYeAXmePOrgFaWHAjJvNU
This successfully allows me to refactor
genisoimage out of the critical path
of creating virtual machines. This should also allow me to migrate VMs between
zfs send in the future without having to copy around the
cloudconfig seeds. I should be able to enable editing cloudconfig data too.
I hope this preview of what is to come with waifud was interesting. Future tasks will include making a management panel with Xeact and Xess, making NixOS modules to automate installation/configuration of machines and so much documentation.
Even in this minimal state though, waifud shows a lot of promise to be used even
more than the
mkvm prototype tool. If I need an Ubuntu, I can get one in
seconds. Faster than AWS even. waifud is the future of my infrastructure.
waifud makes an init snapshot every time it creates an instance. This will be used to roll back instances to the "fresh out of the box" state if you need to undo everything and go back to a fresh state as an emergency hammer. When you do the kinds of cursed distro testing that I do on a regular basis, this is a very useful hammer to beat your infrastructure with when you need it.
This is going to be instrumental to how my future clusters work. Originally I was going to pair this with assimil8, but cloud-init unfortunately has mass market adoption and I don't feel like fighting that. The only images I have to build for myself are the Alpine Linux ones. Every other image is the unmodified upstream image that is shipped to cloud providers.
As a fun added bonus/easter egg, whenever you set up an Ubuntu machine with waifud you get an extra message in your MOTD:
Welcome to waifud <3
I will probably remove or edit this in the future. I'm still figuring out how cloud-init vendor data works. The total lack of useful documentation and examples here is quite annoying. If you have any examples to give me, please do.
More to come when I have more things to write about.
This article was posted on M02 06 2022. Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.
This post was not WebMentioned yet. You could be the first!
The art for Mara was drawn by Selicre.
The art for Cadey was drawn by ArtZora Studios.