My homelab has a few NixOS machines. Right now they mostly run services inside Docker, because that has been what I have done for years. This works fine, but persistent state gets annoying*. NixOS has a tool called Nixops that allows you to push configurations to remote machines. I use this for managing my fleet of machines, and today I'm going to show you how to create service deployments with Nixops and push them to your servers.
For this example, let's deploy a chatbot. To make things easier, let's assume the following about this chatbot:
default.nix
that builds the service and
includes any supporting files it might needI will also need to assume that you have a git repo (or at least a folder) with all of your configuration similar to mine.
For this example I'm going to use withinbot as the service we will deploy via Nixops. withinbot is a chatbot that I use on my own Discord guild that does a number of vital functions including supplying amusing facts about printers:
<Cadey~> ~printerfact
<Within[BOT]> @Cadey~ Printers, especially older printers, do get cancer. Many
times this disease can be treated successfully
We will need to do a few major things for defining this service:
In order for the program to be installed to the remote system, you need to tell
the system how to import it. There's many ways to do this, but the cheezy way is
to add the packages to
nixpkgs.config.packageOverrides
like this:
nixpkgs.config = {
packageOverrides = pkgs: {
within = {
withinbot = import (builtins.fetchTarball
"https://github.com/Xe/withinbot/archive/main.tar.gz") { };
};
};
};
And now we can access it as pkgs.within.withinbot
in the rest of our config.
In your configuration folder, create a folder that you will use for these
service definitions. I made mine in common/services
. In that folder, create a
default.nix
with the following contents:
{ config, lib, ... }:
{
imports = [ ./withinbot.nix ];
users.groups.within = {};
}
The group listed here is optional, but I find that having a group like that can help you better share resources and files between services.
Now we need a folder for storing secrets. Let's create that under the services folder:
$ mkdir secrets
And let's also add a gitignore file so that we don't accidentally commit these secrets to the repo:
# common/services/secrets/.gitignore
*
Now we can put any secrets we want in the secrets folder without the risk of committing them to the git repo.
Let's create withinbot.nix
and set it up:
{ config, lib, pkgs, ... }:
with lib; {
options.within.services.withinbot.enable =
mkEnableOption "Activates Withinbot (the furryhole chatbot)";
config = mkIf config.within.services.withinbot.enable {
};
}
This sets up an option called within.services.withinbot.enable
which will only
add the service configuration if that option is set to true
. This will allow
us to define a lot of services that are available, but none of their config will
be active unless they are explicitly enabled.
Now, let's create a user account for the service:
# ...
config = ... {
users.users.withinbot = {
createHome = true;
description = "github.com/Xe/withinbot";
isSystemUser = true;
group = "within";
home = "/srv/within/withinbot";
extraGroups = [ "keys" ];
};
};
# ...
This will create a user named withinbot
with the home directory
/srv/within/withinbot
, the group within
and also in the group keys
so the
withinbot user can read deployment secrets.
Now let's add the deployment secrets to the configuration:
# ...
config = ... {
users.users.withinbot = { ... };
deployment.keys.withinbot = {
text = builtins.readFile ./secrets/withinbot.env;
user = "withinbot";
group = "within";
permissions = "0640";
};
};
# ...
Assuming you have the configuration at ./secrets/withinbot.env
, this will
register the secrets into /run/keys/withinbot
and also create a systemd
oneshot service named withinbot-key
. This allows you to add the secret's
existence as a condition for withinbot to run. However, Nixops puts these keys
in /run
, which by default is mounted using a temporary memory-only filesystem,
meaning these keys will need to be re-added to machines when they are rebooted.
Fortunately, nixops reboot
will automatically add the keys back after the
reboot succeeds.
Now that we have everything else we need, let's add the service configuration:
# ...
config = ... {
users.users.withinbot = { ... };
deployment.keys.withinbot = { ... };
systemd.services.withinbot = {
wantedBy = [ "multi-user.target" ];
after = [ "withinbot-key.service" ];
wants = [ "withinbot-key.service" ];
serviceConfig = {
User = "withinbot";
Group = "within";
Restart = "on-failure"; # automatically restart the bot when it dies
WorkingDirectory = "/srv/within/withinbot";
RestartSec = "30s";
};
script = let withinbot = pkgs.within.withinbot;
in ''
# load the environment variables from /run/keys/withinbot
export $(grep -v '^#' /run/keys/withinbot | xargs)
# service-specific configuration
export CAMPAIGN_FOLDER=${withinbot}/campaigns
# kick off the chatbot
exec ${withinbot}/bin/withinbot
'';
};
};
# ...
This will create the systemd configuration for the service so that it starts on
boot, waits to start until the secrets have been loaded into it, runs withinbot
as its own user and in the within
group, and throttles the service restart so
that it doesn't incur Discord rate limits as easily. This will also put all
withinbot logs in journald, meaning that you can manage and monitor this service
like you would any other systemd service.
In your target server's configuration.nix
file, add an import of your services
directory:
{
# ...
imports = [
# ...
/home/cadey/code/nixos-configs/common/services
];
# ...
}
And then enable the withinbot service:
{
# ...
within.services = {
withinbot.enable = true;
};
# ...
}
Now you are free to deploy it to your network with nixops deploy
:
$ nixops deploy -d hexagone
And then you can verify the service is up with systemctl status
:
$ nixops ssh -d hexagone chrysalis -- systemctl status withinbot
● withinbot.service
Loaded: loaded (/nix/store/7ab7jzycpcci4f5wjwhjx3al7xy85ka7-unit-withinbot.service/withinbot.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-11-09 09:51:51 EST; 2h 29min ago
Main PID: 12295 (withinbot)
IP: 0B in, 0B out
Tasks: 13 (limit: 4915)
Memory: 7.9M
CPU: 4.456s
CGroup: /system.slice/withinbot.service
└─12295 /nix/store/qpq281hcb1grh4k5fm6ksky6w0981arp-withinbot-0.1.0/bin/withinbot
Nov 09 09:51:51 chrysalis systemd[1]: Started withinbot.service.
This basic template is enough to expand out to anything you would need and is what I am using for my own network. This should be generic enough for most of your needs. Check out the NixOS manual for more examples and things you can do with this. The Nixops manual is also a good read. It can also set up deployments with VirtualBox, libvirtd, AWS, Digital Ocean, and even Google Cloud.
The cloud is the limit! Be well.
This article was posted on M11 09 2020. Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.
Series: howto
Tags: nixos
systemd
This post was not WebMentioned yet. You could be the first!
The art for Mara was drawn by Selicre.
The art for Cadey was drawn by ArtZora Studios.