systemd: The Good Parts

Sun May 16 2021


The slides link will be at the end of the post.

Hello, I'm Xe and today I'm going to do a talk about systemd. More specifically the good parts of systemd. This talk is going to go fast because there's a lot of material to cover and the notes are going to be on my website. I have been an Alpine user for almost a decade and it's one of my favorite linux distributions.

The best things in life come with disclaimers and here are the disclaimers for this talk:

What is systemd?

When doing a talk about a thing I find it helps to start with a good definition of what that thing is. Given this talk is about systemd let's start with what systemd is.

A map of systemd components

systemd is a set of building blocks that you can use to make a linux system. This diagram covers most of the parts of systemd. There is everything from service management to log management to boot time analysis, network configuration, and user logins; but we're only going to cover a tiny fraction of this diagram. At a high level systemd provides a common set of tools that you can build a linux system with; kind of like lego bricks. It does just manage services but it does more than just service management.

Something else that's useful to ask is "why does systemd exist?" Well, looking back at that diagram, computers are actually fairly complicated. There's a lot going on over here. There's log management, there's disk management, there's service sequencing, network configuration, containers user sessions and most importantly all of these things need to happen in order or bad things can happen. I mentioned that systemd is more than just a service service manager because it has optional components that manage things like dns resolution, network devices, user sessions, and user level services among other things.

One of the big differences between systemd and other things like OpenRC is that systemd is a very declarative environment. In declarative environments you specify what you want and the system will figure out what it needs to do to get there. In an imperative environment you specify all of the steps you need to do to get there. It's the difference between writing a sql statement and a for loop.

So, pretend that this somewhat realistic scenario is happening to you: it's 4:00 am you just got a panicked call from someone at your company that the website is down. You log into a server and you want to see if the website is actually down or if it's just dns. You probably want to know the answers to these basic questions:

systemd includes a tool called systemctl that allows you to query the status of services as well as start and stop them; but for right now we're going to look at the systemctl status subcommand. Here is the output for the systemctl status command for the service powering So let's go down the list:

If you haven't seen all of this in action before you might be wondering something like "Wait, where did it get those logs from?"

I mentioned systemd does more than just start services. systemd has a common log sink called the journal. Logs from the kernel, network devices, services, and even some other system sources that you may not think are important automatically get put into the journal. It's similar to Windows event logs or the console app in macOS except it's implicit instead of explicit (Windows and macOS make you use some weird logging calls to make sure that log lines actually get in there, but systemd will capture the standard output, standard error and syslog for every service managed by systemd). Something neat about the journal is that it lets you tail the logs for the entire system with one command: journalctl -f. Here's that command running on a server of mine:

journalctl output

There's a lot more to the journal involving structured logging, automatically streaming the logs to places, and advanced filtering based off of different units, services, or other arbitrary fields; however that is out of scope for this talk. The important part is that it has support for that in case you actually need it.

Now this is all great, and you might be think asking yourself "well, yeah, this stuff is cool; but how does Alpine fit into this? Alpine can't run systemd because systemd is glibc specific." However we're not talking about systemd directly, we're talking about the philosophies involved and the truth is that this kind of experience is what people already have elsewhere. By not having something competitive Alpine is less and less attractive for newer production deployments.

Now there's at least four classes of benefits for systemd and I'm going to break them down into the following groups:

In general people that are developing services that run on systemd get the following benefits:

Packagers or people that are putting software into packages get the following benefits:

system administrators of systemd systems also get the following benefits:

As far as users go:

I really hope Alpine comes up with something similar to systemd. Alpine can really benefit from a tightly integrated service manager that does at least some of the things that systemd does. Declarative really is better than imperative because declarative is easier for distracted operators.

People get distracted. It happens, and when distracted people do things it can sometimes have bad consequences. So if we make the tools powerful, but implicitly correct, then it will just be a lot better overall and users will have a lot less worry involved.

On that note we are very close to hitting time so here's my shout outs to people who either help make this talk happen or I think are cool.

if you have any questions please feel free to ping me on twitter, in the irc room, or on the compact page on my website. I enjoy these kinds of questions and I openly welcome you to ask them.

Thank you, have a good day.

Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.


View slides