Starting services on hotplug

I want systemd to start a service when a USB device is plugged in and stop it when i remove it.

Use systemctl to get a list of units:

$ systemctl
UNIT                                       LOAD   ACTIVE SUB       DESCRIPTION
sys-subsystem-net-devices-gamelink0.device loaded active plugged   PL25A1 Host-Host Bridge

There’s no configuration required here - the .device unit just appears in response to udev events without any configuration. I’ve previously set up udev rules so that my USB host-to-host cable is consistently named gamelink0, but even without that it would show up under its default name.

The simplest way is just to take advantage of WantedBy. In gamelink.service:

$ cat /etc/systemd/systemd/gamelink.service
Description = Gamelink cable autoconf

ExecStart=/usr/bin/python3 /home/john/

WantedBy = sys-subsystem-net-devices-gamelink0.device

And then install it:

$ systemctl enable gamelink
Created symlink from /etc/systemd/system/sys-subsystem-net-devices-gamelink0.device.wants/gamelink.service to /etc/systemd/system/gamelink.service.

The WantedBy directive tells systemctl enable to drop the symlink in a .wants directory for the device. Whenever a new unit becomes active systemd will look in .wants to see what other related services needs to be started, and that applies to .device units just as much as .service units or .target units. That behaviour is all we need to start our daemon on hotplug.

The BindsTo directive lets us stop the service when the .device unit goes away (i.e. the device is unplugged). Used in conjunction with After we ensure that the service may never be in active state without a specific device unit also in active state.

Multi-core twisted with systemd socket activation

With a stateless Twisted app you can scale by adding more instances. Unless you are explicitly offloading to subprocesses you will often have spare cores on the same box as your existing instance. But to exploit them you end up running haproxy or faking a load balancer with iptables.

With the SO_REUSEPORT socket flag multiple processes can listen on the same port, but this isn’t available from twisted (yet). But with systemd and socket activation we can use it today.

As a proof of concept we’ll make a 4 core HTTP static web service. In a fresh Ubuntu 16.04 VM install python-twisted-web.

In /etc/systemd/system/web@.socket:

Description=Socket for worker %i

ListenStream = 8080
ReusePort = yes
Service = www@%i.service

WantedBy =

And in /etc/systemd/system/web@.service

Description=Worker %i

ExecStart=/usr/bin/twistd --nodaemon --logfile=- --pidfile= web --port systemd:domain=INET:index:0 --path /tmp

Then to get 4 cores:

$ systemctl enable --now [email protected]
$ systemctl enable --now [email protected]
$ systemctl enable --now [email protected]
$ systemctl enable --now [email protected]

Lets test it. In a python shell:

import urllib
import time

while True:

And in another terminal you can tail the logs with journalctl:

$ sudo journalctl -f -u www@*.service
Apr 26 02:43:51 ubuntu twistd[10441]: 2017-04-26 02:43:51-0700 [-] - - - [26/Apr/2017:09:43:51 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:52 ubuntu twistd[10441]: 2017-04-26 02:43:52-0700 [-] - - - [26/Apr/2017:09:43:52 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:53 ubuntu twistd[10444]: 2017-04-26 02:43:53-0700 [-] - - - [26/Apr/2017:09:43:53 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:54 ubuntu twistd[10452]: 2017-04-26 02:43:54-0700 [-] - - - [26/Apr/2017:09:43:54 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:55 ubuntu twistd[10452]: 2017-04-26 02:43:55-0700 [-] - - - [26/Apr/2017:09:43:55 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:56 ubuntu twistd[10447]: 2017-04-26 02:43:56-0700 [-] - - - [26/Apr/2017:09:43:56 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"
Apr 26 02:43:57 ubuntu twistd[10450]: 2017-04-26 02:43:57-0700 [-] - - - [26/Apr/2017:09:43:57 +0000] "GET / HTTP/1.0" 200 2081 "-" "Python-urllib/1.17"

As you can see the twisted[pid] changes as different cores handle requests.

If you deploy new code you can systemctl restart www@*.service to restart all cores.

systemctl enable will mean the 4 cores are available on next boot, too.

Building the Linux kernel on a Mac inside Docker: Attempt #2

Todays failure is about xargs. For whatever reason inside a qemu-user-static environment inside docker it can no longer do its part to help build a kernel:

  CLEAN   arch/arm/boot
/usr/bin/xargs: rm: Argument list too long
Makefile:1502: recipe for target 'clean' failed
make[2]: *** [clean] Error 126
make[2]: Leaving directory '/src/debian/linux-source-4.7.0/usr/src/linux-source-4.7.0'
debian/ruleset/targets/ recipe for target 'debian/stamp/install/linux-source-4.7.0' failed
make[1]: *** [debian/stamp/install/linux-source-4.7.0] Error 2
make[1]: Leaving directory '/src'
debian/ruleset/ recipe for target 'kernel_source' failed

It looks like I need to patch the kernels Makefile to workaround some limit qemu is introducing.

Building the Linux kernel on a Mac inside Docker: Attempt #1

I’ve recently been using my Docker for Mac install to try my hand at packaging the latest upstream kernel for my Raspberry Pi. I have a Debian Jessie ARM rootfs with kernel-package installed and the ideas was to run make-kpkg on an OSX-local checkout. It was able to build the main kernel but then:

make[3]: *** Documentation/Kbuild: Is a directory.  Stop.
Makefile:1260: recipe for target '_clean_Documentation' failed
make[2]: *** [_clean_Documentation] Error 2
make[2]: Leaving directory '/src/debian/linux-source-4.7.0/usr/src/linux-source-4.7.0'
debian/ruleset/targets/ recipe for target 'debian/stamp/install/linux-source-4.7.0' failed
make[1]: *** [debian/stamp/install/linux-source-4.7.0] Error 2
make[1]: Leaving directory '/src'
debian/ruleset/ recipe for target 'kernel_source' failed
make: *** [kernel_source] Error 2

The moral of this story is to not try and build kernels on case insensitive filesystems.

Finding when a file was added to Git and when it was last changed

I recently built a visualisation of all Django migrations in a project and the dependencies between them. I was most interested in recent migrations, and in particular if a migration had been changed after it had been deployed. So adding the tag a migration was introduced in (and the tag it was last modified in) seemed like a good idea.

My first attempt was to query each migration with git log with a diff filter to find out when it was added. Then I could use git tag to see which tags it was in:

$ git log --format="format:%H" --follow --diff-filter=A touchdown/core/
$ git tag --contains 184d8e88017726e695ee9cb22e428b667f6d22de

This was slow. I was traversing the same log again and again 100’s of times. So for version 2 I traverse the log in tag order just once.. What changed between 0.0.1 and 0.0.2? What changed between 0.0.2 and 0.0.3?. If it’s changed and i’ve never seen it before then it must be a new file. The new version is much faster.

import subprocess
from distutils.version import StrictVersion

# Finds the root commit of the repository
tags = [
        "git", "rev-list", "--max-parents=0", "HEAD"
# Every tag in order
        subprocess.check_output(["git", "tag", "-l"]).split(),

added = {}
changed = {}

for left, right in zip(tags, tags[1:]):
    files = set(subprocess.check_output([
        "{}...{}".format(left, right),

    for file in files:
        if file not in added:
            added[file] = right
        changed[file] = right

for file, version in added.items():
    print file, version, changed[file]

One additional changed I could make is to remove files from added and changed if they aren’t in git ls-files {tag}.