LinuxCNC Buildbot: Buildslave Admin's Guide

This document describes how to set up a Buildbot buildslave for LinuxCNC.

Running a buildslave requires only a computer that can check out and compile the linuxcnc git repo, can run twisted python, and can make outgoing TCP connections to the buildmaster computer.

  1. Disable the screensaver or set it to only blank the screen (so as not to soak up CPU cycles with useless fancy eye candy).

  2. Make a user to run the buildslave. It is not a good idea to run the buildslave as yourself. I'll refer to the user as just "the buildslave user" in this document. You can name the user anything you want. Give this user a sneaky password that only you know.

  3. Install the following packages:

    apt-get install ntp lsb-release fakeroot build-essential tree pbuilder time lintian

  4. Do an anonymous Git checkout as the buildslave user (in a throw-away directory out of the way) to make sure sure you can, and to make sure you have the ssh hostkey of the Git server machine.

    git clone linuxcnc-dev

  5. Several of the build steps require password-less sudo for the buildslave user.

    Add these lines to /etc/sudoers:

    buildslave ALL = ALL, NOPASSWD: /usr/bin/apt-get, /usr/lib/pbuilder/pbuilder-satisfydepends, /usr/sbin/pbuilder, /bin/umount, /bin/dmesg -c, /usr/bin/make V=1 setuid

    Defaults!/usr/sbin/pbuilder env_keep=DEB_BUILD_OPTIONS

  6. At least on Hardy and Lucid, apt automatically updates the package list using a daily cronjob, /etc/cron.daily/apt. The buildbot also tries to update the package lists when it builds, and this can lead to conflict and spurious build failures. Thus you should disable automatic updating of the package lists, and let the buildbot handle it.

    The cronjob is configured by a bunch of APT::Periodic apt-conf variables. A reasonable way to set these variables is to install the update-notifier-common package, then change /etc/apt/apt.conf.d/10periodic so APT::Periodic::Update-Package-Lists is "0".

  7. Install a recent version of buildbot. 0.7.10 is the oldest version known to work. Versions before 0.7.10 don't work well with Git.

    The buildbot package in Ubuntu Lucid is new enough that it Just Works:

    apt-get install buildbot

    Ubuntu Hardy has buildbot 0.7.6, which is too old. If your buildslave is Hardy or older, you need to install buildbot from somewhere other than the default Ubuntu repository. I've had success with Michael Hudson-Doyle's PPA

  8. On Debian Stretch and possibly other distros (but not on Debian Jessie and older, and not on Ubuntu Precise and older), you need to set a sysctl variable to allow the buildslave user to run dmesg.

    Add a file named /etc/sysctl.d/allow-dmesg.conf, containing the line: kernel.dmesg_restrict=0

    Reload sysctl, for example by running sudo systemctl restart systemd-sysctl, or by rebooting the machine.

  9. Now we're ready to set up the buildslave itself. You will need two pieces of information from the buildmaster admin: the slave name and the slave password. Ask for this on the emc-devel mailing list.

    The slave name is just a unique identifier for your buildslave. It should ideally be descriptive of the environment that you're building on, for example "hardy-rtai-x86".

    The slave password is totally unrelated to the Unix login password of the buildslave user.

    mkdir ~/BuildBot/slave/$SLAVE_NAME

    buildbot create-slave --umask=022 ~/BuildBot/slave/$SLAVE_NAME $SLAVE_NAME $SLAVE_PASSWORD

    (The buildslave directory above is just a suggestion, you can put it anywhere you like.)

    If you're using buildbot` version 0.7.9 or older, you need to edit buildbot.tac to set "usepty = 0". 0.7.10 and newer do not need this change.

  10. On Ubuntu releases from Dapper through Lucid, the buildbot package provides an init script that starts the buildslave automatically on system boot. This script sources /etc/default/buildbot, so you need to fill it out with the particulars of your setup.

    On other distros, buildbot may be started differently.

  11. Edit the info/admin and info/host files in your buildslave directory (~/BuildBot/slave/$SLAVE_NAME in the example above). admin should have your name and email address, and host should have a brief description of your host. For some examples (minus the email addresses), see the BuildSlave page.

  12. Add the git release tag signing keys to the buildslave's gnupg keyring. (These have to be long-format key IDs because Jeff Epler's key, used for signing 2.4 release tags, has a collision. But anyway, long key IDs are just a good idea in general.)

    gpg --keyserver hkp:// --recv-keys 25DD353196935D7D 445B1785BC92B87F 26ADE41E54621DFA 0A30317D741499B0

  13. This step is optional!

    If you want your buildslave to run the real-time test suite, you need to allow the buildslave user to lock more memory than the default configuration allows. If you're starting buildbot from the init scripts at system boot, the easiest way to do this is to add this line to /etc/default/buildbot:

    ulimit -l unlimited

  14. This step is optional!

    If you want your buildslave to build debs, you need to set up a pbuilder chroot:

    1. Create the pbuilder chroot:

      1. fetch the archive signing key: gpg --keyserver --recv-key 8F374FEF

      2. export the key to a new keyring: gpg --armor --export 8F374FEF > linuxcnc-keyring

      3. create the pbuilder tarball: sudo pbuilder --create --basetgz $HOME/pbuilder-$DIST-emc2.tgz --distribution $DIST --components 'main universe' --othermirror "$LINUXCNC_ARCHIVE" --keyring linuxcnc-keyring

      LINUXCNC_ARCHIVE is the LinuxCNC debian package archive for your $DIST. For example, for Lucid this is "deb lucid base" and for Hardy it's "deb hardy base"

      The pbuilder chroot tarball needs to be located in the buildslave's home directory, and needs to have the name specified above.

    2. Give the buildslave user password-less sudo by adding the following line to /etc/sudoers:

      buildslave ALL = ALL, NOPASSWD: /usr/bin/apt-get, /usr/sbin/pbuilder, /bin/umount

    3. Add the buildmaster's source repo to /etc/apt/sources.list.d/linuxcnc-buildbot.list. The exact form you use here depends on what distribution you're building for, look at the buildbot front page for specifics. For Lucid realtime you'd use this:

      deb lucid master-rt
      deb-src lucid master-rt
      deb lucid v2.5_branch-rt
      deb-src lucid v2.5_branch-rt
      deb lucid scratch-rt
      deb-src lucid scratch-rt
    4. Add the linuxcnc debian archive signing key to apt's keyring.

      • buildbot: EF1B07FEE0EE663E
      • 3CB9FD148F374FEF

      sudo apt-key adv --keyserver hkp:// --recv-keys EF1B07FEE0EE663E 3CB9FD148F374FEF

Updating pbuilder on a buildslave

Run the same command as when creating the pbuilder tarball, but with instead of --create say --update and --override-config: sudo pbuilder --update --basetgz $HOME/pbuilder-$DIST-emc2.tgz --override-config --distribution $DIST --components 'main universe' --mirror --othermirror "$LINUXCNC_ARCHIVE" --keyring linuxcnc-keyring

That's it! I can't believe it was so easy!