This document describes how to set up a Buildbot buildslave for LinuxCNC.
Running a buildslave requires only a computer that can check out and compile the linuxcnc git repo, can run twisted python, and can make outgoing TCP connections to the buildmaster computer. A buildslave machine can run comfortably on a 20 GB disk.
Disable the screensaver or set it to only blank the screen (so as not to soak up CPU cycles with useless fancy eye candy).
Make a user to run the buildslave. It is not a good idea to run the buildslave as yourself. I'll refer to the user as just "the buildslave user" in this document. You can name the user anything you want. Give this user a sneaky password that only you know.
Install the following packages:
apt-get install ntp lsb-release fakeroot build-essential tree pbuilder time lintian aptitude
Do an anonymous Git checkout as the buildslave user (in a throw-away directory out of the way) to make sure sure you can, and to make sure you have the ssh hostkey of the Git server machine.
git clone https://github.com/LinuxCNC/linuxcnc.git linuxcnc-dev
Several of the build steps require password-less sudo for the buildslave user.
Add these lines to /etc/sudoers:
buildslave ALL = ALL, NOPASSWD: /usr/bin/apt-get, /usr/lib/pbuilder/pbuilder-satisfydepends, /usr/sbin/pbuilder, /bin/umount, /bin/dmesg -c, /usr/bin/make V=1 setuid
At least on Hardy and Lucid, apt automatically updates the package list using a daily cronjob, /etc/cron.daily/apt. The buildbot also tries to update the package lists when it builds, and this can lead to conflict and spurious build failures. Thus you should disable automatic updating of the package lists, and let the buildbot handle it.
The cronjob is configured by a bunch of APT::Periodic apt-conf variables. A reasonable way to set these variables is to install the update-notifier-common package, then change /etc/apt/apt.conf.d/10periodic so APT::Periodic::Update-Package-Lists is "0".
Install an appropriate version of buildbot. 0.7.10 is the oldest version known to work. Versions before 0.7.10 don't work well with Git. 0.8 works, 0.9 and newer will not work until I upgrade the buildmaster.
The buildbot package in Ubuntu Lucid is new enough that it Just Works:
apt-get install buildbot
Ubuntu Hardy has buildbot 0.7.6, which is too old. If your buildslave is Hardy or older, you need to install buildbot from somewhere other than the default Ubuntu repository. I've had success with Michael Hudson-Doyle's PPA
Debian 10 "Buster" has buildbot 2.0.1, which is too new. The buildbot-slave package from Debian 9 "Stretch" installs and works fine, but make sure you mark it as "hold" so apt won't try to upgrade to the Buster version.
curl -O http://ftp.us.debian.org/debian/pool/main/b/buildbot-slave/buildbot-slave_0.8.12-1_all.deb
sudo apt install ./buildbot-slave_0.8.12-1_all.deb
sudo apt-mark hold buildbot-slave
On Debian Stretch and newer distros (but not on Debian Jessie and older, and not on Ubuntu Precise and older), you need to set a sysctl variable to allow the buildslave user to run dmesg.
Add a file named /etc/sysctl.d/allow-dmesg.conf, containing the line: kernel.dmesg_restrict=0
Reload sysctl, for example by running sudo systemctl restart systemd-sysctl, or by rebooting the machine.
Now we're ready to set up the buildslave itself. You will need two pieces of information from the buildmaster admin: the slave name and the slave password. Ask for this on the emc-devel mailing list.
The slave name is just a unique identifier for your buildslave. It should ideally be descriptive of the environment that you're building on, for example "hardy-rtai-x86".
The slave password is totally unrelated to the Unix login password of the buildslave user.
buildslave create-slave --umask=022 ~/BuildBot/slave/$SLAVE_NAME buildbot.linuxcnc.org:5133 $SLAVE_NAME $SLAVE_PASSWORD
(The buildslave directory above is just a suggestion, you can put it anywhere you like.)
If you're using buildbot version 0.7.9 or older, you need to edit buildbot.tac to set "usepty = 0". 0.7.10 and newer do not need this change.
On Ubuntu releases from Dapper through Lucid, the buildbot package provides an init script that starts the buildslave automatically on system boot. This script sources /etc/default/buildbot, so you need to fill it out with the particulars of your setup.
On other distros, buildbot may be started differently.
Edit the info/admin and info/host files in your buildslave directory (~/BuildBot/slave/$SLAVE_NAME in the example above). admin should have your name and email address, and host should have a brief description of your host. For some examples (minus the email addresses), see the BuildSlave page.
Add the git release tag signing keys to the buildslave's gnupg keyring. (These have to be long-format key IDs because Jeff Epler's key, used for signing 2.4 release tags, has a collision. But anyway, long key IDs are just a good idea in general.)
gpg --keyserver keys.openpgp.org --recv-keys 25DD353196935D7D 445B1785BC92B87F 26ADE41E54621DFA 0A30317D741499B0
Note: Debian Stretch (and newer distros?) doesn't need this.
If you want your buildslave to run the real-time test suite, you need to allow the buildslave user to lock more memory than the default configuration allows. If you're starting buildbot from the init scripts at system boot, the easiest way to do this is to add this line to /etc/default/buildbot:
ulimit -l unlimited
This step is optional!
If you want your buildslave to build debs, you need to set up a pbuilder chroot:
Create the pbuilder chroot:
Fetch the linuxcnc.org archive signing key: gpg --keyserver keys.openpgp.org --recv-key 3CB9FD148F374FEF
Export the key to a new keyring:
pre-Buster: gpg --armor --export 3CB9FD148F374FEF > linuxcnc-keyring.gpg
Buster and newer (pbuilder 0.230.4): gpg --export 3CB9FD148F374FEF > linuxcnc-keyring.gpg
On Raspbian only, override root's pbuilder config: cat > /root/.pbuilderrc
Create the pbuilder tarball: sudo pbuilder --create --basetgz $HOME/pbuilder-$DIST-emc2.tgz --distribution $DIST --components 'main' --othermirror "$LINUXCNC_ARCHIVE" --keyring linuxcnc-keyring.gpg
LINUXCNC_ARCHIVE is the LinuxCNC debian package archive for your $DIST. For example, for Lucid this is "deb http://www.linuxcnc.org lucid base" and for Hardy it's "deb http://www.linuxcnc.org hardy base"
The pbuilder chroot tarball needs to be located in the buildslave's home directory, and needs to have the name specified above.
Give the buildslave user password-less sudo by adding the following line to /etc/sudoers (already done in the sudo step above):
buildslave ALL = ALL, NOPASSWD: /usr/bin/apt-get, /usr/sbin/pbuilder, /bin/umount
Add the buildmaster's source repo to /etc/apt/sources.list.d/linuxcnc-buildbot.list. The exact form you use here depends on what distribution you're building for, look at the buildbot front page for specifics. For Lucid realtime you'd use this:
deb http://buildbot.linuxcnc.org/ lucid master-rt deb-src http://buildbot.linuxcnc.org/ lucid master-rt deb http://buildbot.linuxcnc.org/ lucid v2.5_branch-rt deb-src http://buildbot.linuxcnc.org/ lucid v2.5_branch-rt deb http://buildbot.linuxcnc.org/ lucid scratch-rt deb-src http://buildbot.linuxcnc.org/ lucid scratch-rt
Add the linuxcnc debian archive signing key to apt's keyring.
sudo apt-key adv --keyserver keys.openpgp.org --recv-keys EF1B07FEE0EE663E 3CB9FD148F374FEF
Run the same command as when creating the pbuilder tarball, but with instead of --create say --update and --override-config: sudo pbuilder --update --basetgz $HOME/pbuilder-$DIST-emc2.tgz --override-config --distribution $DIST --components 'main universe' --mirror http://archive.debian.org/debian --othermirror "$LINUXCNC_ARCHIVE" --keyring linuxcnc-keyring
That's it! I can't believe it was so easy!