Categories

BusPirate.com: Compile PIC and ARM firmware on a cheap server

Posted on Thursday, September 20th, 2018 in Bus Pirate, compilers, firmware by Ian

Buspiratecom-head

The Bus Pirate project currently has four firmware builds (v3/v4/v5/vNG1) under two toolchains (PIC C/ARM C). To make this more manageable, we use a cheap VPS to check for new code in our git repo and compile the firmware automatically. Fresh compiles are available for everyone immediately, without any intervention from developers or friendly forum members.

We wanted extremely tight integration with BusPirate.com so we rolled our own script instead of using a tool like Jenkins. This post covers:

  • Installing MPLABX and PIC XC16 compiler for automated builds
  • Installing ARM GCC and libopencm3 for automated builds
  • A simple build service to periodically update repos and compile firmware

Is there new code to compile?

Creating builds for a bunch of platforms is boring, let’s automate it! First we find out if there’s new code to compile. With git we can check the log before and after a ‘pull’ command.

#git log --pretty=format:'%H' -n 1
cba17f0d1cf0f7a2b5e8ad5954390feae08b6ba3

Check the git log for the long hash of the current local repository.

git pull

Pull the latest commits and update the local repository.

git log --pretty=format:'%H' -n 1
42f05e5fe033f37d029a0483c967a01a81222c7c

Check the long hash again. The hash changed! Let’s compile some code!

make clean && make

Compile the latest code and then do something useful with the output.

Server setup

A build server can definitely run in your basement or at the hackerspace, but a $5/month VPS from Vultr or DigitalOcean is a super slick option. We run Ubuntu 14.04 LTS 64bit because some of libraries we need aren’t available for newest versions of Ubuntu.

Install instructions for both toolchains are in shell script .sh format. You shouldn’t run these! Defaults and version numbers change. Some user interaction is needed. Open a shell terminal and paste the lines one at a time. Notes in the .sh file explain each step.

Setup PIC MPLABX and XC16 compiler for automated builds

PIC firmware can be compiled on a headless Linux server since Microchip released MPLABX and the XC compilers. Installation is a bit tedious, but the toolchain works great. bp-install-mplabx.sh documents our setup. Credit to this solution.

The Bus Pirate MPLABX project needs a little preparation before compiling on the server. There’s three active hardware versions to compile for the PIC-based Bus Pirate. Each hardware version needs a separate MPLABX project, the project configuration passes the hardware version #define via a compiler flag.

Setup ARM GCC compiler and libopencm3 for automated builds

ARM GCC is easier to setup than the PIC toolchain. Install the compiler, pull the source, then compile libopencm3 and the Bus Pirate NG firmware. bp-install-armgcc.sh documents our install.

Install and configure automated build service

buildv2.py runs ‘git pull’ every 10 minutes. If there’s new commits, the source is compiled and the firmware is uploaded to a server for further processing. Follow the steps in bp-install-autobuild.sh to install and configure the build service.

buildv2_tasks.py has settings for the repositories and builds. Each repository can have multiple build tasks. The PIC-based Bus Pirate repo is pulled once, then compiled for three hardware configurations (v3/4/5).

The service can be started from the command line with:

/etc/init.d/bp-build start

And stopped with:

/etc/init.d/bp-build stop

If the service is installed in /etc/init.d it will start with the operating system. The script has been stable for months at a time, but we play it safe and use monit to restart the service if it crashes.

Backend processing

buspiratecom-CDreleases

The firmware and build logs are packaged into a JSON file and uploaded to an API at BusPirate.com. Here’s an example of the JSON output, the important variables are described in the table below.

‘error’ 0 if make executed, 1 if make returned system error (build failed with errors).
‘timestamp’ start/stop build timestamp
‘firmware’/’hardware’ Identifying info
‘starthashlong’/’endhashlong’ Commit hash before/after git pull command
‘gitoutput’ Output from the git pull’ command
‘makeoutput’ Output from the make command
‘apikey’ Identifying info
‘firmware_type’ The file extension of the firmware. Added to automate naming in the backend. PIC uses .HEX, ARM uses .BIN
‘base64encbin’ Base 64 encoded firmware file

BusPirate.com serves up the fresh build and notifies anyone subscribed to the mailing list. We also grab all the commit notes and change history from GitHub so it’s easy to see what’s in the build. You can check it out here.

Up next

For the rest of this week we’ll be testing a new version of Dirty Cables at DirtyCables.com.

Later next week we’ll receive a few hand-assembled Bus Pirate v5 prototypes. We ruined another Bus Pirate v5 prototype by tearing the USB Micro B connector and traces off the PCB. Instead of building another board in-house, we sent the parts to a Chinese PCBA that does two-off hand-assembled prototypes. Let’s see how they turn out!

This entry was posted on Thursday, September 20th, 2018 at 11:06 am and is filed under Bus Pirate, compilers, firmware. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

4 Responses to “BusPirate.com: Compile PIC and ARM firmware on a cheap server”

  1. Squonk42 says:

    I would love to get more information regarding which Chinese PCBA you chose and the cost for the BP NG.

  2. John Peck says:

    Really impressive! Thanks for putting so much documentation together!

  3. chrismec says:

    What is this Bus Pirate v5? Is this the demo board or is there a new bus pirate in the works?

  4. Alex B says:

    To see the current commit hash, “git rev-parse HEAD”

Leave a Reply

Notify me of followup comments via e-mail. You can also subscribe without commenting.

Recent Comments

  • Edward Mallon: While I loved the Nokia 5110 LCD's readability in full sun, the pressure sensitivity was a real problem for the underwater units. So I started...
  • ConnyCola: I like this article a lot! And in contrast to Max I love to hear what you guys are doing in Shenzen and Hong Kong....
  • Max: Thanks for the writeup, it's always interesting to read about a different culture. Other than that though I'm afraid I was spoiled by living neither...
  • Tepa: Heps keikkaa
  • Dan: Pick Me!!!