Page MenuHomeSoftware Heritage

Survey of continuous integration systems
Closed, MigratedEdits Locked


We need to make an assessment of the existing continuous integration tools, and the amount of integration we can get with our current setup.

References : (Upstream phabricator integration with external build tools)

For this evaluation, I have focused on self-hosted tools as they are the most flexible and the most easily trustable (allowing us to do things like package builds etc. within our infra).

Here are the contenders for this evaluation:

  • Harbormaster (Phabricator built-in CI tool)
  • Jenkins
  • buildbot
  • Drone CI (
  • GitLab CI

Here's a comparison table on the solutions

CriterionPlain HarbormasterJenkinsBuildbotDrone CIGitLab CI
Installation (master)Built into phabricatorUpstream Debian packages; Docker ImagesDebian packagesDocker Images (only option supported upstream)Built into GitLab (Upstream Debian packages available)
Installation (workers)Manual (and fairly rough)Jenkins agents (Jenkins can also execute tasks on the master)Debian-packaged buildbot worker daemonUse the master host itself as a docker providerDebian packages for the runner, uses docker in backend
Phabricator integrationBuilt-inWith third party plugin (supports build on diffs, maybe on revisions?)Could not find anyone who did it. webhooks or polling ?Could not find anyone who did it. webhooks ?Only integrated with GitLab
Community supportlowhigh (lots of installs of all sizes)med-high (lots of high profile, heavily custom installs)low-ishmed-high (lots of small installs)
Configuration formatBring your own scripts - Actual jobs setup in phabricator interface
Default: database in the CI master; Can do declarative configuration in-repository Jenkinsfile; Can do declarative configuration generation out-of-repository with jenkins-job-builderDeclared as python scriptsIn-repository yaml fileIn-repository yaml file
Debian package building integrationBring your own scripts used by grml,, ...i3 does it: your own scriptsBring your own scripts; Debian uses GitLab for its development now and some people probably have the proper glue setup already
CriterionPlain HarbormasterJenkinsBuildbotDrone CIGitLab CI

Event Timeline

olasd changed the task status from Open to Work in Progress.Apr 13 2018, 11:08 AM
olasd triaged this task as High priority.
olasd created this task.

I think we went with Jenkins + jenkins-job-builder at this point...