Janitor News #6

published

#1

Hello,

This is your monthly* burst of good news about the Janitor.

* Well ok, this is actually more of a quarterly thing at the moment, but that doesn’t mean nothing is happening—in fact quite the opposite is true. See for yourself:

Janitor & Mozilla’s A-Team

In September, I officially joined Mozilla’s Engineering Productivity team (also known as “A-Team”) to help boost the productivity of developers working on Mozilla’s software projects.

If this sounds like the Janitor to you, then you’re spot on! In fact I’m now working full-time on improving the Janitor, with the support of some amazingly talented A-Team folks, and with the goal of making contributions to Mozilla’s projects like Firefox, Servo and Rust (coming soon) much faster and easier for everyone.

And because Mozilla is awesome and a fierce supporter of Open Source in general, non-Mozilla projects like Chromium and KDE will continue to be supported, and we won’t stop there—over the next months we’ll continue adding more and more amazing projects to the Janitor, like for example Git, Mercurial, Vim, Clang, and one day maybe even the Linux Kernel. The Janitor’s long-term objective is to make open source contributions a breeze for everyone, anywhere in the world.

Say hello to Docker Host 2

Due to a swift increase in popularity over the last months, the Janitor’s back-end resources became saturated much faster than expected. After a quick datacenter visit to install a new 1 TB drive on Docker Host 1 (which is now 98% full), we realized that adding more disks wouldn’t help much longer.

So we re-designed the Janitor from the ground up and turned it into a multi-server cluster, which will allow the service to grow significantly over the coming months.

This is the Janitor cluster today:

  • Docker Host 1 (16 CPU, 64 GB, 1 TB) hosted by IRILL in a French datacenter,
  • Docker Host 2 (16 CPU, 32 GB, 1 TB) hosted by Mozilla on an EC2 instance.

Everything is still hosted on Docker Host 1 (98% full), but all Mozilla projects (about half of the entire load) will be gradually and seamlessly migrated to Docker Host 2 soon. This will free up resources to support more users and projects, and will enable more frequent image updates as well as some overdue server maintenance.

We’ve also put special efforts into completely automating the addition of new hosts, from automatic TLS and Let’s Encrypt certificates to cluster discovery and software updates, so that adding more resources becomes quick and easy. We could even make the cluster grow automatically following demand now.

If you’re one of the almost 40 people still on the waitlist, we’re so sorry to keep you waiting. We worked relentlessly on building a cluster with more resources, and we’re happy to announce that the waiting is almost over—new invites should be headed your way over the coming weeks.

JSON API

A cool spin-off from the cluster effort is the JSON API we designed to automate all things Janitor. It’s based on Self API, a system that can automatically document and test API functions against their own examples, removing the need to maintain separate documentation or test suites.

The resulting API documentation will soon be available on the website, and anyone interested in using it will be able to do so using the website’s password-less authentication, or dedicated OAuth 2 access tokens.

Even more news

  • On November 5, there was a Rust / Servo hackathon with Janitor support at the Coredump hackerspace in Zurich.
  • On November 12, Patrick Trottier gave a great talk about open source contributions and the Janitor at GDG DevFest in Sudbury. Here are his slides and a video.
  • On December 7, I will present a few Janitor-based productivity hacks at the Mozilla All Hands in Hawaii.
  • We’re currently evaluating how many contributions we’ve enabled. If you shipped code using the Janitor, please send us links to your contributions so we can count them.
  • We recently published an NPM module which will eventually allow running private cluster hosts to support your own software projects on the Janitor (coming soon™).
  • We would like to open a Discourse. If you know how that works and want to help, please reach out.

And this wraps up today’s news! If you like to review code, you’re welcome to pick a Janitor commit and post feedback on what you don’t like or don’t get about the code. Many thanks!

Until next time,
Jan