Development Infrastructure Evaluation
This is an evaluation of development infrastructure solutions, with a view to replacing GNOME's cgit and Bugzilla installations.
Contents
Background
For a long time, GNOME has been using cgit for code hosting and Bugzilla for issue tracking. Over the years, these tools have become increasingly antiquated: the user experience hasn't significantly improved and new competitors make them feel old, in tools and features provided, integration, workflows and look.
GNOME needs to improve its development infrastructure. It needs to provide its developers with a more efficient, integrated, featureful and enjoyable set of tools. It also needs to provide a developer experience that does not act as a barrier to new contributors, to ensure the future of the GNOME project.
Nowadays, GitHub is used by most new Free Software projects and sets the expectations of new contributors. GitHub is not Free Software, of course, which makes it unacceptable to many in the GNOME community.
Available Solutions
Available alternatives to cgit and Bugzilla include GOGS, gitea and Pagure. However, these were rejected early on in this evaluation, due to their contributor levels and sustainability.
This evaluation therefore focuses on two main solutions: GitLab and Phabricator. If you are unfamiliar with these, please see the introduction to these two technologies. Be aware that GitLab isn't simply a GitHub clone!
Either of these options would be hosted on GNOME infrastructure.
Migration Possibilities
GNOME has an established Bugzilla instance with many thousands of issues stored in it. Migration is therefore a critical issue when discussing switching to a different solution.
Our contention is that copying/moving every existing GNOME issue to a new issue tracker is impractical and, in many situations, undesirable. Instead, we propose the following migration plan:
- Shortly after the new solution is setup, disable issue reporting in Bugzilla (as well as new accounts, milestones, etc).
- Provide tooling to allow maintainers to do a "shallow" migration of their module's issues to the new solution (copying the content of the issue over as well as a copy of its comments, but not reproducing CCs, accounts, detailed history).
- After a length of time (to be decided), potentially switch Bugzilla into read-only mode (no new comments, attachments, etc)
This is just a sketch and the team behind this proposal are committed to listening to the GNOME community and creating a migration plan based on their concerns.
Methodology
This review has been conducted by a small group of GNOME contributors who have a range of experience within the project, including module maintenance, newcomer experience, code review and design.
An effort has been made to consult with relevant parties, including the Translation Team, Sysadmin Team, Release Team and Bug Squad. We have also reached out to both GitLab and Phabricator and people using them in a daily basis.
Other steps that have been taken as a part of the review:
- Studied existing GNOME workflows and requirements
- Setup of test installations, along with some stress testing for issue database size
- Detailed feature review
- Consultation with users of each of the options under consideration
Despite all these steps, we are not claiming to be experts or to have complete knowledge. We therefore welcome any input, particularly from the GNOME community.
Background Information
Evaluation
Phabricator and GitLab are both excellent tools. However, it is important to realise that they are quite different in their priorities and how they are structured. This in turn requires that their respective strengths and weaknesses be recognized and considered alongside GNOME's priorities and requirements.
Sysadmin deployment/maintenance
GitLab is available as RPMs, which makes it trivial to deploy and update - our experience is that it is very quick to set up. Phabricator isn't difficult to set up, but it does involve a bit more work.
Usability/user friendliness for newcomers
GitLab provides a UI for common code hosting operations, including forking, merging, editing files and committing (useful for documentation more than code). It also provides an experience which is similar to GitHub, which has advantages when attracting developers who are familiar with that platform.
GitLab is quite easy to navigate, and shows the key resources for each module at a glance.
Phabricator doesn't provide a web UI for common code tasks. Its modular approach means that the UI is less integrated as a whole: while different Phabricator "applications" can use common links to data objects, the site navigation as a whole doesn't always provide a clear hierarchy or overview.
Code hosting
Phabricator's code hosting features are somewhat rudimentary by today's standards. As already mentioned, it doesn't provide UI for repository management. It doesn't allow grouping repositories by teams or area (something GNOME does currently for infrastructure, different types of module).
Phabricator is designed to work with a variety of version control systems - it isn't Git-focused.
GitLab provides a more modern code hosting experience, which is obviously Git-focused.
Issue tracking
Phabricator is primarily a project management tool. It has comprehensive capabilities for tracking and managing issues over the various stages they go through in the development process. It allows management of team workflows and workloads, and is highly milestone and release-focused. It is highly customisable.
GitLab's issue tracking is primarily organized around repositories. However, it does also allow a global project-wide view of issues. Freeform tags allow organizing issues (tags can be for the whole of GNOME or per-repository). It also has a kanban-style task management feature, called the Issue Board.
Code review
Code review in Phabricator is very good and people seem to like it a lot. However, it is also patch-based - it requires that the developer generates patch files locally and uploads them through the browser, or use the arcanist tool.
GitLab's code review is also good. The standard workflow is to review a branch as part of a merge request.
Other features
Phabricator and GitLab provide a range of additional tools that could be beneficial for the GNOME project. This section only covers some of them.
Continuous integration:
GitLab has a CI runner that can be used to run jobs for every commit. This is interesting to GNOME, particularly in relation to Flatpak.
- Phab doesn't really have CI.
Task management/kanban:
Both Phabricator and GitLab have task management boards. Phabricator's is probably more advanced and operates at a project level. GitLab's task management boards can be created for groups of repositories, and so could be used by the Release Team.
Design review:
- Phabricator has a design review module which allows commenting on mockups, tracking changes to mockups and linking of issues to designs. Its project management features also allow explicit design review steps as a part of the development process.
CLI tool:
With GitLab, a number of operations that currently require git-bz can be done using Git only (for example: fetching the changes for a bug fix). There is a CLI tool which allows a wide range of actions to be done from the command line, although it isn't particularly user friendly.
- Phabricator
- Collabora has a git-phab tool which is very similar to git-bz but better
Arcanist is a cli interface to Phabricator
Long-term sustainability
Both GitLab and Phabricator have companies behind them. GitLab is a larger organization, but both have a excellent development rate. Both of the solutions we are considering are open source, and so could be picked up by communities should the companies behind them fail.
Phabricator does have relatively high adoption in the FOSS world and so could be said to have more community buy-in.
Conclusion
GNOME's requirement is for a complete solution that provides integrated code hosting and issue tracking. Key requirements include a modern code hosting solution, easy adoption and navigation by newcomers (who might already be familiar with GitHub) and a relatively simple set of issue tracking features.
While Phabricator provides some excellent features and has first-class project management capabilities, its code hosting provision is not at the level being sought by GNOME.
It is the recommendation of this evaluation that, due to its rounded feature set, modern code-hosting and integrated UI, GitLab is the best choice for the GNOME project.
As a final part of this evaluation, we have documented the expected GNOME contributor workflows when using GitLab.
You have a GitLab test instance in our infrastructure to play with. Note that the projects in this instance are artificially fill with issues reports and other testing use cases, and doesn't represent accurately a real use case. You can find more realistic cases in projects hosted in gitlab.com itself.
Consultation and Comments
A consultation will be held with the GNOME community before a final decision is made about future infrastructure choices. In the mean time, feel free to leave comments on the comments page.
Feedback and Comments
In this page we will put the community input the initiative received and the points we need to take into consideration. Please don't edit this page, rather add a comment in the comments page or through the email or IRC.
See community feedback for a summary of the responses we've had to the migration proposal.