In truth, it began before that. It began in 2004 with a new laptop, and a download of this new-fangled Firebird browser that I’d been hearing about. It began with faux conviction – “Oh, it’s not from big, evil Microsoft” -  but I was young and naïve.  I used the browser, updated the browser, but filed no bugs, tested no nightlies, waxed lyrical about web standards to no-one. Besides, what could little old me contribute to a big, famous project like Firefox?

Nevertheless, I gradually began to feel I should give something back. I hit the docs. Bug triage, huh? Hmm, maybe I could do that. I floated around Bugzilla, watching, waiting, trying to understand the processes. That December 10th, I took the plunge. And so it began.

As I wandered around Bugzilla over the following days, a crazy notion took hold: could I contribute more? There sure seemed to be a lot of bugs looking for owners. I’d programmed in C in 3rd year of university, though never C++, but at least it was a base to build from. There was more to learn too: getting the source, CVS, just building this thing., pestering Ted with the sort of autoconf question he must have answered a thousand times yet still patiently answered. And what then? What to work on, and how to find my way around this huge codebase?  Eventually I discovered “Good First Bugs”.  Bug 393922 looked easy – real easy! I arrogantly rushed through a quick patch, identified a reviewer – the legendary sdwilsh -  seemingly managing to r- the patch myself before I figured out how to correctly ask for review. Just over an hour later, some bugmail appeared. My patch was reviewed! Review-. Chastened, embarrassed, angry (only at myself) I wasn’t going to let that happen again.

There was no turning back now. I was hooked.

More bugs fell. Editbugs privileges were attained. I watched bugs I didn’t know how to fix, to watch those who did at work. (Likewise, I watched a fellow new contributor who was accomplishing great things; they call him Ehsan). After positive reviews, I waited breathlessly for someone to sweep through the checkin-needed queue and land my patch, for which I was eternally grateful. I watched in horror as one patch turned the tree extremely orange, when 99% of Mozilla was in Whistler without power, leaving me to shamefully stare for hours at an orange Tinderbox and an empty #developers channel.  (Incidentally, the further disaster that befell that summit produced my favourite bug).  Coupled with the switch to Mercurial, it felt like it was time to get commit access. I started paying attention to the world of sheriffs, random orange and landing. I learned how to decipher the crazy page of coloured rectangles and cryptic messages called Tinderbox, silently booing whenever a developer was called out on IRC when they checked in on orange. I filed for commit access, posted my committer agreement to Mountain View. My chest puffed with pride as assorted Mozilla luminaries wandered by to vouch for my approval. Typing ‘hg push’ into my own terminal, and watch firebot announce my checkin was quite the moment. It was, however, somewhat ruined – I was called out on IRC – gah, I’d lost the moral high ground, having not noticed a Moth run had just went orange before my push.

My Mozilla education continued. I explored new areas of the codebase. I devoured Planet Mozilla. I discovered the Mozilla QDB – and understood the in-jokes! I bantered and raced with philor  and gavin to star those pesky oranges (and for you youngsters, this was before TBPL suggested bugs!) . I started to eagerly follow Mozilla as the mission focused organisation, rather than thinking of it simply as the source code I worked on. I stared in disbelief when I  witnessed the demo of Flight Of the Navigator – this is all in-browser? This is the web? And the adventures continued…

Reflections

So, what have I accomplished? Lots, but not enough. Apparently, Find Toolbar is still my favourite component (although favourite is surely not the right word), with 22.64% of my Bugzilla contributions. (Incidentally, I thought others may be equally curious to check that data for themselves – you can check your own favourite component). I’m glad I’ve challenged myself when it comes to where I work in the codebase.  I knew early on that I’d steer clear of layout – there was an area that scared the hell out of me. Though apparently not enough to stop me from later writing some small layout patches. I curiously read Boris Zbarsky’s appeal for someone to fix bug 215405, but knew I certainly wouldn’t be me. Except it was. It burrowed into my brain, and wouldn’t let go. (Pro-tip: if you’re work has bz vocally celebrating in IRC, you’re doing something very, very right).

I moved beyond Firefox. m-cMerge took far, far too long to finish, occupying an unhealthy chunk of the year. However, it’s now the go-to tool for inbound sheriffs when merging, successful far beyond what I’d hoped. It also makes me feel an affinity with the A-team, who surely have the coolest group name in Mozilla!

However, I’m still fairly anonymous and unknown within the community. The 2010 summit is my biggest regret. It was awesome – scratch that, it was a mind-blowing, crazy, spectacular cavalcade of awesomeness. It was truly great to put names to faces, and witness the fierce burning passion for Mozilla in every attendee. However, I was too quiet, too shy; I know I didn’t make the most of such an amazing opportunity. And I still owe roc a game of Settlers.

The future’s bright though – joining Vidyo for the coding contribute group meeting was a big step. (Incidentally, I’ve completed 9-11 of the conversion points – why a range? For two of them it depends how they’re measured . This will be my first significant non-code venture. Also worth noting, I simply showed up, unannounced, and was immediately welcomed. “Open” is an adjective that applies to more than just Mozilla code. The opportunities and possibilites for involvement are truly endless!

Involvement is as rewarding, as intoxicating, as fulfilling today as it was in 2007. There’s plenty more to come! (And if you know anyone looking to hire someone with 5 years Mozilla hacking experience, please send them my way!)

I didn’t get here on my own: I truly stand on the shoulders of giants. Reviewers, bug commenters offering advice, those engaging in random #developers chat… I’m fortunate to have been helped along the way by so many Mozillians. I should call out Gavin Sharp, the inimitable gavinbot, who seems to have the misfortune of dealing with me the most. Thanks you, everyone!

Anyway, enough chat. Let the next 5 years begin…

Separately there is an ongoing focus within Mozilla to grow the Mozilla community and increase the number of contributors to all areas of the project. As a volunteer contributor, this is close to my heart. It may be one of the most important efforts underway within Mozilla. The coding contribute group are wrestling with how to attract, retain and reward coders to the various Mozilla software products. One area of focus is tracking contributors as they reach important milestones in their Mozilla career, and issuing rewards for certain milestones. Open badges have been raised in that context, as a potential reward for certain milestones.

Badges for releases

Aside from individual accomplishments, we hit an important milestone every 6 weeks: a new version of Firefox is released to the world, containing hundreds of patches submitted by contributors in the previous 18 weeks. When your code reaches millions of Firefox users…well, that’s an achievement. That is surely something worth celebrating. Why not issue a badge to anyone who contributed new code to a particular release? By waiting until the release is out the door, we can capture all relevant contributions, rather than trying to measure a moving target like Aurora and Beta. Such a badge could have real value, particularly if backed by evidence – currently if a third-party wants proof that someone contributed, the unattractive options available to them are grepping “hg log” or learning to query Bugzilla. A per-release badge might even be a subtle tool for contributor retention – contributors might aim to earn a badge for every release. Gotta catch ‘em all!

Assessing whether someone has earned a badge for a particular release should be trivial. It’s simply a Bugzilla query – though we need to make sure it’s the right query. This made me curious about the badge issuing process; I was pleased to find it fairly straightforward. In fact, I was quickly able to put together a simple proof of concept, a simple site that will issue a badge to anyone who contributed new code to Firefox 17, with the caveat that the earner will be mightly disappointed when faced with my terrible badge-drawing skills!

Proof of concept

To receive a badge for Firefox 17 contributions, visit graememcc.co.uk/badger and enter your bugmail.

Enter the email address used on bugzilla.mozilla.org

The site then queries Bugzilla, querying if you landed code within the relevant timeframe. The specific criteria are:

• Assignee == supplied bugmail
• Resolution == FIXED
• One of:
  • Target Milestone == Firefox 17
  • Target Milestone == mozilla17
  • status-firefox17 == fixed

If this search finds qualifying bugs, you will be sent to the badge page, complete with awful badge artwork (patches welcome!):

A badge is issued – if you earned it! (Note, the badge issued now is actually for Firefox 17 contributions)

The button below the badge allows you to add it to your Open Badge backpack, by signing in to their backpack via Persona, and creating a backpack if necessary. The Open Badges Issuer API takes care of these steps. If you view the badge in your backpack, you can see the associated metadata:

Viewing the badge on beta.openbadges.org. Note the evidence field.

An Open Badge assertion contains various pieces of metadata associated with the badge. The most interesting field, the one which gives badges real practical value, is “evidence”. This allows the badge issuer to attach proof that the badge earner has acquired the knowledge or participated in a way that merits a particular badge. Given access to this badge is judged on Bugzilla data, it makes sense to present this data as the evidence. The assertion for your badge is a URL for a Bugzilla query to list all your code contributions to Firefox 17.

Again, note this is just a proof of concept; the badge will expire 28 days after it is issued.

Users that didn’t earn a badge, are directed to a page suggesting they visit whatcanidoformozilla.org to get started!

Feedback

I hope you’ll give Badger a quick spin – and I hope you earned a badge! Remember, this isn’t an official Mozilla badge – such a badge would be far, far prettier for starters – however, the evidence in your badge backpack will speak for itself. I’m particularly interested to here from anyone who did contribute code to Firefox 17 yet doesn’t receive a badge. The search criteria above should be sufficient, however I definitely want to know if they need refining. I would love to hear people’s opinions as to whether this is something worth exploring further.

The code is on github: https://github.com/graememcc/badger

This sounds like a job for an add-on.

My add-on in action

And so I built one. The input is conveniently autofocused, so one can just Ctrl-T and start the search. I guess I’ll eventually want to make things like the search engine configurable, and it’s not particularly pretty, but for now, it’s a great help!

Edit: the add-on is now up on addons.mozilla.org.

Bug Assignees

m-cMerge will now attempt to set the bug assignee in certain circumstances. Specifically, the following must hold:

• The current assignee for the bug is nobody@mozilla.org
• If there is more than one changeset for a particular bug, the changeset author must be the same for each of the bug’s changesets
• The changeset author email must be a valid Bugzilla user email

The summary page will highlight any bugs where the assignee was set:

When m-cMerge has set the assignee, it will highlight this

In-Testsuite Flag

m-cMerge has been updated to allow you to set the in-testsuite flag. Note m-cMerge doesn’t make any predictions, or in any way try to automatically set the flag: this is completely manual.

The flag has been added below the “Comment/Resolve” checkboxes

The flag appears next to the checkboxes for commenting/resolving a bug – provided the bug belongs to a component that supports the flag. Initially, it will show the current value of the flag.

To allow you to judge whether the flag should be set, you can now view the files for a particular changeset.

You can view the files affected by a particular changeset

Simply click View/hide files to list the affected files. You can of course also click on the changeset ID to open the changeset diff on hg.mozilla.org in a new tab/window.

Well, now the rules are clear, so m-cMerge has been brought in to line.

As has been previously reported, m-cMerge can now handle pretty much any tree one would care to throw at it. Most of these trees are unaffected, but m-cMerge now has some more work to do for mozilla-central, -aurora, -beta, -release, and -esrN (and the equivalent comm- trees). When you give it a changeset for one of those trees, m-cMerge will calculate the correct tracking and status flags for that tree and changeset – for example, tracking-firefox18 or status-thunderbird-esr10. Later, if it finds the relevant tracking flag set, then it will set the status flag when posting to the bug, where appropriate.

Note this means m-cMerge is no longer purely client-side JS: there’s now some server-side logic to crawl the relevant repo to find the version.

So, when does m-cMerge think it appropriate? I took the view that there should be some sort of assertion of a bug fix being sent to the bug. For -aurora, -beta, -release, and -esr, this means there should be at least one comment being sent to the tracked bug. For mozilla-central and comm-central, I added the additional constraint that you must have selected to resolve the bug. (We can’t rely on that constraint for the other trees, as the patches might be backports, and thus already resolved). Additionally, the status flag must be either unset or set to ‘affected’ – m-cMerge will refuse to change the status flag if it is set to any other value.

The help text at the top of the page will be dynamically updated as you select/deselect “resolve” and “comment” checkboxes, allowing you to keep track of which bugs will have their status flag set on submission.

The help text highlights relevant bugs

As always, I hope this is a useful change. Feel free to report any issues you find – thanks to edmorley for reporting this bug!  You can read about previous m-cMerge updates here.

I’ve now taken this to its logical conclusion, and added the ability for m-cMerge to handle backouts as well. You often see sweeping backouts on mozilla-inbound, trying to get the tree back to a green state. Most of the time each of those bugs will be getting commented with the same information – a reason for the backout, and the rev URLs of the backout changesets. That sounds perfect for automation!

m-cMerge will now handle backouts in certain circumstances. Specifically, there can be no fixes landing in the same push, or a fix and its backout landing in the same push. I don’t think we lose anything with these restrictions – you’re unlikely to be landing new code as well as trying to clean up the tree. Equally, you’re unlikely to land a fix and back it out again in the same push – the only scenario where that could happen is a merge, and the backout commenting should already have been done when the backout hit the integration repo.

When given a backout revision, m-cMerge will attempt to attach the relevant bugs to pushes, as normal.

m-cMerge will detect the bugs affected by the backout

As always, the “Add”, “Remove” and “Change” buttons allow for manual review, and correction of anything that has been misdetected.

If you are dealing with a mozilla-central backout, you will be offered the opportunity to reopen the bug.

RESOLVED FIXED? Let me reopen that for you.

Note in this case the “comment” and “reopen” options are linked – you wouldn’t want to reopen a bug without explanation, nor would you post the backout revisions without making the patch author aware he/she was backed out.

You can manually edit the comments if you wish. However, if you need to add the same backout explanation to every bug, there is another solution. When you hit submit, m-cMerge will look at all the bugs whose comment still starts with the changeset URL, and for those, it will present you with the following dialog:

Adding the same explanation to multiple bugs is easy

This will allow you to add some explanatory text prior to the changeset URL. Checking the checkbox will add the same text to any unedited bug comments that have not yet been submitted, and won’t prompt you again. Leaving it unchecked will prompt you to enter text for the next bug when it is ready to be submitted. Hitting ESC or Cancel will transmit the change for this bug without any additional comment text. Leave the textarea empty and check the box to send all comments as they are.

As always, report any bugs in the BitBucket bug tracker. I hope this proves useful.

When pushing, you should now see something like this:

graememcc@Vitalstatistix:~/moz/hghooks/fakes/mcclone$ hg push
pushing to /home/graememcc/moz/hghooks/fakes/mozilla-central
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
You can view your change at the following URL:

https://hg.mozilla.org/mozilla-central/rev/b945bf7017be

If you’re pushing a large number of changes, the individual URLs cease to be useful. Instead, you will see:

You can view the pushlog for your changes at the following URL:

https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=4464ac64628a

Admittedly this won’t directly, help you with marking bugs, but it’s perfect for pasting into m-cMerge for automating the bug commenting (m-cMerge can now handle general commits to any tree not just mozilla-central).

Finally, when pushing to Try, the results of your builds are more relevant. Hence:

You can view the progress of your build at the following URL:

https://tbpl.mozilla.org/?tree=Try&rev=15ecf264d328

I hope you find these changes useful!

Wait a minute, after the merge the sheriffs did what?!?

We have some smart people acting as inbound sheriffs. This does not seem like the best use of their time. It always bugged me, even though I’ve never merged myself. I wanted to do something about it. However, I felt it would be difficult to fully automate – even with our current restrictions, a wide variety of commit messages is possible. Accurately parsing them 100% of the time would be…hard. Throw in security bugs, and inbound’s propensity to get so hosed that it has to be closed for a bit of “BACK OUT ALL THE THINGS!”…

That said, I didn’t think all was lost. As Kernighan and Plauger said, most users will be willing to meet you halfway, and be ecstatic if you manage 90% of the work.  I decided to see if we could get 90% of the way there.

m-cMerge

m-cMerge is the fruit of that labour. It lives at http://www.graememcc.co.uk/m-cmerge/, with the source available at https://bitbucket.org/graememcc/m-cmerge. Designed to be used immediately following a merge to mozilla-central, it will load the specified pushlog, and attempt to work out what fixes made it, what got backed out, and separate out merge changesets, no-bug changesets and other noise. One can then manually review, correct any misdetections, and hit submit to have the relevant changes made to Bugzilla.

To begin, enter the changeset hash of the merge:

First, enter the relevant changeset

m-cMerge gets busy, loading the pushlog, categorising the changesets, and loading the relevant bug data. The results will be split into 5 categories:

• Fixes (pushes that stuck!)
• Pushes backed out
• Backouts of things not in this merge
• Merge changesets
• Other changesets

If a particular merge does not contain any pushes of a particular type, it will be omitted.

Pushes are categorised into one of five categories

The text at the top calls out various special cases, for example bugs with “leave open” in the whiteboard (which default to comment without resolving), and security bugs. Note that you cannot resolve or comment in security bugs in m-cMerge – I feel those are best altered by human hands. Underneath, you will find the relevant pushes, ordered from newest first, to mirror the pushlog, with any bugs that m-cMerge believes should be commented immediately below.

Most of the controls should hopefully be self-explanatory. If the “Comment:” checkbox is checked, a comment will be posted in the bug. If the “Resolve:”  flag is checked, the bug will be marked as RESOLVED FIXED. Additionally, when resolving, one can change the target milestone, by selecting from the drop-down menu. Where possible, m-cMerge will suggest a sensible value for the target milestone. You cannot set the target milestone when only commenting; if the bug is not being resolved at this point, I do not believe you can make any claims about the milestone.

Comments

Click “View/hide comment” to view the suggested comment for a particular changeset, and verify it is correct. You can amend the comment as necessary.

Clicking view/hide shows a textarea which allows you to edit the comment

If there are a series of changesets all associated with the same bug, i.e. you have pushes of the form “Bug XXX – Part 1: fix something”, ”Bug XXX – Part 2: fix something else” and so on, the comments will be merged into a single comment on submission, with the comment for the oldest changeset first.

The comment checkbox works on a per changeset basis: if you have 3 changesets associated with one bug, and deselect the comment option on one changeset, the other two will still be commented unless you instruct otherwise. In contrast, the resolve checkbox works on a per-bug basis, affecting all instances of that bug.

Mis-detected bugs

Only 90% of the way, remember? Due to the wide variety of commit messages, m-cMerge will sometimes get it wrong, particularly when a commit message references multiple bugs. Next to each bug are “remove” and “change” buttons, to allow you to disassociate a bug, or correct which bug will be resolved.

Not m-cMerge’s fault (for once!). Should have been 763238. Hit the “change” button!

Additionally, the “add bug” button, does the obvious thing, allowing you to send a comment about a particular changeset to multiple bugs.

Submitting

When you’ve reviewed all the changes, scroll to the bottom to find your friendly neighbourhood “submit” button. You will then be asked to supply your Bugzilla credentials, which will then be used to communicate with the Bugzilla REST API over https, and make the specified changes.

The submit button reverts the tree to Firefox 3.5. Or something.

Backouts

The creativity in commit messages reaches whole new levels when it gets to backouts (pro-tip: flick to the pushes backed out section to check for any misdetections before submitting any fixes). Where possible, m-cMerge will associate the pushes backed out with the thing that backed them out.

Pushes backed out are associated with the thing that backed them out.

Both backouts and the pushes backed out default to neither commenting or resolving, but that can of course be changed via the controls provided. As seen in the picture above, m-cMerge can cope with a push that backs out multiple other pushes. Indeed, it should cope with ranged backouts e.g. ‘backout aaabbbcccddd-111222333444′ or ’backout aaabbbcccddd through 111222333444′ , and ’backout aaabbbcccddd : 111222333444′. When you messy pups really hose the tree, it can deal with ‘revert tree to changeset aaabbbcccddd’. For normal backouts, it understands ‘revert’ is a synonym for ‘backout’.

m-cMerge’s backout detection really is only as good as the commit messages. I would encourage the use of changeset hashes in the backout commit message to specify exactly what was removed.

Summary

After all the individual sections, m-cMerge provides a handy summary of the damage unleashed on Bugzilla.

The summary shows what changes were submitted

If it looks like you forgot to submit, m-cMerge will issue a warning.

Red text! Looks like we have some kind of warning.

It will also nag you about your part of the deal: resolving security bugs.

Don’t forget the security bugs

Patches welcome

I hope this is enough to be useful to inbound sheriffs. It could be prettier though. I’m not entirely sure how best to present the information in the summary; the dialogs use an off-the-shelf jQuery theme, and don’t tie in with the rest of the site. The “loading” message should stay centered on the screen when you scroll, and I thought I should just have been able to set “position: fixed;” on it – instead it defeated me until the bitter end. And of course, with all your prodding and poking, you’ll likely find other bugs: on the other hand, you’ll now have lots of free time to fix them, now that you’re not resolving bugs all day!

He could have said “yes, I’ve set up an appropriate configuration for you”. Or even “it’s possible, but it’s not something I have time to deal with”. Instead he effectively said “I’m just going to trust you to do the right thing“.   Ignoring the specifics of the request, I feel that is quite something. Gerv and I don’t intersect much within the project, and I’m still not particularly visible within Mozilla – I’m not a module owner, or a peer, or a MoCo employee. Those facts alone would be enough to inspire caution. And yet he defaulted to “you’re a fellow Mozillian; I can trust you”. That is rewarding on a pretty deep level, to receive an unexpected affirmation that not only are you fixing bugs, you’re quietly earning the trust and respect of your peers.

Just another reason why being a member of the Mozilla community is so great.

• Bug 686203 – Focused contenteditable can prevent other elements from getting focused on right-click, which also fixed bug 578771 and possibly more!
• Bug 740784 – Undo (Ctrl+z) in textarea adding a newline (\n) to the text
• Bug 747163 – TelemetryHistogramType returns failure for HISTOGRAM_FLAG
• Bug 747379 – Cloning a flag histogram with Telemetry::HistogramFrom breaks the “only one count” invariant

Currently awaiting reviews for:

• Bug 663585 – push hooks should print out hg url for the changeset
• Bug 724513 – Startupcache can load invalid cache if unable to delete startupCache file

I’m also still hard at work on my as yet undisclosed web app. To be concluded and disclosed soon, I promise!

As I find the model of education interesting, I’ve also been taking the courses over at Udacity. As a computing science graduate, I’m relieved to report I got the highest possible award on CS101! I’m currently following their CS212 class – because Peter Norvig! – and CS262 – because, well these courses are slightly addictive. I do find myself wanting to fix up various annoyances and bugs on their site though!