Upstreaming patches to android-sync for people who like to pretend android-sync doesn’t exist

I find it rather irritating to have to keep manually upstreaming things to android-sync, and nobody seemed to have automated this.

Behold: a magical script for this. Unfortunately it uses mq, but it shouldn’t be a big deal to refactor it for less terrible ways of doing things.

So, you’ve written a patch that touches both Fennec and a-s code. You got it reviewed and now you’ve landed it on fx-team. It’s time to upstream the a-s changes, ideally before someone starts shouting at you for the merge failures you’ve created.

qpush the mixed patch, invoke the script, and copy-paste the pull request url it prints into bugzilla. It automagically sucks the a-s-specific stuff out of your patch, creates a branch named bug-$NUMBER_FROM_COMMIT_MESSAGE-$NAME_OF_PATCH_FILE with it and issues a pull request on a-s (leaving your original patch unchanged).


  • hub A wrapper script for Git that makes interacting with GitHub much nicer. If you use GitHub you should already be using this. `git pull-request` becomes a thing.
  • moz-git-tools A bunch of handy scripts for working with Git at Mozilla. All we want from it is hg-patch-to-git-patch. Make sure that script at least it on your path (or update my script to point to it wherever you put it).

Stick the script somewhere handy, make sure you set up the config variables at the top in the obvious way, and have fun.

In the event a patch doesn’t apply cleanly, you’ll have to go and manually fix it up in your local a-s repo using `git am` in the usual way, before continuing. The script pauses after trying to apply the patch so you have a chance to do this before it finishes up and publishes anything.

The script (apparently my blog mangles inline code rather horrifyingly):


Preventing mercurial from eating itself

Mercurial doesn’t seem to handle being interrupted particularly well. That, coupled with my tendency to hit ^C when I do something stupid leads to Mercurial ending up in an inconsistent state about once a week, causing me to manually restore my patch queue state with judicious use of `strip` (or otherwise).

I’ve just about had enough of this. A simple hack (that hopefully others will find useful) is to alias `hg` to a script containing:

hg “$@”&

exit $!

Sort of evil hack, but does the job (provided you don’t kill the terminal you’re typing in: but I’m yet to meet someone who is suffering from reflexively executing `exit`. Now you can ^C Mercurial all you want and it will blithely ignore you. This seems preferable to it half-doing something, throwing a tantrum, and eating itself…


A novel (well, not really) way of optimising nine-patches

While tinkering with shrinking Fennec’s apk size, I came across a potential optimisation for nine-patches that’s oddly missing from Android’s resource preprocessing steps.


There’s a pretty decent explanation of what nine-patches are over here.


When a nine-patch contains multiple scalable regions along an axis, the system guarantees to maintain their relative size (though no promises are made about aspect ratio of the entire image).

In the simpler (and more common) case where only a single scalable region exists along an axis, if all the columns (or rows) are identical, they may safely be collapsed to a single column (or row). The renderer will happily later upscale this single-pixel region as much as necessary with identical results.

Apparently this isn’t something that’s done automagically during the packaging step, so I’ve written a neat (and slightly badly-structured) utility for performing this optimisation, available here:

It takes a list of .9.png files as input and overwrites them with optimised versions (if the optimisation is found to be safe. It does not currently perform any optimisation where multiple scalable regions exist, even though it should be safe to do so under some circumstances (if the regions each consist of homogenous rows/columns and you reduce their size in a way that preserve their relative size)).

Note that this program doesn’t preserve PNG indexes, so if you feed it an indexed png you’ll get an unindexed ARGB image as the result (causing the file to grow in size). Simply repeat your png quantisation step to obtain your optimised image.


Bludgeoning IntelliJ IDEA into being useful for mobile Firefox development

IntelliJ IDEA is my favourite IDE. It is truly terrifying – it’ll blithely pop up with insights like “This seems like an inappropriate assignment” based on the names of the variables involved, or “This boolean logic can be simplified”, or “This might throw a NullPointerException”. It has a level of code understanding far deeper than any other IDE I have seen. Writing dodgy code using it is made much harder. Write some code with possibly unclosed resources – IDEA will shout at you. Write some particularly dodgy synchronization code – IDEA will shout at you. Programming with IDEA is like having an extremely OCD version of Glados peering over your shoulder continually explaining in great detail in exactly what ways you’re an entirely incompetent programmer. It’s great!

The slight problem is that the Fennec codebase is… eccentric, bordering on insane. Most IDEs just flatly refuse to understand it – but IDEA is different. Sure, configuring it to understand Fennec involves much faffing, but since IDEA understands all the programming languages involved, it should be possible to convince it to understand the entire Mozilla Central codebase.

Alas, I am  but a relatively inexperienced intern. I can provide specific instructions for getting the areas of the code I’ve been dealing with up and running on IDEA. The rest is left as an exercise for the reader :P. (And it seems like it would be a pointful thing to set up. No more patch “nits” if we configure the styling correctly, no more of certain classes of silly bug, and no more developer frustration when a reviewer says “I like it, but I want you to rename absolutely all the variables” (In IDEA: Put cursor on occurance of variable to be renamed. Push SHIFT+F6. Type new name. Push enter. All references in code, references using Reflection with a string literal, and Javadoc references are updated automatically. Remaining references in comments are presented in a prompt for your approval before updating. You see why I like this IDE now?))

So, let’s begin…

Getting started

Head over to and grab yourself a copy.

IntelliJ has a frightening number of extensions which collectively provide support for a disturbing number of programming technologies and inspections. I encourage you to enable as many of the inspection packs as possible (More OCDness is better, generally). For Fennec development, you will need to enable the C/C++ and the Mercurial extensions. Other parts of Mozilla Central will probably also require the Python support.

You should disable as many of the bundled plugins as you don’t want (You’re prompted for desired plugins on your first run). Enabling lots has a drastic effect on startup speed of the IDE.


If you are new to mobile firefox development, proceed below. Else jump to “Creating IDEA project over existing m-c” (Which you should subsequently skip if you are new to mobile firefox development)


Clone fresh M-C

After loading the IDE, you are presented with the welcome screen:


If this is not your first time running the IDE, you may need to close an open project first. From here, select “Check out from Version Control”. For the repository URL, enter:

Note that after pulling, the code will be placed in a directory named $”Directory name” in the parent directory specified. Select a sane name – this is also going to be the name of the IDEA project (Although that is not made clear at this point).

You will be asked if you want to create an IDEA project from the sources. Say yes, and jump to “Initial project configuration”

Creating IDEA Project Over Existing M-C

If you already have the Mozilla-central source downloaded, there is no need to redownload it. From the welcome screen, select “Import Project”, select the root directory of m-c, and you should end up in the same place as those who took the other branch. Proceed to “Initial project configuration”

Initial Project Configuration

You should find yourself in a wizard asking if you want to create a project from existing sources or import a project from an external model. You want the former.

Ensure the name/location options on the next screen match the name of the directory the code is in.

The next screen will present the source directories IDEA autodetected. My suggested configuration is thus:


As the test sources should be added seperately as such from the project structure screen later on. (I’m not very familiar with WebRTC code, so it’s not clear if this is correct for developers working on that. Hopefully by the end of this post you’ll be able to figure it out for yourself).

The next screen states the libraries IDEA autodetected in the project. As far as I can tell, every single one of these is incorrect – I unticked the lot:Image
We do need some libraries, but we’ll be adding them manually later on.

Next we have the module structure page:


The autodetection here is not perfect. The one called “android” is approximately correct for the majority of the Fennec Java code. It seems to have autodetected the WebRTC stuff as two modules, and much of the rest of the code there is a mystery to me.

Just accept the defaults and move on. Module structure tweakage is the main route to getting things working in IDEA anyway.


Next you are prompted to select project SDK. It doesn’t really matter which one you select here, since you’ll be needing to select SDK on a per-module basis later (Since different modules in different languages will need different SDKs anyway). The one you choose will be the default for modules. I selected Android 4.2.2. Ensure you have the JDK, ASDK, ANDK, and CPPSDK installed and properly configured here. (And possibly Python, too)Image

It is of value, but not required, to also obtain the source code and documentation for the SDKs in use and add these on the sourcepath and documentation tabs of the SDKs in question. If you do this, IntelliJ is able to help you more – it can autocomplete with information derived from reading the documentation, it will show documentation in completion tooltips, etc.

IDEA now autodetects two Android frameworks. Correctly, for a change.


After pushing finish, the project should load. You will most likely now get a warning about having C/C++ code outside of source roots. This is because the autodection wizard we just went through is not sufficiently magical to get everything correct, and it doesn’t let us manually configure our source roots and suchlike. We’re about to fix these problems, at least to some extent, shortly.
Choose “Cancel” from the dialog. The autodetection it attempts to do if you select yes is entirely unhelpful. You will need to manually configure the C++ module later on.

You should now be presented with a promising looking project-is-loaded screen:


Resist the urge to open files and start doing things. IDEA’s default configuration is a bit… overzealous. It starts optimising imports and the like the moment you touch a file – a sure fire way to annoy your patch reviewers. By default, IDEA likes to conglomerate imports from the same package into a single, tidier, import somepackage.* line. Go to file -> Settings. Under Project Settings, go to Code Style -> Java. Select the imports tab, and set it somewhat like this:


You now have an existent, but extremely broke IntelliJ project for Fennec. Much reconfiguration is needed.

Unbreaking the Configuration

If you now proceed to open up one of the Java source files, it becomes immediately obvious that we’re not done yet. Before starting with the next step, open up a terminal and
get mach building the project for you now (We’ll be needing the contents of the objdir in a minute)

Then, navigate to tools -> Project Structure

If Fennec has moved to Java 1.7 yet, change the language level to 7 and rejoice in not having to use so many redundant types in your diamonds.

Go to the modules tab – at the moment IDEA is unable to find a number of things that are required by the project, so it ends up painting all your code in unhappy shades of red.
We are initially interested in the unimaginatively named “android” module, which is the one which has autodetected the main bulk of Fennec.
For the purposes of this post, I’ve renamed it to “GeckoApp”.

The UI here is sort of odd. You select directories from the tree on the right and then use the buttons above the tree to mark them as either source directories, test source directories,
or excluded (Don’t analyse the content of this directory).

Mark the tests directory as containing test sources.

IntelliJ does not automatically correctly register the android support libraries. Go to the Libraries tab, click add, and navigate to extras/android/support/v4/android-support-v4.jar within your android SDK root. Add the library to the GeckoApp module. Upon hitting apply a satisfyingly large amount of the redness should disappear from the code.
Now add additional libraries for the directories platforms/android-X, for 8 <= X <= 16. Have fun with that. When complete you should be looking roughly like:


Only a small number of reds will remain. References to org.mozilla.gecko.updater.UpdateService and related classes will fail because they’re in the wrong directory (There’s a patch for this on bug 890586 which will hopefully cure this eventually).
References to R.something will also, for now, be red, so will AppConstants references. Both of these are created at compile time (Hence our running mach concurrently with you doing that extremely fun adding-of-libraries task)
You should now have an objdir-droid directory (Even if Mach is still chuntering away). Return to the modules tab, select your GeckoApp module, and click “Add content root”. Here select objdir-droid/mobile/android/base. Set every subdirectory of as “Excluded”, and base itself as a source directory. (You can multiselect items in the tree. I suggest you collapse every child node of objdir-droid in the tree before you do this, else IDEA explicitly adds exclude entries for each subdirectory, which is redundant and just clutters your config.)
Finally, there is a tiny “P” symbol adjacent to an “X” symbol adjacent to where it lists the source folder you just marked in the content root list view. Click this, and input “org.mozilla.gecko”. This is the interface for adding package prefixes to content roots in IntelliJ and is, in general, how we avoid having to munge our directory structure as is necessary for Eclipse.
Your final setup should look something like this:

If Mach is still building, your code will remain red for the time being as the generated versions of these files are not produced until quite late in the build process.
If done correctly, you should now have a mostly working setup for working on the Java parts of Fennec (Good enough to use a fair few of the Skynet-like inspections). Clearly, you should not be editing things in the objdir – this link to it as a “source directory” is just to help the IDE.


Some other helpful things to set up…

Javascript settings

Go to the top-level entry in the project settings tree for “Javacript” and set the Javascript language version as high as it will go (This prevents IDEA from getting upset at constructs in JS which aren’t valid at the default level).

Notice the existence of the JS code quality tools. Those might be worth investigating.


Code style settings

“Nits” on patches are extremely annoying. Thankfully, IntelliJ is much better at remebering to add spaces in all the right places than you are – it’s just a matter of setting it up. Configure the currently-applicable code style parameters under the project code style settings tree for each language you care about (The information is available at

Any you miss will surely become apparent after your first patch submission.

Then, before submitting a patch, go to Code->Reformat Code from the main menu, tick “Only VCS changed text”, and run. IDEA will now automagically apply the code style rules you just configured it with, but only to text you have changed, so you don’t get lots of extra changes in your diff. (Alas, IDEA’s view of “VCS changed text” is “Stuff that appears in hg diff” – so if you’ve qrefreshed some of your code into a patch already then IDEA will miss that (The source of all my “nits”, in fact).
If you’ve written a new file, you might like to also run “Rearrange code” to neaten it up somewhat.


Code for Mozilla must always contain the license boilerplate. Adding this by hand is annoying, so let’s automate that. In the settings menu, navigate to Editor->File Templates and add the appropriate mundungus to each of the existing templates. Now, whenever you create a new file using IntelliJ this boilerplate is already created (And handily folded away so you don’t have to think about it)


File Types

Mozilla uses some slightly nonstandard file extensions (Filename pattern matching is how IDEA identifies filetypes). Firstly, Mozilla’s custom preprocessor operates on source files that have the suffix “.in” appended to their filenames. So IDEA can at least attempt to process these files, add the appropriate *, *, etc. patterns in the right places.

IDEA does not by default recognise .jsm files as being Javascript files. Select the Javascript files type from the file types menu and add *.jsm as a pattern.
Our Makefile is preprocessed – under “Traditional makefiles” add “” as a pattern – syntax highlighting on will now work correctly.
Alas, the custom preprocessor directives we have on the *.in files do still confuse IDEA. Plausibly an extension could be written to correct this.. (Or we could redefine those directives to start with //…)

HG Ignore

IntelliJ creates a number of metadata files to handle project settings (a .idea directory in the root of the project and a .iml file in each module root). At time of writing, only
.idea is in Mozilla’s .hgignore – but that’s okay.
To stop hg from bothering you about the .iml files (And, in general, if you want to have your own personal .hgignore that hg won’t track), create a file at .hg/hgignore.
For IDEA, add the following:


And to the existing file at .hg/hgrc add the following at the bottom:

ignore = /full/path/to/repository/.hg/hgignore

In general, anything you add to this new hgignore file will be ignored by your local hg, but the changes to this ignore file are not tracked by the VCS, so it will never tell you
about them, either.



There exists a great deal of code I have covered in very little detail… The configuration described gets my corner of Mozilla Central working in IDEA – I would appreciate input from others on how to expand this configuration to encompass more of the codebase (It very much is possible to make one project to rule them all, so to speak, but setting up the module and facet structure required more familiarity with much of the code than I currently have.)

It is my hope that the information presented here is enough to at least get some people started on this. Certainly, the convenience gained outweighs the considerable faff to make it work.



After much dithering, I finally got round to creating a blog… It remains to be seen if I’ll actually find enough interesting content to populate it.


So, behold, a channel for the hopefully at least vaguely interesting ramblings of myself…