Skip to content

Instantly share code, notes, and snippets.

@zbraniecki

zbraniecki/README.md

Last active May 10, 2017 21:40
Show Gist options
  • Save zbraniecki/15897258e01ad6582c2110afccbc4576 to your computer and use it in GitHub Desktop.
Save zbraniecki/15897258e01ad6582c2110afccbc4576 to your computer and use it in GitHub Desktop.
Download ZIP
Building Fennec for DLC
Raw
README.md

Intro

Getting DLC support in Fennec requires us to change a couple things about how Gecko retrieves and negotiates locales. In particular, it requires us to introduce a new resource registry specifically designed for localization purposes.

The side effect of that, is that until we finish the transition, we'll have two registries storing resources -

  • Old one called "Chrome Registry"
  • New one called "L10n Registry"

We will only be able to get DLC hooked into the new system, which means that we have to recognize that for the time being, Chrome Registry will have "all" locales that we want to make available, while L10n Registry will have only a few and will receive additional ones as we install the DLCs.

We have a solid language negotiation logic that will allow us to make sure that we end up with a language that both registries have resources for, but for the time being, we did not hook it yet completely, to allow us to work on the system.

Steps

Compile Firefox with new logic

First step is to pull the hg bookmark for where the work is going on. I'll describe the steps as I would take them, and feel free to adapt them to your setup:

The bookmark lives in https://bitbucket.org/zbraniecki/mozilla-unified/branches/compare/fennec%0Dcentral It currently has multiple patches, that we're landing in mozilla-central one by one. In the end, only the last one will be blocked on us getting the whole system working, the rest is just introducing new APIs and cleaning up the callsites around Gecko to work with proper language selections.

  1. Add "bitbucket=https://bitbucket.org/zbraniecki/mozilla-unified" to your [paths] in .hg/hgrc
  2. hg book fennec
  3. hg pull -u bitbucket -B fennec
  4. Add to your mozconfig:
mk_add_options 'export MOZ_CHROME_MULTILOCALE=en-US pl'
mk_add_options 'export L10NBASEDIR=/home/zbraniecki/projects/l10n-central'
ac_add_options --with-l10n-base="/home/zbraniecki/projects/l10n-central"
  1. ./mach build

Merge locales

In my build, I use one locale that is bundled in the .apk - en-US, and one that ChromeRegistry has strings for, and L10nRegistry is supposed to retrieve from DLC: pl.

Becuase DLC is not ready yet, I am mocking the DLC downloading, until we get this bit working. I'll explain it in detail later.

  1. create a directory parallel to "mozilla-central" one called "l10n-central"
  2. Enter it and clone https://hg.mozilla.org/l10n-central/pl/ there
  3. Add a file https://gist.github.com/zbraniecki/ef75fa40c0fddec80f2fee5af051e899
  4. Switch to mozilla-central directory
  5. for loc in pl; do make -C ./obj-arm-linux-androideabi/mobile/android/locales merge-$loc LOCALE_MERGEDIR=/Users/zbraniecki/src/larch/obj-arm-linux-androideabi/merge-$loc; make -C obj-arm-linux-androideabi/mobile/android/locales chrome-$loc LOCALE_MERGEDIR=/Users/zbraniecki/src/larch/obj-arm-linux-androideabi/merge-$loc; done;

At this point, you have Fennec with two languages - 'pl' and 'en-US'. The languages are registered in multilocale.json file, so Fennec picker will show them, and both are bundled in the package.

Package & Install

  1. ./mach package
  2. ./mach install

Test the result

  1. Open Fennec
  2. Make sure that the language is "en-US"
  3. Open Tools>Addons

You should now see the Addons page localized with the new localization API served via L10nRegistry. All in english. The title should say L20n Your Add-ons.

Now, if you go to Settings and switch locale to pl, you should see most of the UI in Polish, but if you open Tools>Addons, this document's title will still be in English.

The reason for that is that I artificially limited L10nRegistry locales to en-US in my last patch on the fennec branch. That means that L10nRegistry will always negotiate locales down to en-US until we install the "DLC".

The next step is to connect the WebIDE and "trigger DLC download". In reality, the DLC will be triggered by language selection or by post-install code that will check OS locale and download the matching locale from DLC.

But for now, we trigger it manually. In the WebIDE, main process or Add-ons tab, do:

  1. const { L10nRegistry, IndexedFileSource } = Components.utils.import("resource://gre/modules/L10nRegistry.jsm", {});
  2. L10nRegistry.getAvailableLocales(); // should return ["en-US"]
  const src = new IndexedFileSource(
    'langpack-pl@mozilla.org',
    ['pl'],
    'resource:///localization/{locale}/',
    [
      'resource:///localization/pl/mobile/aboutAddons.ftl',
    ]
  );
  1. L10nRegistry.registerSource(src);
  2. L10nRegistry.getAvailableLocales(); // should return ["pl", "en-US"]

The moment you execute step (4) you will see the language switch to 'pl' in the About Addons document in Fennec.

If you got that working, the setup is complete, and we can start working on plugging DLC into the system. If you did not end up with the right responses from Gecko, let me know and we'll get you here.

Technical details

Ok, so what is the patchset and what happened above from the code perspective.

Patches

The patches are divided into two buckets:

  1. MessageContext, L10nRegistry and Localization API - the first batch of patches adds L10nRegistry, which is a pluggable registry for localization resources, and adds a new Localization API that binds into DOM and L10nRegistry to localize UI.
  2. The last patch is the actual patch we need to land to switch Fennec to the new system. It's the only one Fennec developers will have to deal with. In this patch I'm also artifically limiting the resources registered in L10nRegistry for this demo purposes.

Now, because we're working not on Fennec localization, but plugging DLC into L10nRegistry, I need to explain more than just the last patch.

FileSource

In L10nRegistry terms, we have so called sources. Each source has a name, locales it provides and a path. DLC will register new sources for each langpack" it registers.

The first one, registered everywhere is a FileSource which is added to L10nRegistry in intl/l10n/L10nRegistry.jsm at the bottom.

As you can see, it's a pretty simple object that says that the name of the source is app, it has en-US locale, and it's base path to resources is resource://localization/{locale}/.

That's all L10nRegistry needs - from that moment it knows that it has resources for en-US and that's what it returns when asked (L10nRegistry.getAvailableLocales()). That in turn is used by Localization API to negotiate against user requested languages.

As I said, for now, we don't impose the limitation outside of this Addons tab, so you can freely select any of the languages you have in Fennec and both the Java UI and all the Gecko except of this one tab will follow.

But Addons tab is now localized using the new Localization API, so whatever language you select, it'll check what locale it has in L10nRegistry and snce we only registered en-US, that's the locale that it will use.

Adding an IndexedFileSource

This one is similar to FileSource except that it also lists the absolute resource paths. My hope is that you will be able to retrieve the list of resources that a DLC brings and provide it to the L10nRegistry. If not, we could skip it, and use FileSource, but I'm afraid about performance cost of probing I/O on files from each addon base path first. With the list of resources, if a user asks for a file that the langpack doesn't have, L10nRegistry knows it without any I/O.

Once you register the IndexedFileSource, L10nRegistry fires an event l10n:available-languages-changed, which the LocaleService API is listening to. It reacts to it by renegotiating languages between L10nRegistry and requested locales from the user and if the result list is different from the old one, it fires an event called intl:app-locales-changed. Each Localization object listens to this one and reacts by triggering retranslation of the document into the new language.

That's basically all!

Plugging DLC

I imagine DLC system to plug itself by registering its resources into L10nRegistry as soon as the new package is downloaded and then at every app startup before any UI is displayed.

This means that when the first page wants to negotiate languages, L10nRegistry will already have all sources registered and we can reason about what language we should use.

When DLC package gets updated, L10nRegistry has updateSource method, and I imagine that eventually we'll also add removeSource for the case when DLC will delete a package.

Next steps

As I said, I'm working on getting the patchset reduced by landing pieces of it. I'd like to get to the point where we have L10nRegistry landed in Gecko within a couple weeks, so that the work on DLC can happen against mozilla-central, and not my bookmark.

I'd like you to take over the part where DLC gets plugged into L10nRegistry bug 1347802. It's a good moment to iron out any potential problems between DLC functionality and L10nRegistry API.

If we can get this working, I believe the rest will be rather straightforward as we already have running code with good performance characteristic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment