Open GApps logo

Happy Open GApps user? Vote for us at ProductHunt!

It ain't no fun to play with Toybox (yet)


The title might sound cryptic to most readers, but for most GApps developers 2016 did start with a lot of bug reports from CM Recovery users. Reports with errors like “I got error 255” while we don’t have such error code specified ourselves, or “mount failed with error 1”. Cause of all this trouble is a new command line tool called Toybox and the decision of Cyanogenmod that the Christmas holidays were the perfect moment to embed it in their recovery instead of Busybox.

Most users will probably wonder: what is Toybox and why did Cyanogenmod switch to it and what was Busybox in the first place?


Let’s start with the last question first: what is Busybox? Busybox is a GPL-licensed effort that offers many Unix tools (commands that offer functionality like “copy this file”, “search for this string”, “mount this partition”) in a single small binary and already exists since 1999. The concept of it is great, it offers a lot of functionality, comparable to an operating system, but within a small 2MiB binary. As such Busybox is perfect for use in embedded software or, more applicable to our situation, the Android recovery. That is why TWRP, CWM and until recently CM Recovery were all using Busybox as core provider of functionality. It is a powerful tool that enabled us to use advanced logic within the Open GApps installer.


But what is Toybox? Toybox’s goal and functionality are meant to be just like Busybox and it is even being developed by the very same person that maintained Busybox for many years (Rob Landley). But where Busybox is licensed as GPLv2, Toybox is using the BSD license. Because Google has chosen for a policy to not include any GPL in Android’s user space, it disqualified Busybox to be included in Android’s upstream AOSP recovery repository. But Toybox does have a compatible license which enables it to be upstreamed into AOSP. This is in general a very positive development, because a more powerful recovery in AOSP is very welcome.

Toybox is not yet finished though (see the current status here). While the original recovery in the AOSP project had barely any capabilities, for them the adaptation of the still under-development Toybox is a huge step forward. But the alternative recoveries were using the powerful Busybox. CM Recovery decided to switch from Busybox to Toybox. But because Toybox still does miss some essential commands that we rely on in the Open GApps installer (unzip, awk, xz) that are not implemented (yet). This breaks the Open GApps installer on CM Recoveries that are bundled with the CM13 nightlies.

CM Recovery

Why did Cyanogenmod switch? We don’t know. Of course it convenient to stay as close as possible to upstream as possible, but in our opinion Cyanogenmod has switched way too early from Busybox to Toybox for their recovery. As Open GApps we are very unhappy with the timing of their decision. We believe that at least the posix subset should have been fully implemented in Toybox before they should have switched. As Open GApps we now have the choice whether

  1. we will rewrite our code to the limited Toybox capabilities (if possible at all),
  2. defy CM Recovery as supportable effort,
  3. bundle our own Busybox in the installer,
  4. hope that Toybox will develop the missing commands at lightspeed,
  5. or that Cyanogenmod reconsiders their decision.

UPDATE: we went with option 3

An AROMA developer insight


AROMA Open Gapps (previously known as AROMA LP-GAPPS) was created at the beginning of 2015. What triggered its creation, you ask?


From a simple discussion between two friends about how awesome it would be, if you could only install those GApps that you really want in a simple manner. Because the standard packages (nano, pico, mini, stock, etc.) didn’t satisfy our needs. This sparked the idea for AROMA LP-GAPPS and raulx222 started to develop a config for AROMA-Installer and made a package based on PA GApps.

Joining Open GApps

On 1st of June, LP-GAPPS was further integrated with the Open GApps project, a new GApps project that used the same (keyword) install mechanism from the, at that point discontinued, PA GApps. This was a new beginning for LP-GAPPS and the best thing that happened to the project.

The main benefit of integration with the Open GApps project is that the AROMA packages were now build everyday containing the latest GApps, compared to the old situation where the packages were the AROMA-scripts had to be repackaged manually every release.

AROMA Installer

Now let’s talk more about AROMA Installer, an essential ingredient for the project. What is AROMA Installer?

“AROMA” was taken from Bahasa Indonesia (Indonesian Language) and it means “Scent”, but it is also an abbreviation of “AMARULLZ ANDROID ROM MANIFESTATION”. It is an advanced update-binary for Android Recovery. It contains many features, like Wizard Installation, Touch User Interface (AROMA UI), Customizable Packages, System Inspecting, Themeable, and User Interactive. - amarullz (Ahmad Amarullah)

AROMA Installer was created somewhere in 2012. This piece of software got very popular very fast and many ROMs developers were using it and still use it, because it is simply unique and very powerful. It does let users customize their installations based on their preferences through a very attractive GUI within the recovery environment.

AROMA maintenance

Somewhere in 2013 the development did almost stop completely, but AROMA Installer still works on most devices with ARM architecture.

Since then the smartphone industry also developed devices with other architectures, like x86, which are not compatible with the current AROMA Installer. But even not all ARM devices are compatible with AROMA Installer, because of various issues. But we believe these could very possibly be fixed through further development.


Since 2013 amarullz started to develop his “aromalib” which is the core component of AROMA Installer. So there’s a newer, modified aromalib of which amarullz confirmed that it would be used in the next update of AROMA Installer. In May 2015 amarullz uploaded a video with a test application of libaroma on TWRP, you can see the video here. But development of aromalib seems to go slow, which worries us as AROMA Installer users.

All we can do is to encourage amarullz to keep working on his powerful aromalib and help him to release a new update! If you appreciate his work and want to encourage further AROMA Installer development you can donate to amarullz and you can check out his other work.

AROMA Installer future

We hope that we can receive an updated AROMA Installer in the near future. But of course we did not stop our own development and hacking around AROMA’s current limitations. Sometimes we faced issues when implementing new features, because they did not work properly in the aroma package. But we managed to fix them! For example we were obliged to compile and bundle our own xzdec (for decompressing xz files on ARM devices) in order to decompress xz files within AROMA Installer. Without our own xzdec tool, using recovery’s built-in decompression tool, the AROMA Installer would failing during decompression, probably because of memory issues.

AROMA Open GApps future

In the future we are planning to make the user interface of the aroma package more dynamic per Android version. E.g. not all keywords are available in the various aroma packages. With the new Super package we did expand our keywords to cover the newest app additions. But the aroma package for 4.4 and 5.0 are still based on the stock package, because super is only available for 5.1 and 6.0. The current aroma config is still shared between these various versions and does contain all the keywords, even when they are not available. When we will make the interface more dynamic, the “unavailable keywords” can be hidden on Android versions where they are not applicable.

Marshmallow builds

Open GApps buildbot

Since the 6th of November we are releasing Marshmallow builds. The very first ones were still experimental and broken in some ways, but by now they should be pretty stable.

Permissions issues fixed

We were lucky that after our last post about Marshmallow challenges Alex Naidis did reach out to us and created a patch for ROMs to fix the described permission issues. This patch is now part of CyanogenMod and most ROMs (also the AOSP-based ones) have added it by now. (It is funny to see the growing collection of references from that commit’s description to our Marshmallow tracking bug.)

Library issues fixed

After these permission problems were fixed we still had to solve the duplicate library issue (extracted+inside the APK). Our x86 maintainer Sean Hoyt did find this commit that gave us the info how libs are now read from the APK. Based on that information we could see that for reading the libs from the APK they have to be uncompressed and aligned within the APK’s ZIP-structure. We were also able to confirm that the APKs were structured in the same manner on Google’s Nexus images.

Re-packaging the APKs

One problem is though, that all APKs distributed through the Play Store are by default compressing their libs. So the build scripts had to be adjusted to repackage the latest APKs when building for Marshmallow. This is done in the scripts during the build process by first temporarily extracting the libs from the APK. Then the (compressed) libs are removed from the APK. The last step is by adding the earlier extracted libs again back to the APK, but with a specific ZIP command to refrain from using any compression when adding.

APK certification

You might wonder: doesn’t that break the validation and certificate of the APK? Which is a valid question, to which luckily the answer is: No. Because even though the outside (deflated state) of the APK does change (e.g. the APK will be larger and will have a different md5 hash) all APK validation and certification mechanisms describe the contents of the ZIP in its inflated form. Because all (content of) the files within the ZIP itself stay the same, all the validation and certification are still valid :-)

GApps devs

Of course all this knowledge was shared with the other GApps devs, so that everybody can benefit from Marshmallow packages from their favourite provider ASAP.


So we have now working MarshMallow builds. Since today’s build also GoogleTTS has been added for all 6.0 packages, since on some ROMs (but not all, apparently) the Setup Wizard demands it. We don’t know why and what triggers it. But we don’t want anybody to experience FCs during their setup.

Marshmallow development


Finally a post about Marshmallow!

No builds yet

Most important information first: we don’t provide any stable builds for Marshmallow yet, in the cause of that is because Marshmallow poses a couple of challenges that have not been properly resolved yet. We only want to offer pre-built Open GApps packages when these issues are resolved.


So, what are these challenges you ask? We will highlight a few:

  • Marshmallow comes with several new core applications. Our scripts had to be updated to package and install these applications too. This has been done and implemented already in our code.
  • Google’s Marshmallow images have for some applications no pre-extracted libraries. They read the libraries from within the APK. For some reason AOSP ROMs do not always accept libraries this way, and still demand pre-extracted libraries. We still don’t know why they differ in behaviour, but this is something that should be resolved by the ROMs and consistent behaviour should be defined.
  • Google finally fixed the security mechanisms for apps on /system/ partition and demands all APKs to be properly signed and verified. This does mean that we cannot remove the libraries from the APK, or it would not pass this verification anymore. In combination with the previous point, this would mean that all libraries would take twice the system space, if they would have to be both within the APK next to being pre-extracted.
  • Only applications that are signed with the platform key are automatically granted all permissions on /system/. All other applications have to request these permissions. On Google’s stock images this is no problem, because both the platform as the Google Apps are signed by Google. But on AOSP ROMs, the ROM is not signed by Google, resulting in that the Google Apps don’t get their permissions automatically assigned.
  • Which leads to our following point, that the core Google Apps assume that they have already been granted all permissions, and if they don’t have the permissions, they just crash and give constant FCs on the screen while the device is booting.
  • The only ‘solution’ to work around this problem for now, is not bundling some core applications like SetupWizard. Which is something that in our opinion is not good enough of a workaround for pre-built packages to the public.
  • The real solution would be to patches in the AOSP ROMs to also automatically grant permissions to Google Apps too, or e.g. use an init script that can interact with the Android Shell to perform the granting of the permissions automatically.
  • But we reasoned that we are not alone in this problem. So we are waiting for some of the ‘good’ vendors that play nice with open source, like Sony, to release the source code of their changes to AOSP and to check how they chose to work around this problem. Because also they don’t have access to Google’s platform key, but they do have to bundle these Google Apps with their ROM.

GApps devs

The very nice thing about these problems though, is that it gives more unity to the GApps devs community and most of us are working together and sharing our knowledge and progress. So kudos to dankoman, benzo and others for their efforts.

Track progress

For further information, please keep an eye on our XDA thread and our GitHub. If you are Android ROM dev and want to help on the ROM-related tweaks, please drop a note on the GitHub issue. HomeSupport by DonationSource on GitHubCommunity on XDA ForumManual on WikiFollow via PushbulletDownload Open GApps AppOpen GApps BlogOpen GApps on FacebookOpen GApps on Google+Open GApps on TwitterOpen GApps on YouTubeOpen GApps AppAds help us to keep packages freeAds help us to keep packages freeAds help us to keep packages free