MicroG Project: A re-implementation of Google's Android apps and libraries
microg.org
microg.org
The underlying theme connecting these libraries is that they're clients for Google's proprietary protocols. Sometimes there are good reasons for these protocols to be proprietary. In the case of Maps (which I worked on some years ago) it's because Google's licenses for some data sets are specific to Google, they aren't allowed to just throw it all out there via an open protocol, and they are expected to discover and block non-Google client apps. In the case of the Market, they want to defend against things like abusive install count inflation. They also like to redesign products and change features whenever they want. All these things are easier when you can change the protocol at will because there's only one client to support, and the client team sit next to the server team.
Note for example that the microG folks have had to implement "DroidGuard", a system that tries to spot scripting of Google's servers from non-approved clients. The microG implementation contains fake data that is sent back to the servers. This risks legitimate users being mis-identified as abusers and potentially having their accounts suspended. That risk must be understood by the microG authors but they don't inform you of it anywhere, which seems poor.
Given that both the protocols and client libraries can change at any time without announcement, I don't see how microG users will ever have stable devices. Nor do I see why they care. Even if they reimplement the client libraries Signal and other apps will still be dependent upon the proprietary servers.
If they want a version of Signal that's entirely Google independent the right thing to do is set up and run a competitor to the actual messaging service, not just reimplement a thin protocol wrapper and call it a day.
That doesn't sound open in the least.
"The underlying theme connecting these libraries is that they're clients for Google's proprietary protocols."
Protocols which are quite difficult to change out. If you're a person who doesn't want to use Google's maps, you're SOL for most applications that show maps within the app, as most of them are using Google, because it's easy. Whereas if the location services were just a system plugin, the user could choose the mapping data provider they wished, and the app would just receive map data and continue as normal.
If you're using an app created by another developer that uses Google Maps as part of its functionality - you have no right to change that anyway.
§69df UrhG
> zur Herstellung der Interoperabilität des unabhängig geschaffenen Programms
"For purposes of interoperation with an independently created software", decompiling and modifying software is legal.
So, yes, under German (and in general also EU law), you have the right to do exactly that.
That isn't entirely true: https://source.android.com/devices/tech/power/mgmt.html#doze
It's like saying Linux is more proprietary because Oracle is closed and runs on Linux.
This is more like if 9 of every 10 apps required a Red Hat subscription, and you said "but technically it's built on an open operating system".
Because I did all of those things, before I gave up and bought a Windows device. Which isn't open either, but at least is honest about it.
Android lacks apps so badly it's almost unusable. People should stop calling Android open when in fact they mean Google Play platform and do not realize what's really left there when you leave it pure Android.
[1] http://www.puredarwin.org/
[2] https://github.com/opensource-apple/xnu
[3] Almost, sort of.
https://twitter.com/CopperheadOS/status/772592323112869888
https://www.reddit.com/r/Android/comments/516z23/according_t...
Google apps/libs? That's a growing problem, as more and more apps are starting to rely on those, hence MicroG.
They can’t do that anymore.
What's obvious is that google isn't building AOSP themselves at all anymore and it's suffering from neglect.
If you've ever installed a 3rd party distribution of Android based on the open source project (AOSP), you know how hamstrung things are after a fresh install until you drop Google closed source apps in.
http://arstechnica.com/gadgets/2013/10/googles-iron-grip-on-...
Another perspective: Karim Yaghmour, author of Embedded Android, a book I highly recommend for anyone looking to learn about how the Android platform works, writes, "despite its awkward development model from an open source community perspective, it remains that Google's work on Android is a godsend for a large number of developers. Plus, it has accomplished one thing no other open source project was ever able to: created a massively successful Linux distribution. It would, therefore, be hard to fault Android's development team for its work."
Saying things like "the second you try to take Android and do something that Google doesn't approve of, it will bring the world crashing down upon you" diminishes how much the AOSP does. Because of AOSP, developers and tinkerers have dozens of popular choices for custom ROMs and custom kernels that can be flashed on a bootloader-unlocked device. Google itself tries to make getting bootloader-unlocked devices easy, though carriers and OEMs often have their own policies.
On the other hand, Skyhook v. Google (http://www.gpsbusinessnews.com/Google-Settles-Litigation-wit...) is compelling evidence that Android's value to the rest of Google is as a medium for presenting Google apps and services.
Taken altogether I think the most fair evaluation is that Android's ubiquity is valuable to Google, but that doesn't mean its open-source-ness is any less valuable to developers and users.
It also reminds me of when Cyanogen became a business. It was sorta interesting at first until I listened to the CEO talking about putting a bullet in Google's head. Now Cyanogen tries to sell app space on its users devices.
With the new deep Doze modes in Android, FCM is becoming integral to stuff as simple as push notifications (since FCM is supposed to be what wakes up the phone).
The APIs that let you do stuff like nearby communication are also getting locked down. From 6 and up you can't get a Bluetooth MAC address for Rfcomm without a system level permission, conveniently Googles Nearby API gladly gets around that.
I work on hardware without Play Services and I think what you're forgetting is stuff like FCM requires Play Services. FCM isn't an end all be all BaaS, but it's getting more and more ingrained in how Android works, and it's a little scary
There's F-Droid which is bringing some hope, but pure AOSP with F-Droid still cannot even compete with Nokia N900.
>mapping services arn't part of OS, thus don't have to be open
The mapping services in question was built by scraping the location and details of wifi ap's from publicly broadcast packets, received and logged by your and other consumer's hardware so you would think that this is public information and could be opened up, but it isn't - which seems poor.
>don't see why they care. Even if they reimplement the client libraries Signal and other apps will still be dependent upon the proprietary servers.
The goal is to prevent google from controlling the hardware you paid for and the software you're using. This, while not 100% stable, is better than having no options.
The better way to fix that isn't to reimplement the Google Maps and Google Play clients, it's improving OpenStreetMap and a GNU/Linux-style package manager for Android.
When you're trying to compete with a huge incumbent you always have the Wine vs. Qt/GTK question, where you have to choose between compatibility and design freedom.
But even when you choose compatibility and reimplement the incumbent's API, you don't want to do it using their servers. What happens when they suddenly change the protocol they never promised not to change?
Google has all the data that makes it work + you would need to analyze all that data to end up with server that can take radio levels and output location. It's not as easy saying lets use our own servers.
That would give microG zero time to prevent their software from working uninterrupted.
Also IIRC the Google services are updated automatically in the background anyway by the Play Store app. So I guess rollouts can happen quite quick regardless.
Sure, but in order to actually make apps work (without forking every map project out there) you'd need to fake the Google APIs anyway...
> GNU/Linux-style package manager for Android.
That's basically what f-droid is.
GNU/Linux-style package manager for Android is F-Droid, which bans apps that depend on Google's APIs. The effect of this policy can be clearly seen in the dearth of apps on F-Droid.
In your CS interpretation, OS X is also an open source OS since it's proprietary bits on top of the open source Darwin OS.
> Rather the author seems to be trying to redefine "operating system" to include things that have never been a part of such a term before, like online message routing and mapping services.
This doesn't really ring true in the face of what's happened; Google has kept a lot of things proprietary, sure, but they've also moved a lot of things proprietary that weren't before and abandoned the rest. They're making platform improvements in closed-source rather than open-source, making it difficult and impractical for a lot of developers to build apps that will work on Android, rather than 'Android plus Google Play services etc'.
This even includes the keyboard; Google has made improvements to the Android keyboard, but not in AOSP.
Fundamentally, this feels like an attempt to maintain both the user-visible branding (now everything is Google this and Google that, which reminds users that Google had its hand in things) but also to prevent carriers from shipping stock Android and cutting Google out of the loop.
[0] http://arstechnica.com/gadgets/2013/10/googles-iron-grip-on-...
[1] https://news.ycombinator.com/item?id=12578134
[2] https://developer.chrome.com/multidevice/webview/overview
I'd see the main general motivations for something like MicroG as being a sense of trust, privacy, control. As long as even 0.1% of any packaged end-product is closed source, this isn't possible.
From https://developer.chrome.com/multidevice/faq :
> Is Chrome for Android open source?
> Chrome for Android is derived from Chromium. Since the launch of the first version, we have steadily open sourced all the critical components. You can build various Chromium components for Android as used in Chrome for Android using the instructions here.
This is just a long way of saying "no"
So how do you go about to modify software on your phone? I love it all, but pretending as if Google is not skewing the ecosystem in a way to maintain control is a bit naive. There is economic reason to do so, but it clashes with the intrerests of the wider ecosystem.
> In the case of Maps (which I worked on some years ago) it's because Google's licenses for some data sets are specific to Google, they aren't allowed to just throw it all out there via an open protocol,
Hm, so HTTP is not an open protocol?
The "proprietary" protocols are unlikely to be replaced, since this would disable probably 30% of phones in the field. (not all update play services)
> [...] The microG implementation contains fake data that is sent back to the servers. This risks legitimate users being mis-identified as abusers [...]
The straight forward approach for any malware or threat this system is up against. - If this causes "good" users to be excluded, then google messed up big time. - I assume they anticipated this, making the point moot.
I see no malicious intent here and see no reason to be upset.
Mobile devices have an expectation of having more built-in features than other devices, so it makes sense that we would expand what we define as an OS.
Not to mention that the map services are implemented as core pieces of the OS within Android, so it's very much like having a proprietary libc with other proprietary libraries. You could create a program that doesn't use libc, but nobody does that because of duplicated effort (and the fact that Google won't hold your hand if you don't use their libc).
So I would very much argue that Android is on the path to becoming even more proprietary (it always was proprietary, just because of the fact that effectively all phones require blobs in the kernel as well as other firmware that is updateable but is proprietary).
That ship has sailed. All mainstream users expect those things to be standard in any mobile OS.
> it's as open as it's ever been
You're out of the loop. A lot of apps/APIs/frameworks which were maintained as part of AOSP have been abandoned to the point of being useless after Google introduced proprietary playstore versions.
>Given that both the protocols and client libraries can change at any time without announcement,
How do you then claim that android is "as open as it has ever been", when this was not the case before since android had fewer proprietary libs.
AOSP might be open-source, but running Android without these services makes it much less usable.
And some systems built into Google Play Services like SafetyNet have their purpose and their reason of being present (Android Pay and ensuring the stored data remain secure), but in some case they might stop an enthusiastic user of using a customized version of the OS because of missing features.
Since apps started relying on Google's proprietary Android components. This problem is apparent on ROMs which do not include GApps (like CyanogenMod by default and - in my case - CopperheadOS); there are plenty of applications which depend on things like Google Play Services (Slack being among them), for example. It's very similar (albeit not quite as severe) as the macOS ecosystem's dependence on a proprietary framework (Cocoa) despite otherwise running on a FOSS platform (Darwin).
I'm trying to run a totally open-source android installation, and have opted-out of Google entirely. It's worked out fairly well so far, but the app choices are fairly limited, of course.
Edit: The series is a little out of date, but should give you the general idea. ROMs that are probably new since that article was published that have no google dependencies are Replicant (entirely libre), OmniROM, Copperhead OS, and CyanagenMod w/ a no-gapps-script.
Isn't that a good way to get a modified / malware version of Signal? Anyone can upload to APKMirror, can't they?
This is also useful: http://www.onyxbits.de/raccoon It's a dektop play store client. I tried it out a while back and it worked, don't know about now.
The whole thread is a good read too haha
In other words, moxie is wrong. Again.
Edit: [0] https://f-droid.org/wiki/page/Setup_an_FDroid_App_Repo
The whole point of software stores is giving users the ability to trust the motives of the software they install, because the store and / or its users would never condone hostile software being hosted there.
Once you start having everyone run their own F-Droid repos, you are having independent developers give you their own trust keys, but you have no one else who needed to verify those developers were legitimate.
F-Droid itself is not particularly secure, given anyone can upload anything there, but in the same way the Archlinux AUR, OpenSUSE Build Service, Ubuntu Launchpad, etc work those third party software repositories are at least hosted by a trusted maintainer of the store / repo itself. If anyone ever uploaded malware there, once found out, it would be taken down and the responsible users banned.
With distributed app stores under F-Droid, or the equivalent third party repos for Arch / Suse / Ubuntu, the host has absolutely no control over the behavior of third parties, and thus anyone can host all the hostile malware they want, and if users add those repos they give them absolute trust in doing so.
That isn't a valid security model by any estimation.
The alternative is download static apks today and maintain updates yourself(bad) or remove the freedom to install what you want on your device.
But for, say, an app for a restaurant or a document reader, you would not know or have any reason to trust the vendor, so if they are self-hosting their own repos you are taking a tremendous risk trusting them.
The end result would probably remain the same - users might use third party repos for huge popular apps, but small apps would still need to stay centralized because there is no way to introduce a viable trust model against an organization you never interacted with before.
Most people don't change their default browser, adding third party repos would be similar. Removing the ability for the owner of a device to install software they want fixes one symptom, not the main issue of trust. Also it makes your device into a glorified feature phone. No thanks.
But it does mean you can install apps on the play store that have a dependency on gsf but are otherwise open-source (e.g. maps.me, signal, tomahawk).
I'd love open options instead of Google's services, but when 95% of your revenue is advertising, you have to own the platform, and Google does.
[0] http://www.theverge.com/2013/8/15/4625502/microsoft-responds...
Example: The Android Market API (yes android market, the name it had before play store) is still available and can be used, although it was replaced by a Play Store API years ago and there exist public client libraries for using it and the API is heavily used to grab free apps from the play store. However, as Google Play was never available for Android < 2.3 and there are still some users with this OS, the only way to disable this old API would be to remove Play Store access for a few hundred thousand users, which they refused to do until now.
However 3 years ago I was super excited and got the Alternative Google Store (I think it was BlankStore) installed. It crashed randomly and I was not able to download any App. However I thought that the author would work on it and in some time it would be useable.
Still waiting for that day.
That's why I'm a bit pessimistic about this project...
A better way would be to implement a true container/vm system so google only knows what is in this container, no contacts, no calendar, no mails. Heck, maybe it's even possible to link into that container for the host system apps.
Using the phone with Google Framework and modern OS made it so slow, MicroG (at least when I used it) didn't make it any slower.
I found the Galaxy Mini thrown by it's owner, picked it up and flash the ROM for shit and giggles...
[1] http://forum.xda-developers.com/showthread.php?t=1715375
Also this might be useful a desktop play store client: http://www.onyxbits.de/raccoon
I wonder if it's possible to dual boot without root and also keeping encryption on, that way I can have a safe option if anything breaks.
Damn, why is it so hard to have control over your things these days. It wears me down.
Does anyone know what main apps actually work for an average phone user, i.e. Maps, CityMapper, Dropbox, ...?
The only strange thing is, that for app downloads usually the APK is directly linked - while still using the Play Store logo. That is because there are many different app store providers. However, as soon as the app is installed, it gets updated by that respective app store.
Or random apps able to monitor the SMS, voice calls?
However, just having an API to follow does not mean, that the task of reimplementation is easy - following someone's else API is actually harder than designing your own. Just ask the WINE team.
Frankly, everything else would be a travesty.
My personal wet dream is that in response to this, Google will partner with Microsoft to get dotNET as a VM backend.
LOL Really? A likely win? The CAFC isn't going to overturn a jury verdict.
According to the CAFC's ruling overturning Judge Alsup's initial decision that APIs are not copyrightable, APIs have enough unique expressikn in them to indeed be covered by copyright. Hackers are fond of a loosey-goosey, asterisk-ridden interpretation of copyright law under which copying is OK provided the author doesn't lose money in its primary line of business. (For example, that pirating abandonware is OK, etc.) The CAFC is considerably more strict and applies the letter of the law to copyright cases. While fair use is a concept ill-defined by statute, it is generally acceptes to mean extremely limited copying for express purposes, such as: scholarship, commentary, or parody. Google's copying of Oracle's Java API IP was extensive and directly related to their Android line of business. Therefore, any sensible court would rule that fair use doesn't apply, else American IP law has no meaning.
Google won on fair use.
Jokes aside, implementing a program with the same API should be fine, right? Would love to hear from someone better versed in copyright law!
If this is all it comes down to, Google certainly should be fine with it being that they're in the middle of (end of?) a lawsuit with Oracle over their implementation of the Java API in Android.
[0] http://arstechnica.com/tech-policy/2016/10/its-official-orac...
"The Amazon Maps API offers interface parity with version 2 of the Google Maps API. Most classes and method calls in your Google Maps app work the same on Amazon devices. "
https://developer.amazon.com/public/apis/experience/maps/doc...
Edit: I see I misunderstood. This page talks about opting in to Google services: https://github.com/microg/android_packages_apps_GmsCore/wiki
So they are using them.
They'll sue you for using Java.
(Natively and not as slow emulator/VM).
Maybe that will unify the Linux Mobile, Desktop, Tablet better than the Gnome?
Gps is a basic selling feature, get over it.
This has nothing to do with entitlement or google. This is a way for people to run software on their phones without needlessly going through google.
Isn't this what MicroG does?
If you're referring to the system as a whole, the only real alternative is ios, which is far worse in terms of user control over what the system does.
In an ideal world, I would be running some linux distribution on my phone and be finished with it. We don't live in an ideal world.
GPS, maps and other such features are fundamental features of modern phones. They're no longer "additional addons" on a phone OS.
Also, many apps within FDroid still require some play services, and it's only because of microG that you can even use those free software apps. For example, Signal requires GCM and the project will not consider fixing this (and in fact demanded that FDroid stop distributing binaries). What is a user of that free software meant to do? Maintain a fork? People did that, but Signal won't federate. Move to another messaging service? Maybe, but there's nothing that matches Signal right now (I'm hoping that Matrix gets there soon so I can get the fuck away from Signal).
Personally I'm planning on rewriting some of the bus timetable apps for Sydney and using OpenStreetMaps instead of Google Maps. But I don't know if it's possible to have any form of meaningful GPS information without Google's proprietary services. I'm shit-out-of-luck as a developer, not an entitled brat.