SQLite and Android N (2016)
ericsink.com
ericsink.com
sqlite.so isn't part of public Android API surface, which means the OEMs will regularly replace it, break it, compile it with strange options and deal a lot of pain to you and your users.
So if you weren't shipping your own native SQLite in NDK/Xamarin you already had problems, you just maybe didn't notice them.
Android N is now actually actively preventing linking against system libraries which aren't part of public API surface, which is good - it'll result in less developers doing dumb things which break on minor updates.
You can never stop the Facebook developers doing dumb things on Android.
Google have tried for years, but Facebook always manages to find new ways to fuck up.
https://www.facebook.com/notes/facebook-engineering/under-th...
I've never understood why the Facebook client is so heavy. That isn't a long list of features! It seems like the requirements could be written on the back of an envelope.
Obviously part of the reason is that they move fast and have a lot of developers, so there's much more pressure to add code than to keep it trimmed down. But the client still doesn't do all that much. Surely, surely most of the logic is handled on the server.
Instead of using the platform-provided, battery efficient GCM (now FCM), they insist on using their own MQTT push solution.
Not only does this require a constantly running background service and a TCP socket which has to be reopened on every connectivity change (e.g. a switch from wi-fi to mobile data): They do it for every single one of their apps.
My phone has at least three of them (Instagram, Facebook, Messsenger), and I'm pretty sure that this is a major reason for battery life problems on Android.
Xamarin, Cordova and others are pretty much a cesspool of catastrophically poor libraries so I'm not sure why Google has to take the blame for that.
Why is the NDK so underpowered? Why are they so reluctant (or unable) to publish some useful, stable native APIs? It's missing things like curl, sqlite, icu, OpenSSL. Most of these are built into the OS but not accessible via the NDK. (OpenSSL isn't a great library, I know, but it's patchable. And they could start now with something better.)
The only useful native third-party libraries you get are: zlib, OpenGL ES, OpenSL ES.
Apart from that, there are a bunch of wrappers for a small subset of Java APIs (AssetManager, Bitmap and so on). These are tedious to use, and seem to have been written by hand so in some cases they're buggy -- for example, incorrect string handling in AStorageManager: https://code.google.com/p/android/issues/detail?id=41983
I bet many Android devices already have at least 9 copies of sqlite, since so many apps ship their own copy.
NDK team doesn't look like it's expansive or prioritized at all, so I doubt they have resources or will to do that. TBH I'd rather they focus at improving the tooling and debugging, we can ship our own libraries with our own options compiled in. Heck even on iOS we have to ship a custom SQLite to get all the needed capability.
Google has one of the largest software engineering populations of any company in the world. It supports one of the largest developer ecosystems in the world. Games are the largest catagory in their ecosystem. 2 engineers supporting the NDK...
It wasn't there until 2.2, only got added due to community pressure and they always did the minimum amount of work.
When the migration to Android Studio was announced, we got left with no migration plan.
Only after JET Brains announced CLion, almost 2 years later, did they announce the migration to Studio.
Likewise they moved into Gradle without plan for C NDK support and it took three generations of build tools until they decided to settle on CMake.
As an earlier commenter mentioned, games are one of the biggest app categories and many games rely on the NDK. Pro audio could be another big category, but I think Apple has the market sewn up there because Android is still way behind.
(I'm not saying that there's anything inherently unreasonable about that, although I happen to disagree with the decision. But these things cost money and they decided they didn't want to spend the money.)
Most likely due to game development community.
Specially when compared with the tooling support C and C++ enjoy on iDevices and UWP.
So anyone that wants 2D graphics from NDK side either needs to call into Java, or bring their own library on top of OpenGL.
And the whole mismanaged way how the C++ IDE and build system migration was handled.
What you want is stable APIs for things that interact in fiddly ways with the rest of the system, so they're more easily managed at the OS level. Networking, shared databases. (Edit to add: localization)
zlib is nice to have, but it's also really portable and lightweight. It would be easy for apps to bundle their own copies. Something like libcurl is a lot more awkward (what do you use for https? bundle your own SSL too!)
In any case that was just an example.
Android has nothing like Objective-C++ or C++/CX for easy interoperability with the OS APIs.
Even if you don't need a whole 2D library, it would be very useful to have a stable API for reading the system fonts, so apps don't have to ship their own.
Also, given that Google forked Java, they could have added something like CNI insteand of forcing everyone the JNI pain.
iOS also is not easy to use with any language, it assumes ability to link with a module written in ObjC. The pain of using UIKit with Python is comparable to using JNI.
It seems you never did UWP development.
UWP was designed to be architecture independent since Windows Phone 8.
Application's bytecode is uploaded to the store and gets compiled to each supported architecture using the so called "cloud compiler".
iOS development has moved into that model with bitcode on iOS 9.
Also there have been several C and C++ compilers across the years that could generate single binary on any CPU architecture.
For example compilers for OS/400, TenDRA, PocketPC.
> iOS also is not easy to use with any language, it assumes ability to link with a module written in ObjC. The pain of using UIKit with Python is comparable to using JNI.
Python is not a platform language on iOS SDK, C and C++ are platform languages on iOS, Android and UWP SDKs.
A good lesson in terms of productivity, is to only use SDK supported languages when writing production code any OS.
That's why I wrote: independently from any appstore that could recompile for you.
Google was and is under different constrain, that you refuse to acknowledge: everything, that puts an application onto the device, must be able to run locally at the development system. Amazon store, F-Droid, or whatever the Chinese or Russians are using cannot depend on Google Play Store to recompile for them. The result has to run not only on any combination of ARM/Neon/VFP3, but also on Intels, MIPSes or some future architecture, that someone somewhere used or will use in the Android device.
Yes, they could use pNACL or LLVM bytecode, or whatever. They used a higher level one, dex. Their target was Java class hierarchy, they didn't want their developers to reinvent strings and collections all time.
> Python is not a platform language on iOS SDK, C and C++ are platform languages on iOS, Android and UWP SDKs.
C and C++ are not platform languages for Android. Literally, just read the first two paragraphs of Android NDK's "Getting Started" document. Using C/C++ is only for certain scenarios, that don't require the rich platform APIs. These are written in Java and if you are going to use non-JVM language, you are going to go through the same pain as if you were using Python with ObjC frameworks.
There are already implementations [1][2] of this, but because it is not provided by the Android team it is hardly supported by ORMs, libraries, etc.
[0] https://code.google.com/p/android/issues/detail?id=202658
The problem - from what I understand - is that you used to be able to use the systems sqlite.so to access databases owned by your app from the native environment. With Android N you are no longer able to do this, you need to ship your own sqlite binary.
This can lead to problems when you want to access the same sqlite files from both the Android platform APIs and native (e.g. use it from native for business logic, debug it with a tool like Stetho [0] that uses android.database.sqlite) because of a version mismatch.
Would Google offer (a copy of) the Android API with the ability to plug in a different sqlite-compatible binaries, it would very likely find broad adoption. Then the problem stated by the article could be solved by shipping the app with a sqlite build that is used by both the native code and plugged into the compat library.
SQLite was also shaking out a (very) few bugs out during this time period; pretty sure this auto-delete implementation wiped any chance of semi-easily recovering my first phone's SMS db.
https://stackoverflow.com/questions/7764943/what-can-be-done...
"8. Bugs in SQLite" (2013)
Does SQLite accept patches?
Though, my guess shouldn't stop you looking for previous suggestions to implement this, and if there are none, submitting it yourself :)
1. https://refspecs.linuxfoundation.org/LSB_4.0.0/LSB-Core-gene...
https://code.google.com/p/android/issues/detail?id=213433
Although I mentioned using dlopen and dlsym to find the SQLite functions in the current process, our actual production solution was to use JNI to call the android.database.sqlite classes.
You have no idea what version of sqlite the OS might be using, so you can't rely on consistent behavior. In many cases the vendor will have tinkered with it and messed it up. Comprehensively testing on all Android devices is basically impossible because there are so many models out there.
The reason they can tinker with it is that it's not public; if it were public it would (hopefully) be better-defined, locked down and stable, part of the Android compatibility test suite.
That feature request would let people supply their own sqlite.so, and also redirect the Java API to use it. Seems like a very promising approach.