Briefcase: Convert a Python project into a standalone native application
briefcase.readthedocs.io
briefcase.readthedocs.io
Has anyone solved the compiled libraries / dynamic libraries / linking problems for multiplatform apps? Everything works just fine with these python>native app tools until you hit some esoteric library that's not pure python.
I have deep wounds from pre-chatGPT times of trying to manually figure out libraries. As in otooling each library, error, etc then recompiling into various places until things 'worked.' I'm appreciative of the experience, though now I think with LLMs the problem could be solved with some simple queries.
So I guess you're right, it requires effort, but I was a 10x engineer in the wrong direction and never want to experience that again ;)
A lot of places do this nonsense "spend hours configuring your machine" method for projects. The only reason I can think of doing that is "job security" and who wants to keep a job with horrible tooling?
But yeah, it takes effort to do.
In a way, that time was a chapter in 'job security.' Ran out of money, had to let everyone go, and picked up the pieces to fight another day (self-taught under threat of survival). By the grace of God I'm a reformed 'idea-guy' and am now a builder.
It makes it really hard to distribute certain python libraries.
Is there ever going to be standardization, the way a WebGL shader just works everywhere?
It has a very specific filesystem layout that it calls a package, but this is not a first class citizen; it has a packaging format called "wheel" that is now used for distribution, and that can contain native libraries. That currently mostly handles the distribution of compiled libraries for multiple platforms, but only within the traditional venv-style setup. Being able to ship interpreter+venv+launcher would go a long way towards providing a native-ish experience.
A step beyond that would be being able to distribute a "virtual filesystem" containing a whole tree of python compiled as .pyc, appended to a native executable containing the interpreter and linked against the required native libraries already.
Since venvs on Linux symlink to the system’s Python executable, they should also work after moving.
So…they aren’t then.
So, if activate.ps1 (and the other scripts for other shells) were updated to dynamically identify their own location rather than hardcoding the path, they’d be fully portable? Seems to be a fairly simple update.
> Since venvs on Linux symlink to the system’s Python executable
Hopefully, the executable with which they were created, not the system executable except where they happen to be the same.
I don't think the maintainers have gotten a request to do that. Personally I use a more complicated launcher because I need to set things like environment variables in my deployments, but a venv is almost an out-of-the-box relocatable Python environment. It's when you need to protect/compile your code that things get messy.
>> Since venvs on Linux symlink to the system’s Python executable
>Hopefully, the executable with which they were created, not the system executable except where they happen to be the same.
When you call /usr/bin/pythonx.y -m venv ./myenv, ./myenv/bin/python will link to /usr/bin/pythonx.y. Since there aren't many reasons to use a Python version not installed by the package manager, I call these the "system's Python executables", but sure, it can also point to /opt if you installed Python there.
I used that for my own Python+Electron app, which I wrote about here: https://til.simonwillison.net/electron/python-inside-electro...
As much as I hate how containers are often misused, this is one of the reasons people obsess over them from my experience. The ability to bundle all your crap regardless of how many layers of package management idiosyncrasies your application ultimately woul have to deal with pretty much transparently.
https://cibuildwheel.readthedocs.io/ is a great tool for assisting with that - many projects run that in GitHub Actions as part of their release process.
There's just one catch, in my experience: since GitHub don't offer Apple Silicon machines as part of GitHub Actions, it can be hard to build wheels there that work on those platforms. Maybe you can cross-compile them? I've not figured that out yet.
https://beeware.org/project/projects/libraries/toga/
which uses native widgets and manages basic functionality such as where the "Quit" menu entry is automatically.
Looking into this now --- it looks perfect for my needs, just hoping that there's a nice graphical tool for laying out a GUI (if it has a graphical programming mode like to Blockly or Ryven, then it is perfect for me).
One example of this, as parent mentioned, is where the "quit" button is located. Different platforms put this in different places. Toga will automatically put it in the right one, depending on which platform you're on. Buttons will look like normal buttons on a Mac, but also like normal buttons on Windows -- depending on which platform you're on.
0. https://developer.android.com/training/monitoring-device-sta...
Why not?
It's a half solution at best, but if you control the infrastructure it does dramatically simplify things like Dockerfiles and other deployment stuff. Not super useful for general distribution IMO.
Basically, u zip the package, then add the shebang on the zip binary. Quite cool and simple! No need for external tools :p
https://kivy.org/doc/stable/gettingstarted/installation.html
And after all that, the app doesn't run. So yeah.
I put off learning python for years because of this ridiculousness. Finally I decided to just bite the bullet, because it just isn't getting any better.
It took me a long time to learn that no matter how awesome the concept seems, if it's not extremely popular (aside from very simple things) there's probably an issue somewhere.
At this point I would rather just make web apps for most everything, and I wish they'd just merge android and ChromeOS so we had an easy way to make simple cross platform stuff, while keeping the Android APIs for more powerful stuff.
There are many of these open source packagers and they all share very similar problems:
1. They don't let you do software updates, even though software updates are practically mandatory for any real project. Electron is a stand-out here because it does address this, but their update engines are unmaintained for years and have some major unfixed problems (causes a lot of issues with Windows networks, for example).
2. Even in the very rare cases that they do, they don't let you force updates on launch even though many apps need something like this to keep up with protocol changes. It's one of the reasons people like web apps.
3. They don't help you with signing. For example they don't simplify key management, they don't support cloud signing (essential since May because Microsoft now insist on HSMs for all keys, not just EV keys), or they don't do notarization, or they don't generate CSRs for you.
4. They require the use of CI to cross-build even when apps are written in portable frameworks that don't require compilation. This is because they are just thin wrappers around the native tooling.
5. They're invariably language specific even though there's no good reason to be because 80% of the work is the same regardless of what language or framework you use.
6. They make MSI files for Windows even though MSI is deprecated.
It's possible to bite the bullet, chew glass for a while and solve all these problems, which is what we did for Electron/JVM/Flutter/native apps. You can reimplement all the native tooling so users can cross-build (i.e. make Mac packages from Linux/Windows, Windows packages from Mac/Linux etc), which enables releasing from developer laptops or cheap Linux CI workers. You can support software update by integrating Sparkle on macOS, apt on Debian/Ubuntu and by using MSIX on Windows (and by then working around all the bugs in Windows to make it work well). You can generate download pages that work out the user's OS and gives them the right download, and instructions for how to install self-signed apps if the developer isn't code signing with a recognized certificate. You can abstract platform neutral things and expose platform specific things. Then you can write a parallel incremental build system so doing all the work is as fast as possible, and write lots of code to detect all the myriad mistakes people make and give good error messages or auto-fix them. Then you can make it support GitHub Releases. Then you can document it all.
But that big pile of glass isn't particularly tasty, which is why open source projects don't do it and we ask commercial users to pay for it.
Briefcase looks nice but it also seems to have all the problems listed above. I think once we add Python support Conveyor will be quite useful for the Python community, especially if we can find a workaround for pip not support cross-building of venvs. It would be great if you could just whip up a quick Python script, run one command and your installed clients start automatically updating, your download page updates, and the whole thing is no harder than releasing a static markdown-rendered website.
[2] https://github.com/hydraulic-software/conveyor/issues/73
(Except their own authentication keys, of course)
On that topic, one feature Conveyor has that open source packaging tools usually don't is that it can protect signing keys in your macOS keychain, in some cases. To utilize that the packaging tool itself must be signed, which of course it is here, and this is useful because the OS will protect both the private key and the process that's working with it from malware even if it's not explicitly sandboxed. The keys themselves are protected using the security chip. So it's a very high level of security that's sort of equivalent to having a dedicated user account for signing on Linux, but without needing to set anything up.
Unfortunately not every credential can be stored there yet, only the "root key" from which other keys are derived. Login credentials for Apple's notarization servers and cloud signing services like eSigner/KeyLocker don't get protected in the same way yet. But that'll come.
One of the few things that seem unambigiously worded in Microsoft's disclosure on this topic is that the hackers did actually have a copy of the key itself:
> was forging Azure AD tokens using an acquired Microsoft account (MSA) consumer signing key
> Storm-0558 acquired an inactive MSA consumer signing key and used it to forge authentication tokens for Azure AD enterprise and MSA consumer to access OWA and Outlook.com. All MSA keys active prior to the incident – including the actor-acquired MSA signing key – have been invalidated.
> The method by which the actor acquired the key is a matter of ongoing investigation.
https://www.microsoft.com/en-us/security/blog/2023/07/14/ana...
Which sure sounds like something that wouldn't just happen if these lived in a HSM. (Presumably Microsoft data centers have access controls...?)
My guess is that because these keys see a high volume of operations and rolling out new keys is probably annoying because tons of different pieces validate against them that Microsoft chose to not use HSMs for them for cost reasons, because you can just use all compute power in a server to perform operations with the key (much lower $/sign/s) and if you need to scale up you can simply copy the private key to more instances.
Is it? The WiX Toolset still seems to be actively developed and MSIX doesn't seem to support all use cases. Has MSI been officially deprecated despite that?
Also, MSIX's mandatory signing makes it too expensive and cumbersome for a hobby project, so it would be unfortunate if that became the only option.
MSIX files can be self-signed if you install the cert first. Conveyor has an EXE wrapper that can do that for you. So you can distribute apps with it without buying signing certificates.
I only want permission to modify my own app, not every app of the system. If users have to trust me (and all other developers of self-signed applications) with permission to modify all apps, that makes the whole signing rigamarole kind of pointless.
But if I do want to try this approach, do I have to install the certificate in Trusted Root Certification Authorities, or should it be in Trusted Publishers, or Trusted People? The first one would grant me even more privileges that I don't need, but what about the other two?
If you don't have some kind of signed identity then the concept of "your own" app doesn't exist, so indeed in that model you get permission to do what you want more or less. That's why it requires administrator elevation, whereas real signed packages don't. Of course this model is identical to the normal pre-MSIX model where any app installer can fiddle with any other app's files.
You can put it in Trusted People, I think. I'd have to check the code to see where we put it.