omneity

Supabase, while a great product, does not offer serverless Postgress.

Despite the tone this is excellent advice! I had similar impressions reading the article and was wondering if I missed something.

Great job, congrats for the release! A little feedback, it could be nice to auto-generate a unique url on page load (the one you generate after clicking on "share"). Bonus is that it gives the user a certain confidence to reload the page for example.

Side note, I am wondering if this could be used as a better way to collaborate with coding agents. "Pair with me" instead of the typical "Code for me".

True in principle. Looking at it from Apple’s side, they are selling an ecosystem of clients to digital (or physical) service providers with certain guarantees on how that ecosystem operates.

Opening up automation is a double whammy for Apple:

- More savvy users are able to solve more of their problems themselves, reducing the LTV of their potentially most engaged clients who might buy less on the App Store as a result

- Some guarantees of the ecosystem crumble (banking apps don’t know anymore if it is really you who initiated that transaction, ID verification apps don’t know if that camera stream is really from your device, and plenty other things devs suddenly have to worry about)

As a final nail in the coffin, it also means that the networks that Apple sells to its _users_ are less reliable or have less guarantees as a result - you don’t know if the text you received was really sent now or scheduled in advance etc.

Of course a lot of these rules are not justified, user hostile or plain non-sensical, but what I’m trying to say is that from Apple’s perspective the consideration is not just “user owns device” but a lot of interplaying dynamics that do not seem to be in favor of empowering users.

Thank you! Adding support for `additionalProperties`[0] (and perhaps `patternProperties` too) would be particularly great!

Happy to provide test cases as well if helpful.

0: https://datatracker.ietf.org/doc/html/draft-fge-json-schema-...

It does! I do use CT2!

On a decent CPU I found the translation to take anywhere between 15-30 seconds depending on the sentence’s length, very unnerving to me as a user.

But it’s definitely worth revisiting that. Thanks!

Many languages (with a sizable speaker population) do not have machine translation to or from any other language.

The technique makes sense though, but in the training data stage mostly. BART-style translation models already represent concepts in latent space regardless of the input-output language sidestepping English entirely so you have something like:

`source lang —encoded into-> latent space —decoded into—> target lang`

Works great to get translation support for arbitrary language combinations.

JSONSchema support on Google's OpenAI-compatible API is very lackluster and limiting. My biggest gripe really.

Yes. There's a scoring and filtering pipeline first, whereby I try to automatically check for the quality of the correction using a custom multilingual embedding model, madmon[0] and language identification model, gherbal[1]. Above a certain similarity threshold it goes into the training dataset, below it it's flagged for human review. This is mostly to stave off the trolls or blatant mistakes.

For the continuous training itself, yes I simply continue training the model from the last checkpoint (cosine lr scheduler). I am considering doing a full retraining at some point when I collect enough data to compare with this progressive training.

Apologies for the poor links, it takes a lot of time to work on this let alone fully document everything.

0: https://api.sawalni.com/docs#tag/Embeddings

1: https://api.sawalni.com/docs#tag/Language-Identification

Being alone in this vast city gave me a certain feeling of unease. Reminds me of Stargate episodes where they would find an advanced civilization devoid of life, only to discover they got wiped out in a mysteriously terrible fashion.

Thanks for the tip and cool project! The model I trained is relatively large, as it's a single model that supports all language pairs (to leverage transfer learning).

With that said while running it client-side is indeed an option, openly distributing the model is not something I would like to do, at least at this stage. Unlike the bigger projects in the NMT space, including Marian and Bergamot, I don't have any funding, and my monetization plan is to offer inference via API[0].

0: https://api.sawalni.com/docs

Pandoc can be painful (mostly due to the complexity of all the tools it orchestrates in my opinion), but I find it very hard to beat once you get in non-trivial publishing territory and you refuse to use TeX/LaTeX-style processors.

The learning curve is worth it imo (if you often work on documents and publications).

Marian was a good starting point and allowed me to iterate faster when I first started, but I quickly found it a bit limiting as it performs better for single pairs.

My goal was a google translate style multilingual translation model, and for that the BART architecture proved ultimately to be better because you benefit from cross-language transfer learning. If your model learns the meaning of "car" in language pair (A, B), and it knows it in language (B, C), then it will perform decently when you ask it to translate between A and C. It compounds very quickly the more you add language pairs.

One big limitation of BART (where LLMs become more attractive) is that it becomes extremely slow for longer sentences, and is less good at understanding and translating complex sentences.

> Any special processing needed for Arabic compared to Latin-script-based

Yes indeed, quite a lot. Especially for Moroccan Arabic which is written in both Arabic and Latin scripts (I made sure to support both and they're aligned in the model's latent space). For this I developed semantic and phonetic embedding models along the way that helped a lot. I am in the process of publishing a paper on the phonetic processing aspect, if you're interested let's stay in touch and I'll let you know when it's out.

But beyond the pre-processing and data pipeline, the model itself didn't need any special treatment besides the tokenizer.

I started seeing results as early as 5-10k pairs, but you want something closer to 100k, especially if the language has a lot of variations (aka morphologically rich, agglutinative, or written in a non-standardized way).

Manual proof-reading (and data generation) was a big part of it, it's definitely not a glamorous magic process. But as I went through it I could notice patterns and wrote some tools to help.

There's a way to leverage LLMs to help with this if your language is supported (my target wasn't at the time), but I still strongly recommend a manual review part. That's really the secret sauce and no way around it if you're serious about the translation quality of your model.

Do you have a recommendation for a better incarnation of the principle?

Tangential but I noticed a certain common conflation between pre-rendering and server-side rendering. Very often plain SSR is all it takes for good SEO performance, SSR in this case simply being rendering the page on-demand before serving it to the user.

Pre-rendering (as popularized by static site generators) is the additional step that increases complexity significantly, sometimes security issues too when session-protected cached pages are mistakenly served to the wrong users.

Related: I built a translation app[0]* for language pairs that are not traditionally supported by Google Translate or DeepL (Moroccan Arabic with a dozen of other major languages), and also trained a custom translation model for it - a BART encoder/decoder derivative, using data I collected, curated and corrected from scratch, and then I built a continuous training pipeline for it taking people's corrections into account.

Happy to answer questions if anyone is interested in building translation models for low-resource languages, without being a GPT wrapper. A great resource for this is Marian-NMT[1] and the Opus & Tatoeba projects (beware of data quality).

0: https://tarjamli.ma

* Unfortunately not functioning right now due to inference costs for the model, but I plan to launch it sometime soon.

1: https://marian-nmt.github.io

I used the seamless attribute extensively in the past, it still doesn't work the way GP intended, which is to fit in the layout flow, for example to take the full width provided by the parent, or automatically resize the height (the pain of years of my career)

It worked rather like a reverse shadow DOM, allowing CSS from the parent document to leak into the child, removing borders and other visual chrome that would make it distinguishable from the host, except you still had to use fixed CSS layouts and resize it with JS.

I think that's very smart, thanks for sharing! With the prevalence of coding agents currently the cost of context/language switching is much lower and these best-of-breed multilang setups are likely to become more prevalent in the future.

Seconding this use case! I see pipask helping greatly as another safety net when using coding agents.

Thank you for the answers. Sounds reasonable in my book (what's one more internet stranger to trust, hah).

I'll give it a try!

What makes Rails stand out is the focus on convention-over-configuration as a guiding principle in the ecosystem which results in a lot less code (have you seen these relatively thin models and controllers?), as well as established dependencies and the lack of tendency to bikeshed in libraries (geocoder or devise for example have been mostly stable over close to a decade, with few popping up to replace it)

I switched from Rails to the node.js ecosystem back in the 3.2 to 4 transition, however looking back I share a similar sentiment as the OP.

I recently initiated the backmigration and my approach thus far however has been to take out the "administrative" part out into Rails to benefit from all the useful conventions there, but keep the "business services" in JS or Python and have the two communicate. Best of both worlds, and the potential of all of rubygems, npm and pypi combined.

This looks great, congrats on the release! So what are the potential risks/downsides?

As in what’s the tradeoff that is being made when relying on pipask?

Congrats for shipping!

Have you seen LiteLLM Gateway before? As a bonus it supports a lot of different providers.

https://docs.litellm.ai/docs/simple_proxy

This rings such a vague and distant bell...

I'm several videos in and totally hooked, thank you for sharing. This would be an amazing interactive music app in VR, both to perform and to record trippy music videos.

You are certainly right, managed platforms make finetuning much easier. But managed/closed model finetuning is pretty limited and in fact should be named “distribution modeling” or something.

Results with this method are significantly more limited compared to all the power open-weight finetuning gives you (and the skillset needed in return).

And in either case don’t forget alignment and evals.

I read your comment first and was like it can’t be that bad. I then opened the page, scrolled frantically down and boy oh boy were you right.

This assumes the team deploying the RAG-based solution has equal ability to either engineer a RAG-based system or to finetune an LLM. Those are different skillsets and even selecting which LLM should be finetuned is a complex question, let alone aligning it, deploying it, optimizing inference etc.

The budget question comes into play as well. Even if text is repetitively fed to the LLM, that might happen over a long enough time compared to finetuning which is a sort of capex that it is financially more accessible.

Now bear in mind, I'm a big proponent of finetuning where applicable and I try to raise awareness to the possibilities it opens. But one cannot deny RAG is a lot more accessible to teams which are likely developers / AI engineers compared to ML engineers/researchers.

It's okay for extracting simple things like addresses, or for formatting text with some input data, like a more advanced form of mail merge.

I haven't evaled these tasks so YMMV. I'm exploring other possibilities as well. I suspect it might be decent at autocomplete, and it's small enough one could consider finetuning it on a codebase.