813 karma · joined October 11, 2023
Fable and Opus 5, I suspect, will become textbook examples of RL collapse.
Right now, it’s a little hard to follow what the actual findings are, why they are what they are, and why that’s important.
Off-topic but this is the first time I’m learning we (Australia) have a space agency :O
We've tried to take a first-principles approach to our end goal of 'legal superintelligence' that has involved identifying the areas of our domain most in need of improvement and releasing models that raise the bar on quality in those areas.
We've been around for a couple months now and ended up starting with retrieval and enrichment. The models we've released to tackle those problems have indeed been smaller in size than their competitors, yet they still rank ahead on open-source benchmarks.
Them being so small also helps with their accessibility — as I mention in our post, our models can be deployed on ordinary hardware, not a supercomputer.
Next on our roadmap is reasoning and research, which will require more infrastructure to support, but again, we aim to be judged by performance at the time of release.
> As much as I agree with the message, this reads like marketing copy trying to make a big deal out of a tiny model being hosted privately.
The point of this point is really just to reaffirm our commitment to sovereignty and accessibility and contrast our approach with that of major AI labs. It _is_ possible to commercialize LLMs while still keeping them accessible. A customer using a self-hosted deployment today does not need to worry about our models no longer being available tomorrow. We think that's a good thing. And moving forward, we want to keep that option available for anything we do, instead of trying to pull up the ladder while we're ahead.
There’s a massive issue with extrapolating to more complex tasks, however, where either you run the risk of prompt injection via granting your agent access to the internet or, more commonly, an exponential degradation in coherence over long contexts.
We'd argue it makes a lot of sense to appropriate 'graphitization' as a term for a model designed to transform data into knowledge graphs.
Our pricing page can be found in our documentation here: https://docs.isaacus.com/pricing/prices. We're planning on making it more visible on our website; thanks for the feedback!
We do already have a government-facing client using one of our self-hosted deployments given the privacy and security concerns the legal industry tends to have (rightfully in our view) around AI.
As @lopuhin points out, they already claimed that context window for previous iterations of GPT-5.
The funny thing is though, I'm on the business plan, and none of their models, not GPT-5, GPT-5.1, GPT-5.2, GPT-5.2 Extended Thinking, GPT-5.2 Pro, etc., can really handle inputs beyond ~50k tokens.
I know because, when working with a really long Python file (>5k LoCs), it often claims there is a bug because, somewhere close to the end of the file, it cuts off and reads as '...'.
Gemini 3 Pro, by contrast, can genuinely handle long contexts.
Voyage's terms say:
> you grant Voyage AI (and its successors and assigns) a worldwide, irrevocable, perpetual, royalty-free, fully paid-up, right and license to use, copy, reproduce, distribute, prepare derivative works of, display and perform the Customer Content: ... (iii) to train, improve, and otherwise further develop the Service (such as by training the artificial intelligence models we use).
Cohere's terms say:
> YOU GRANT US A ... RIGHT TO ... USE ... ANY DATA ... TO ... IMPROVE AND ENHANCE THE COHERE SOLUTION AND OUR OTHER OFFERINGS AND BENCHMARK THE FOREGOING, INCLUDING BY SHARING API DATA AND FINETUNING DATA WITH THIRD PARTIES ...
Jina's terms say:
> Jina AI shall, subject to applicable mandatory data protection requirements, be entitled to retain data uploaded to the Jina AI Systems or otherwise provided by the Customer or collected by Jina AI in the course of providing the Services and to use such data in anonymized/pseudonymized format for its business purposes including to improve its artificial intelligence applications.
I can understand though, perhaps in a work environment where management is unlikely to be able to retain high skilled talent, you may want 'low-profile' workers that aren't going to have as many competitors chasing after them...
Encoder-decoders are not in vogue.
Encoders are favored for classification, extraction (eg, NER and extractive QA) and information retrieval.
Decoders are favored for text generation, summarization and translation.
Recent research (see, eg, the Ettin paper: https://arxiv.org/html/2507.11412v1 ) seems to confirm the previous understanding that encoders are indeed better for “encoder task” and vice-versa.
Fundamentally, both are transformers and so an encoder could be turned into a decoder or a decoder could be turned into an encoder.
The design difference comes down to bidirectional (ie, all tokens can attend to all other tokens) versus autoregressive attention (ie, the current token can only attend to the previous tokens).
Why do we then have lots of invisible characters that are intended essentially as semantic markers (eg, zero-width space)?
Over the past couple months, we, a team of Aussie legal and AI experts, have been working on building a new type of legal AI company — a company that, instead of trying to automate legal jobs, is trying to automate legal tasks.
We want to make lawyers’ lives easier, not replace them.
We’ve been laser-focused on building small and efficient yet still highly accurate, specialized models for some of the most time-consuming and mundane legal tasks lawyers have to perform. Stuff like running through a thousand contracts just to locate any clauses that would allow you to get out early.
We just finished training our first set of models, focused on document and clause classification, probably the most common problem we see come up. Our benchmarks show our models to be far more accurate and almost more efficient than their closest general-purpose competitors.
Today, we’re making those models publicly available via the Isaacus API, the world’s first legal AI API.
Our models don’t require any finetuning because they’re zero-shot classifiers — you give them a description of what you’re looking for (for example, “This is a confidentiality clause.”) and they pop out a classification score.
Because our models are so small, which they have to be to be able to process reams of legal data at scale, they can sometimes be a bit sensitive to prompts. To help with that, however, we’ve preoptimized an entire library of prompts, including what we call, universal templates, which let you plug in your own arbitrary descriptions of what you’ve looking for.
We’ve made our prompt library available via the Isaacus Query Language or IQL. Another world first — it’s a brand-new AI query language designed specifically for using AI models to analyze documents.
You can invoke query templates using the format “{IS <query_template_name>}”. You can also chain queries together using Boolean and mathematical operators, like so: “{This is a confidentiality clause.} AND {IS unilateral clause}”.
We think our API is pretty neat and we hope you will too.
This is just the beginning for us — over the course of this year, we’re planning on releasing text extraction and embedding models as well as a second generation of our Kanon legal foundational model.
Below are some links for your convenience.
- Quickstart guide: https://docs.isaacus.com/quickstart
- Announcement: https://isaacus.com/blog/the-worlds-first-legal-ai-api
- Docs: https://docs.isaacus.com/home
- Sign up: https://platform.isaacus.com/accounts/signup/
- LinkedIn: https://www.linkedin.com/company/isaacus/
- Email: hello@isaacus.com
Unlike JSON, you can’t just open a MessagePack file in Notepad or vim and have it make sense. It’s often not human readable. So using MessagePack to store config files probably isn’t a good idea if you or your users will ever need to read them for debugging purposes.
But as a format for something like IPC or high-performance, low-latency communication in general, MessagePack brings serious improvements over JSON.
I recently had to build an inference server that needed to be able to communicate with an API server with minimal latency.
I started with gRPC and protobuf since it’s what everyone recommends, yet after a lot of benchmarking, I found a way faster method to be serving MessagePack over HTTP with a Litestar Python server (it’s much faster than FastAPI), using msgspec for super fast MessagePack encoding and ormsgpack for super fast decoding.
Not sure how this beat protobuf and gRPC but it did. Perhaps the Python implementation is just slow. It was still faster than JSON over HTTP, however.
```python
import sys sys.set_int_max_str_digits(0) # Allows for printing very large numbers.
x = (2 * 136_279_841) - 1
print(x)
```
If you have labelled training data (or semi-labelled), BERT takes the cake, both in terms of accuracy and efficiency. In fact, you can have luck with getting a CLM to generate noisy labels and then training BERT/RoBERTa on that to get a robust strong classifier.
Do you have a source for this?