newhouseb

I built a similar thing to Grant's work a couple months ago and prototyped what this would look like against OpenAI's APIs [1]. TL;DR is that depending on how confusing your schema is, you might expect up to 5-10x the token usage for a particular prompt but better prompting can definitely reduce this significantly.

[1] https://github.com/newhouseb/clownfish#so-how-do-i-use-this-...

Ah, this makes total sense. I was thinking about FLOPs in the abstract and not about the wall-clock time. Thanks for the explanation.

Nope, a decoder only transformer is a variant of the original architecture proposed by Google [1]. All variants of GPT that we know about (1 through 3) all roughly use this same architecture which takes only the decoder stack from the original Google paper and drops the encoder [2]

[1] Original Google Paper - https://arxiv.org/abs/1706.03762

[2] Original GPT Paper - https://s3-us-west-2.amazonaws.com/openai-assets/research-co...

Ah, you're thinking about embeddings which are basically the encoder stack on a traditional transformer architecture. Modern GPT-like models (including Claude), however, drop the encoder and use decoder-only architectures.

I could imagine something where encoders pad up to the context length because causal masking doesn't apply and the self attention has learned to look across the whole context-window.

This doesn't match my mental model (or implemented model in the case of GPT2) of how self-attention works (you need to calculate the residual stream for each individual token, attending to all prior tokens before it). Have a link?

Naively, yes, but you can cache the bulk of that "rerunning" [1]. That said the (non-flash) attention costs go up with the length of the sequence so perhaps this is just a simpler way to approximate these costs.

[1] https://kipp.ly/blog/transformer-inference-arithmetic/

I wonder why this is? Naively there's no difference between the two from a transformer standpoint.

Perhaps it's because under the hood there's additional safety analysis/candidate generate that is resource intensive?

Ah yeah, I had the outdated 30M in my head.

Oh absolutely, I'm just imagining what I might think if I was a super conservative director at Google who is accountable for the balance sheet of a large org.

But Google hasn't disclosed which version of Bard, right?

I pop into Bard every once in a while to test its performance, but I never know if I'm getting the best Google has or just what Google can tolerate running cost-wise publicly given they potentially have at least an order of magnitude (if not two, edit: 1.5) more users than OpenAI.

This tool is really lovely, great work!

I'd be curious to see Softmax Linear Units [1] integrated into the possible activation functions since they seem to improve interpretability.

PS: I share your curiosity with respect to things like deep dream. My brief summary of this paper is that you can use GPT4 to summarize what's similar about a set of highlighted words in context which is clever but doesn't fundamentally inform much that we didn't already know about how these models work. I wonder if there's some diffusion based approach that could be used to diffuse from noise in the residual stream towards a maximized activation at a particular point.

[1] https://transformer-circuits.pub/2022/solu/index.html

My emerging conception of this is to split this into two separate questions:

1. Is the architecture _capable_, i.e. is it possible for a model with a given shape possible to perform some "reasoning"

2. Is the architecture _trainable_, i.e. do we have the means to learn a configuration of the parameters that achieves what we know they are capable of.

Recent interpretability work like that around Induction Heads [1] or the conclusion that transformers are Turing complete [2] combined with my own work to hand-specify transformer weights to do symbolic multi-digit addition (read: the same way we do it in grade school) has convinced me that reasoning over a finite domain is a capability of the even tiniest models.

The emergent properties we see in models like GPT4 are more a consequence of the fact that we've found a way to train a fairly efficient representation of a significant fraction of "world rules" into a large number of parameters in a finite amount of time.

[1] https://transformer-circuits.pub/2021/framework/index.html

[2] https://jmlr.org/papers/volume22/20-302/20-302.pdf

Small world!

Thanks for the link (which I've now skimmed beyond the abstract). What wasn't obvious to me from the abstract is that different attention heads have different penalty strengths, so if some prediction task requires long range dependencies you might expect one of the less-penalized heads to end up specializing. I wonder what would happen if the penalty for one head is zero? (The paper suggests this might've been tried and just made things worse, but unclear)

I must admit that this is a wonderfully elegant (and interpretable) way to do this... much more intuitive (to me at least, a wannabe practitioner) than all of the trig-based embeddings.

First - thank you for open sourcing this! It's a real gift to the community to have a model intended for "commercial use" that's actually licensed as such.

I'd be very interested to hear about the choice/evaluation of the ALiBi approach for positional embedding (perhaps in the technical report).

My intuition suggests that while this allows for better generalizability for longer sequence lengths, it penalizes scenarios where an LLM might need to check for things like a function signature far away from where the next token is generated. My initial testing of this model tracks with this intuition but that's by no means a rigorous evaluation.

Thank you! ReLM is a great find! I like that it drives the generation itself so that it can explore different beams more intentionally. And to do the JSON Parsing well against enums/unions/oneOf, you really have to support backtracking in a way that works basically the same as it does for regex so I'm looking forward to digging into their implementation.

I haven't spent time going deep here but my current hypothesis is that interoperability will more or less end up looking like toolformer where the "tools" are just separate LLM runs with task-specific context. So for example:

> Instruction: Write a poem and them emit a structure that follows a schema named X.

> Completion: [map-schema X "roses are red, violets are blue"]

Conceptually this is basically just a function call where context is local to the function.

Sometimes, but it very much depends on the context (no pun intended). If it's a pure syntax issue, OpenAI models will almost certainly make the right correction. If it's more abstract, like the LLM has hallucinated a property that is invalid as part of some larger schema you can quickly descend into the LLM gaslighting you into saying that it has fixed things when it hasn't.

Oh nice! I built a similar system a few weeks ago: https://github.com/newhouseb/clownfish

I think the main differentiating factor here is that this is better if you have a simpler JSON schema without enums or oneOf constraints. If you do have these constraints, i.e. let's say you wanted an array of different types that represented a items on a menu { kind: pizza, toppings: [pepperoni] } or { kind: ice_cream, flavor: vanilla | strawberry } then you would need something more sophisticated like clownfish that can ask the LLM to pick specific properties (and an ability to do some backtracking so you can do proper beam search).

For completeness, another common approach can be found here: https://github.com/ShreyaR/guardrails which essentially boils down to "provide the schema in the prompt and ask the LLM to correct things if it fails to get the schema right the first time."

And I adapted Jay's work to Typescript (without the numpy obviously, just raw typescript/javascript): https://github.com/newhouseb/potatogpt

The code supports both local and OpenAI as a backend, I added OpenAI as a backend because their models are still miles better than anything I can run locally (even 65B LLaMA)

Nice! I've been wondering similar things about whether you could use this to eek out more intelligence through methods like these, to quote the end of my write up:

> Does structured decoding increase the observability of emergent world models in these models? To make an analogy: I may not represent an opinion of how something works if I am not confident in it, but if I am forced to present an opinion we might find out that I in fact have (or have not) grasped something.

In practice, however, without tight integration with beam search, the autoregressive nature of these models means that the syntactic steering may result in the models rabbit-holing themselves without forward looking visibility that's obvious from the defined grammar. I.e. if it was forced to choose between "Don't Jump" and "Do run" in some hypothetical example, the set of tokens that it would likely be deciding between is "Don't" and "Do" with no idea what is going to end up syntactically required after those tokens.

There are a couple different approaches:

- Use multi-shot prompting with something like guardrails to try prompting a commercial model until it works. [1]

- Use a local model with a final layer that steers token selection towards syntactically valid tokens [2]

[1] https://github.com/ShreyaR/guardrails

[2] "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish (full disclosure: this is my work)

I was having a conversation with some friends the other day about what schemes might mitigate some of these risks, something like an anti-safe word, i.e. a "danger word" that someone can be used to remotely validate that a loved one is authentically in danger.

This is fairly low-tech and likely susceptible to various kinds of social engineering, but I'm curious what a more robust approach might look like that doesn't involve us all regurgitating 6-digit codes like robots all the time.

There are a couple different approaches:

- Rerun the prompt until you get a format that is consistent

- Steer the output token selection towards a predefined prompt

For the latter, I've built a proof of concept that takes in a JSON schema with a huggingface transformer and constrains token selection by modifying the output probabilities such that only schematically valid tokens can be emitted, see "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish. Unfortunately, given OpenAI's current API this is only possible on locally run models. That is... at any level of cost effectiveness. It's technically possible but (worst case) quadratically expensive against their current APIs.

Last weekend I built some tooling that you can integrate with huggingface transformers to force a given model to _only_ output content that validates against a JSON schema [1].

The challenge is that for it to work cost effectively you need to be able to append what is basically a final network layer to the model that is algorithmically designed and until OpenAI exposes the full logits and/or some way to modify them on the fly you're going to be stuck with open source models. I've run things against GPT-2 mostly but it's only list to try LLaMA.

[1] "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish

We also use GPT to perform actions in the software I build at work and we hit the same issue of inconsistency which lead me down a long rabbit hole to see if I could force an LLM to only emit grammatically correct output that follows a bespoke DSL (a DSL ideally safer and more precise than just eval'ing random AI-produced Python).

I just finished writing up a long post [1] that describes how this can work on local models. It's a bit tricky to do via API efficiently, but hopefully OpenAI will give us the primitives one day to appropriately steer these models to do the right things (tm).

[1] https://github.com/newhouseb/clownfish

Yes, sorry for not making that more obvious. Since we work with US-based governments at various levels at the moment it's simplest to stick with folks authorized to work in the US. This may change in the future as we look at doing work internationally, fingers crossed!

AidKit | Remote | Full-time | https://aidkit.org | $130k + equity | Social Impact / GovTech | TypeScript

AidKit runs the largest guaranteed income programs in the country. We replace convoluted workflows of glued together spreadsheets, bank portals and human tedium with one unified platform to allow organizations that do good, to do so faster and more effectively. AidKit is in many ways a technological petri dish to explore how we can bring the leverage of a bespoke software team to those without a software background through domain modeling and low-code tooling.

We're looking for an experienced all-around engineer (frontend & backend) who cares about our mission to bring good software to doing good. You'll work (with me!) as part of our small tech team. Our stack is mostly Typescript and Postgres (with some Python that we're deprecating).

We're bootstrapped and profitable.

If you're interested, please reach out to me at ben@aidkit.org!

You could configure a second Cybiko, hooked up to your computer, to act as a gateway between an untethered Cybiko and the internet. They teased a standalone device to do this (CyWIG) but I don't think that ever actually shipped.

At one point I did set this up and managed to browse some (terrible) WAP websites and I think I managed to get AIM working somehow too. No memory as to if you could tunnel through to do other wireless things with other bridged Cybikos though as the setup was so clowny I had no other friends to try it with.

You might be thinking of this [1] paper? The core idea being if your problem space lends itself to monotonicity (see the paper for a more precise definition than I can give), then you can build a globally consistent database (around a CRDT) where the end-user doesn't need to concern themselves with inconsistent states while consistency is reached.

[1] https://arxiv.org/pdf/1901.01930.pdf