Mfw when the paper on dishonesty in fact dishonest
19 karma · joined April 23, 2022
Mfw when the paper on dishonesty in fact dishonest
Looks really impressive, although one issue I do see is consistency. For example, it seems to highlight `null` as a value, unless it is on the left side of an `===` expression, since it probably hasn't seen that in training.
Was it ever actually tested on languages not in the training set? Would be interesting to see if a model this small can actually generalize...
Uh... how about..., no...? What?
> Controllable from any device, anywhere
...
> There is a kill switch
Oh great, sounds like you're confident this is safe then!
Someone wake me up, please
> or cannot control any of the inputs to the llm
Seeing as LLMs are non-deterministic, I think even this is not enough of a restriction.
Tangentially, I'd be far from the first to point out that these LLMs are now polluting their own training data, which makes filtering simulatenously all the more important and impossible.
If I find myself needing to ask a clarifying question, I always edit the previous message to ask the next question because the models seem to always force what they said in their clarification into further responses.
It's... odd... to find myself conditioned, by the LLM, to the proper manners of conditioning the LLM.
I've since had to issue 17 patches (it's been 3 weeks) even though the package has 100% test coverage.
Out of curiosity: How exactly did it work (for users)? I assume you'd write a grammar on the one pane, but how did you teach the tool to convert it into the output language? Was that embedded into the grammar?