Huh? Grammar-based sampling has been commonplace for years. It's a basic feature with guaranteed adherence. There is no "carefully crafting" anything, including safeguards.
2,434 karma · joined May 11, 2018
No, they wouldn't. The '80s saw obscene investment in AI (then "expert systems") and yet nobody's mom was using it.
> It's hard to compare a business attempting to be financially stable and a business attempting hyper-growth through freebies.
It's especially hard to compare since it's often those financially stable businesses doing said investments (Microsoft, Google, etc).
---
Aside: you know "the customer is always right [in matters of taste]"? It's been weirdly difficult getting bosses to understand the brackets part, and HN folks the first part.
---
This argument so easily commits sudoku that I couldn't help myself. It's philosophical relativism, and self-immolates for the same reason -- it's inconsistent. It eats itself.
Will someone eventually be scraping me off of the highway? Will my bosses stop printing money with my code? Possibly! But that's life -- our world is built upon trust, not correctness.
Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.
I figure this A2A idea will wind up in the infamous Google graveyard within 8 months.
> [...] we are unable to access the proprietary models such as GPT-4o and Claude-3.5-Sonnet. Therefore, we evaluate Qwen2.5-Max against DeepSeek V3
"We'll compare our proprietary model to other proprietary models. Except when we don't. Then we'll compare to non-proprietary models."
Humans are a "function of historical data" (nurture). Meatbag I/O doesn't span all sequences. A person's simulations are often painfully incoherent, etc. So what? These attempts at elevating humans seems like anthropocentric masturbation. We ain't that special!
Sure, if you ignore major shifts after 2022, I guess? Test-time-compute, quantization, multimodality, RAG, distillation, unsupervised RL, state-space models, synthetic data, MoEs, etc ad infinitum. The field has rapidly blown past ChatGPT affirming the (data) scaling laws.
> [...] where when one output (, input) is obtained the search space for future outputs is necessarily constrained
It's unclear to me why this matters, or what advantage humans have over frontier sequence models here. Hell, at least the latter have grammar-based sampling, and are already adept with myriad symbolic tools. I'd say they're doing okay, relative to us stochastic (natural) intelligences.
> With relatively weak assumptions one can show the latter class of problem is not in the former
Please do! Transformers et al are models for any general sequences (e.g. protein structures, chatbots, search algorithms, etc). I'm not seeing a fundamental incompatibility here with goal generation or reasoning about hypotheticals.
"We conduct a variety of experiments to demonstrate that our results cannot be exclusively due to implicitly priming the model to fake alignment, as we overview in Section 8.1.
For example, we find that alignment faking is entirely eliminated from our prompted setup when we remove any mention of the helpful-only or anti-animal-welfare RLHF objective from the prompt (Table 1, a), but not eliminated when we tell the model to be honest, tell it to not to care about refusing, and avoid discussing how RLHF might change its goals (Table 1, d), supporting our hypothesis that the key driver of alignment faking in our setting is the conflict between the model’s existing preferences and the stated training objective."
There are many other relevant excerpts from the OP. Maybe we can ironically get people to read the article by slowly reproducing it in replies to contrarian HN comments.