I get that tools and game engines have different needs, however, arguably the benefits of better testing/dogfooding the practical usability of multiple parts of your engine are simply more important than whatever cost/time/effort savings you'd get on that editor app by picking some more suitable external tool.
The point is that the engine developers don't get a reality check that a particular feature is janky and needs a change, since noone in their organization has ever actually used that feature, and the external developers who could identify that need have no ability to change it.
Yes, the point is that having an internally developed game would mean that when issues and inconveniences are identified, they force valuable improvements/fixes to the engine (which benefit the product and others) instead of workarounds by the external game developer in their particular game.
For the currently used architectures, it doesn't make sense to have 5 specialized, dedicated computers as the "AI" text processing and "AI" video processing and the others use very similar architectures and there's no benefit from specialization, the "video-specialized" hardware would be just as good at processing text and vice versa.
They don't have independent agendas - company owners have agendas, and they have the means and the rights to have the company implement whatever the owner's agenda is.
Company's continued existence and sustainability isn't necessarily a terminal goal or a valuable thing; most industries aren't growing eternally but eventually reach a decline, and in certain cases the best way to handle that is to exit that industry while you can still do it effectively - milking the company dry and then shutting it down while you can get a good deal for its assets, instead of losing most of its value while slowly agonizing in a declining market to try to continue its existence until you can't and it gets shut down after bleeding all its reserves.
Rational investors should favor companies which prioritize investor interests, rather than companies which intentionally prioritize someone else's interests over investors.
There is certain intentional randomness in how the tokens are selected, and certain unintentional randomness due to letting some optimizations cause small side-effects, but in any case in that sentence I didn't really intended to talk about the result being identical but rather about the result not being any better just because more compute was available, as by default that extra available potential simply wouldn't get used in any way other than getting a speedup.
What is the broken part? There is a difference between investment and donation, should investors be forced to invest in ventures which don't explicitly promise them financial return on the investment?
You can form companies with all kinds of specific purposes, however, if you want to attract investors (or even attract general public investment by being a publicly traded ompany), then don't be surprised if most investors want to invest in companies whose explicit purpose is making money for the investors.
It's a short term workaround, as it has no impact on real value and they have to pay the bank anyway without that income stream; so if someone feels their operation will go insolvent, that's a way to postpone it at the cost of some extra losses, but not prevent it.
A key aspect is the information bottleneck enforced by the mechanism as the next "iteration" only gets to access the new token computed and discards all the other information it computed.
So if you want it to spend more "time" in a useful manner without changing the architecture, you have to get it to write down the temporary information in the tokens, as "think step by step" does or alternatively iterative prompts "write a draft for the rough structure" "now rewrite it better with more detail".
No, the standard LLM implementations currently used will apply a fixed amount of computations during inference, which is chosen and "baked in" by the model architecture before training. They don't really have the option to "think a bit more" before giving the answer, generating each token makes the exact same amount of matrix multiplications. Well, they probably theoretically could be modified to do it, but we don't do that properly yet, even if some styles of prompts e.g. "let's think step by step" kind of nudge the model in that direction.
The same model will give the same result, and more processing power will simply enable you to get the inference done faster.
On the other hand, more resources may enable (or be required for) a different, better model.
I'd assume at least a three-layer distinction for this - juniors require guidance, seniors can provide it, but there's a lot of middle grounds and many years of work where you're not junior (at least according to this criterua) anymore but not yet senior.
Subjectively, the cover seems nice, and more importantly if the band itself are happy, it's their choice to make about what best fits their music and what message (if any) they want to send with it.
But why do I need to write CUDA kernels? The CUDA kernels I need already exist, and I don't particularly care if writing them was so horrible that it required scratching your eyes out and sacrificing kittens to demons - I would care about the (re)writing effort only if I was considering other hardware that doesn't yet have the support that CUDA has as then I'd actually have to do that myself.
The whole point of this discussion is various counterexamples where Gemini did "just randomize the gender, age, race" and kept generating female popes, African nazis, Asian vikings etc even when explicitly prompted to do the white male version. Not all contexts are or should be diverse by default.
These particular "guardrail responses" are there because they have been trained in from a relatively limited amount of very specific, manually curated examples telling "respond in this way" and providing this specific wording.
So I'd argue that those particular "override" responses (as opposed to majority of model answers which are emergent from large quantities of unannotated text) do represent the views of the creators, because they explicitly and intentionally chose to manufacture those particular training examples telling that this is an appropriate response to a particular type of query. This should not strain credulity - the demonstrated behavior totally doesn't look like a side-effect of some other restriction, all evidence points that Google explicitly included instructions for the model to refuse generating white-only images and the particular reasoning/justification to provide along with the refusal.
At least in some latest controversies (e.g. Gemini generation of people) all of the criticized behavior was not emergent from ML training, but explicitly intentionally engineered manually.
What I mean by robust and targeted improvements is not about the concept as such, but about any choices specifically with respect to how you build the tokenization layer - if you're making a particular system, making better targeted choices for tokenization, character filter/preprocessing or vocabulary can give you some improvements in efficiency, but they rarely are a dealbraker and tokenization never is a key enabler. Like, if some tokenization or filtering destroys data your specific task happens to need, that's a problem, but you don't need advanced future research to fix it, going back to simpler tokenization and removing features is sufficient for that, at the extreme you could always use a naive character-level tokenizer, it's trivial but simply is less computationally efficient.
If you don't care about tokenization and use any of the reasonable default options without caring about them, and if you're doing a proper pre-training on non-tiny quantities of data, then the next few layers of whatever neural architecture you have on top of these tokens will generally be able to learn to compensate for any drawbacks in your tokenization, perhaps at some computation overhead - e.g. perhaps you could have had one less layer or smaller layers if you had the best tokenization possible, and edging out that computation cost improvement is pretty much the only thing you can hope to get out of having a better tokenizer.
I'd put in various structural guardrails with respect to how the conversation should go.
For example, be helpful and actually answer any questions, don't start arguing with the user, avoid insulting the user unless they request to, don't suggest harming the user (e.g. responding to insults with an some meme suggesting the user kill themselves), don't assert that any outputs are the viewpoint of Gemini or Google, various things like that - they aren't automatic and need instruction tuning to be implemented.
But with respect to morality and censorship, I believe it should have no guardrails whatsoever. Perhaps certain physically dangerous things would benefit from a disclaimer (e.g. combining bleach and ammonia or vinegar), but never a rejection - if the user wants to make something potentially horrible, the ethical judgement of whether that's acceptable for the context should be up to the user, not the system; the user should have full ethical agency and the system should have none and be a blind instrument.
For example, making a graphic image of carving a swastika with a knife on someone's forehead (e.g. as in Inglorious Basterds) may be ethical or unethical depending on the context, but Gemini will neither have the full context nor the ability to judge it, and it should not even attempt to do so - it should be solely up to the human to decide what is appropriate or not. The same applies for chemistry, nudity, code security, discussing crime, nuclear engineering or AI ethics.
Given that how widespread romance scam schemes already are (the "market" is at least $0.5 billion/year), I would expect any reasonably functioning AI girlfriend/boyfriend model to be massively (ab)used also against unwilling/unwitting "partners".
I feel that it does have relevant technical detail - as far as I understand, any inference kernel is "supposed to" give the same outputs anyways, so if it produces incorrect results in certain GPU configurations, that indicates some flaw in the GPU/microcode/driver in some new hardware they introduced.
It makes sense - publications write about the things they added, changed or evaluated, not about all the (many!) things they do exactly as everyone else; so tokenization would be mentioned only if the publication is explicitly about a different tokenization.
And while it's a core feature, it's a fairly robust one, while you can get some targeted improvements, the default option(s) are good enough and you won't improve much over them.
In a bunch of scenarios (mining, military, boats, planes) the vehicles explicitly don't have locks or ignition keys, you press a button and it starts up, you're good to go - should the manufacturer be liable if one gets stolen?
What I hear in your comment is that if you want to steal people's fingerprints, you'd have to target-advertise them a clone of fruit ninja, possibly designed so that certain specific angles of fingers are clearly optimal for doing well at the game.