"We will continue to offer a suite of safety filters that developers may apply to Google’s models. For the models released today, the filters will not be applied by default so that developers can determine the configuration best suited for their use case."
Pricing and speed doesn't matter when your call fails because of "safety".
This is a query I did recently that got rejected for "safety" reasons:
Who are the current NFL starting QBs?
Controversial I know, I'm surprised I'd be willing to take the risk with submitting such a dangerous query to the model.
I don't recall exact prompt but it should be something close to that. I really wonder what filters they had about kink tracks and why? Do they have a problem with Beyond standard model searches /s.
I think I bumped into Anthropics once? And I know I hit ChatGPTs a few months back but I don't even remember what the issue was.
I hit Google's safety blocks at least a few times a week during the course of my regular work. It's actually crazy to me that they allowed someone to ship these restrictions.
They must either think they will win the market no matter the product quality or just not care about winning it.
Using their models in a medical setting is impossible. It refuses to describe scientific photos and will not summarize HCP discussions (texts) that it misinterprets.
NFL Starting Quarterbacks for the 2024 Season Note: Quarterback situations can change throughout the season due to injuries, trades, or poor performance. AFC • Baltimore Ravens: Lamar Jackson Buffalo Bills: Josh Allen
But then it gets wiped and you cannot see it even in the drafts. The text above is from the screenshot I managed to make before the response vanished.
This non-deterministic unpredictable behavior blended with poor “safety” policies is one of those major “dealbreakers” that pushes me back from trusting any existing LLMs.
For example, this prompt was apparently unsafe: "Summarize the conclusions of reputable econometric models that estimate the portion of import tariffs that are absorbed by the exporting nation or company, and what portion of import tariffs are passed through to the importing company or consumers in the importing nation. Distinguish between industrial commodities like steel and concrete from consumer products like apparel and electronics. Based on the evidence, estimate the portion of tariffs passed through to the importing company or nation for each type of product."
I can confirm that this prompt is no longer being filtered which is a huge win given these new lower token prices!