Amazon Titan | Hacker News Reader

Amazon Titan

aws.amazon.com

109 pointsnycdatasci80 comments

With years of expertise reading AWS blogs and documentation, let me attempt to translate:

- Bedrock, an AWS service that makes base LLMs ("Foundational Models", FM) provided by both Amazon and third-parties accessible via API.

- Amazon Titan is one of these foundational models, provided by Amazon. Others include: ... things I've never heard of from companies I barely recognize.

Customer -> Bedrock -> Titan. Swap Titan for something better from a new AI start-up. It's another marketplace play.

where's the paper on Titan LOL ... can't be good if no paper or metrics saying it's better then GPT-4 or even GPT-3.

but it's amazing that Google has all the papers and models and the TPU and BERT is pervasive in their products and they can't even stake a better claim to leadership than AWS does here, Amazon stock probably goes up, Google announces anything their stock goes down.

I guess the reason of Amazon's stock going up while Google's stock going down might be that no one expected anything from Amazon/AWS in generative AI space yet, while everyone already expected Google to launch something like Bard for search engine and when launched, it was not as per market's expectation that had priced the value already. Also, Google has a potential of losing large search market to Bing and thus losing the revenue. While Amazon with this launch does not have the same story, it might (or might not) help them to gain more market in a new space but might not lose the existing ones, at-least not the major revenue source like search for Google.

> One of the most important capabilities of Bedrock is how easy it is to customize a model. Customers simply point Bedrock at a few labeled examples in Amazon S3, and the service can fine-tune the model for a particular task without having to annotate large volumes of data (as few as 20 examples is enough)

I can't even. Does anyone remember Amazon Kendra [1]? They promised the same there. "Here's an ML powered search engine! Point it at an S3 bucket with your own data and you're done". Amazon Kendra is one of the worst products I've ever used, and we ripped it out as quickly as possible.

AWS has a long history of over promising and under delivering when it comes to anything AI / ML.

1 - https://aws.amazon.com/kendra/

What was wrong with Kendra?

Result relevancy was worse than random full text search with your weapon of choice and it costs an order more to use.

Sitting there manually adding synonyms to a thesaurus to try and get something of value wasn't what we wanted to spend time doing.

Every relevancy-based search I've used pretty much sucks, so Kendra sounds about par for the course.

I will only be impressed when I see Amazon use it to correct all of their category and brand mappings for products. It has been a travesty since inception for a company purportedly data-driven.

There's also Alexa. I've seen no noticeable improvement in 5 years. It is difficult to believe that Amazon has any credible AI expertise to sell.

I'd like to see them use it to get rid of the plague of counterfeit products (I guess that's arguably a subset of "correcting").

But that would make the arrows go down so you know they'll never do that.

LLMs are Foundational Models, but not all Foundational Models are LLMs.

If you are interested in this space, it does probably make sense to look up what Stable Diffusion (StabilityAI) and Claude (Anthropic) are. Not recognizing them is fine, the pace is pretty incredible in this space and there's lots to learn, but it doesn't make the point you think you are making.

I wasn't attempting to make any point there except not listing out every company they included in their press release.

Not an expert in the space or LLMs, but I'm 100% their target audience: someone technical trying to decipher the release to see if it's of any use to their own company.

Thanks for providing your translation. It's sadly ironic I found TFA basically useless given it was likely written by someone with "communications" in their job title.

if you haven't heard of stability AI then it is just sad. I don't believe any AI credibility from MS either. OpenAI is not MS and integrating it with Azure is a pure marketing play as well.

This release note reminds me of my wife. She skips any context and starts a story in the middle, leaving me to do the detective work of figuring out what on earth we're even talking about.

I'm not sure what's preferable: your described mystery of a story, or the stereotypical "20 minutes of backstory and how we know their family" before getting to the meat of it.

Is your wife behind all these recipe blogs?

You, sir or modem as the case may be, just made my day.

Welcome. It's sir modem actually.

Yeah, that's the kind I get. A 30-second story stretched to several minutes. Every conversation related line-by-line even when most of it's not important and could have been summed up or omitted. It's... sure something.

She has a much better memory than me, though, and sometimes I wonder if it's because she recounts damn near every event in her life to someone else at least once (often three or four times) in great detail shortly after it happens, effectively giving it a thorough review while it's still fresh, while I pretty much never do that.

... but, if it's not a story, I do also get the starting-to-talk-with-no-context thing where IDK WTF she's talking about. "What do you think about red?" Red... what? To paint something? If so, what? Is that the title of something? For a shirt color? As a color, in general? The entry in the Bleu/Blanc/Rouge trilogy? The Target Red card? Flowers? We weren't even talking about anything! She says she kinda starts conversations in her head and it's not always the first part that comes out when she finally starts talking. Doesn't even realize she's done it until it's pointed out.

My wife is the same way, though I don't think it's anything specific to her or her sex - I suspect it's just inherent in the bond that people form when spending a great deal of time together.

Most of the time, when she starts conversations in the middle, I'm able to instantly intuit the topic and jump right in. I don't think about it and don't even notice it. When I do notice it (because I'm lost) I just tell her that I don't have any context. No problem.

I'll also say that I have no idea of if or how often I do the same thing.

Absolutely prefer more information. I can skim several hundred words per minute and scroll to the bottom if I already know the backstory, but if I don't, the punchline is completely useless and there's nowhere to go to find the context.

The author does not know what background the reader knows or does not know, and furthermore, if your story has multiple readers their level of background knowledge is not going to be equivalent.

Many of my previous and current coworkers do exactly the same thing.

Not only do they skip the context, but they also use vague terms like "it isn't working", "the server is stuck", and "the network died".

I'll often be told incorrect terms like Docker "container" instead of "image", "container" instead of "server" (or machine, node, host, EC2 instance, etc), "VPN connection" instead of "Remote Desktop connection", etc.

Lots of detective work and interrogation is necessary to come to some sort of understanding.

Speaking of detectives and half-stories, my bully of a stepdaughter indirectly taught me the corollary to this:

Only ever tell the end of any story, i.e. "I'm being sexually harassed by a lesbian girl at school!"

Throw yourself a pity party and guests will never question you. Nobody will discover the reason for the "harassment" was because she solicited the "offender."

Completely expected this from AWS: A marketplace (Bedrock) for foundation AI models, including Amazon's own (Titan).

Amazon is surely hoping this will generate enough revenue to make up for all the AWS customers who lately have been looking at their AWS bills with a fine-toothed comb, wondering how is it they've ended up spending so much on cloud services, which were supposed to be more efficient and flexible.

Amazon's hope is well-placed. This will likely generate meaningful revenue in the short run, given how much hype there is around AI right now, and how much pressure is being put by executives everywhere to show they have a "generative AI strategy." Now all those executives can say, "our developers are building new applications using generative AI from AWS," period. No one in enterprise land gets fired for using AWS.

Why would the executives use some unknown AI model from AWS which is not well tested or benchmarked against the plethora of OpenAI LLMs from Azure?

The LLM system of any company might be independent enough to run on a separate cloud. AWS so far has not even talked about any system capabilities they have. They might be hoping for some open source alternative to emerge which could shift the business to hosting in which they might excel.

Because you're already hosted on AWS and it's "good enough (tm)." That's basically the argument for why you use anything on AWS. They're not best-in-class in most of their service offerings.

For MANY companies, the AWS solutions are automatically approved and only require putting on the IT backlog. Any other vendor or cloud could easily take a year or more to bring online and would require an excruciating amount of scrutiny for every integration that you wanted to turn on. AWS often doesn't even require any budget review/approval.

AWS has done a great job selling itself as the trustable, secure and one-stop shop to IT.

Most of them must already be using something Microsoft (Office 365/Windows enterprise). So Microsoft will not be a new vendor. Something like Azure GPT4 will be just so powerful that I doubt most corporations can resist. And it doesn't have to be in the same cloud unlike a database.

In my experience, that has not been true. Setting up Azure is its own major undertaking and requires an entire separate set of security reviews and approvals.

AWS is trustable and secure though. That's why it's easier for enterprises to use, because they have all the compliance and SLAs enumerated and enforced.

Not only this but tons of places have "excess" to spend on AWS and so buy stuff through the AWS Marketplace. All kinds of SaaS sell things through the marketplace because its just easier at many companies to get purchases through them.

Yeah this seems like a big win for Azure overall, which is otherwise not compelling vs AWS unless you are all in on MS stack (Windows, SQL Server)

Wtf is a FM? What is Amazon bedrock and how does it relate to Titan? What a terribly written release note

FM = Foundational Model. Titan is the model, Bedrock is the service.

According to GPT-4, and I think this is a good summary: A "Foundational Model" in AI programming refers to large-scale machine learning models that serve as a basis for a wide range of applications and tasks. These models are pre-trained on massive amounts of data and can be fine-tuned or adapted for specific tasks, domains, or applications.

One of the most well-known examples of a foundational model is the GPT (Generative Pre-trained Transformer) series developed by OpenAI. GPT models, like GPT-3 or GPT-4, are trained on large datasets containing diverse text from the internet, which enables them to generate human-like text, answer questions, translate languages, and perform various other tasks.

Foundational models are significant in the AI field because they allow researchers and developers to create a wide range of applications and solutions without having to train a new model from scratch for each specific task. This approach saves time, resources, and computational power while still providing a high level of performance across different tasks.

From [1]: "What is a Foundation Model" In recent years, a new successful paradigm for building AI systems has emerged: Train one model on a huge amount of data and adapt it to many applications. We call such a model a foundation model.

From their big paper [2] "A foundation model is any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al.2019], GPT-3 [Brown et al. 2020], and CLIP [Radford et al. 2021]."

[1] https://crfm.stanford.edu/ [2] "On the Opportunities and Risks of Foundation Models", Bommasani/Hudson/et al

Who's in charge of naming at Amazon? The product line (Titan and Bedrock) tells me nothing about what it is suppose to do.

I mean, it doesn't have to be Amazon Provided Foundational LLM or what have you, but I just thought it was Amazon cosplaying Gundam Wing before I read the link

And also, wasn't Apple supposed to be working on a Project Titan?

And also, Titan is the well-known name of a line of powerful NVIDIA GPUs that are suitable for AI/machine learning work... So...

First reading the headline I actually assumed this product was some Amazon attempt at high end GPU/compute cards akin to Nvidia Titan models.

Nvidia retired that. I dont think the H100 even has the capability for display output in the silicon, and they just price the top end consumer card like its a Titan now.

Eh, these are short names that aren't taken by anything else in the field of generative AI. Bedrock is a clear nod to foundation models; I think it's a pretty great name.

These names are not especially bad compared to other large software companies and products. What on earth is a "chat gee pee tee" and what does it do? What does something named "Azure" do, let alone "Azure Synapse Analytics"? Don't even get me started on Microsoft's renaming of Office to 365. Yeah, I really love thinking and talking about writing stuff in "Microsoft 365." And what the hell does the company "Hugging Face" do? What industry could they possibly be in!?

Yet another half-baked and poorly supported AWS service. Now that they’re starting to see pressure on their margins I’d love to see AWS focus on their core products, but instead we get more spun off side projects.

> I’d love to see AWS focus on their core products

Agreed. I used to be all-in on the AWS experience, but it has really fallen behind in my opinion. Azure is quickly gobbling up their product scope per my preferences. The only things that are standing strong are Route53 (because Azure doesn't offer domain registration), and S3 (because I'm too lazy to learn something else).

For example, the current UX around S3 makes me really nervous. I felt like I was authorizing the release of radiological material when setting up a new public bucket.

> I’d love to see AWS focus on their core products,

Scope of coverage is an AWS selling point

100%.

My own personal infuriating example: Postgres RDS takes way too long to roll out new versions, major or minor, even those with important security fixes. Their upgrade processes fail when you use PostGIS, so every major version upgrade is a massive undertaking (clone the database locally, upgrade the database locally, drop all objects from the existing RDS, upgrade the RDS, then restore from local). And while their latest versions have relatively recent extensions, those extensions are upgraded carelessly (e.g. PostGIS is on 3.3.2, but the underlying geos library hasn’t been upgraded for several years, and most of the new PostGIS features rely on the newer geos versions). It has gotten so bad we’ve been considering running Postgres on our kubernetes cluster. I can’t imagine the headaches being any worse.

> Amazon Bedrock is a new service that makes FMs available from leading AI startups and Amazon via an API. Bedrock is the easiest way for customers to build and scale generative AI-based applications using FMs, democratizing access for all builders.

Democratizing, with Amazon in the middle taking a cut.

This is a word that continues to be used in product launches, but I can't seem to infer the meaning in the context it's being used.

Is it meant to just mean "made available" so they're simply saying that people can use it now? Or what is it supposed to mean? Before I've only seen it in the context of "introducing democracy" in one or another, but I don't think that's what they mean here, because no one else but Amazon actually controls the product.

> Is it meant to just mean "made available" so they're simply saying that people can use it now?

They're saying you give them money, they give you something in return. Unclear what. An api? No idea.

They mean "public access" in contrast to an alternate timeline where the best LLms are kept in-house and used in products by specific companies but not made available for developers to use in their own applications.

So if someone sells me a good are they democratizing it to me?

In a literal sense yes but you wouldn't use the term democratizing unless it was something previously made only available to a select group and is normally a part of a larger thing.

> Is it meant to just mean "made available"

Yes it's literally that. Like it or not, "democratize" is what we are using now as a convenient single word to mean "make available."

https://news.ycombinator.com/item?id=28451379

You can also search "democratized and made available" and see so many examples where it's used this way.

Someone just announced one again like this on hacker news, so I am asking them here: https://news.ycombinator.com/item?id=35558268

This seems like a weak attempt from Amazon to catch up with generative AI hype. No information at all about the model or performance.

Probably because they were caught with their pants down and only have early prototypes of generative AI models.

Am I stupid? Is there a paper for those models? What sizes? What architecture and setup? What kind of instruction tuning? RLHF?

Where are the infos?

Just click "learn more" and fill out a 20 field form.

It's like they don't want you to use it.

This is a meta comment, but it's interesting to me how fast the existence of the "AI race" has become old news. The angle of the headlines is "Amazon joins the AI race!" not "There appears to be some kind of AI race happening!"

    Wired: Amazon Is Joining the Generative AI Race
    Bloomberg: Amazon Joins Generative AI Race
    Forbes: Amazon Launches AI ... Joining Google And Microsoft In AI Race
    WSJ: Amazon Joins Microsoft, Google in AI Race
    Gizmodo: Amazon Leaps Headlong Into the AI Rat Race
    TechCrunch: With Bedrock, Amazon enters the generative AI race
    CNBC: Amazon CEO Andy Jassy on jumping into the generative A.I. race

Annoying because they stole my company's name (TitanML - https://www.titanml.co/) Fortunately they haven't trademarked it, but still unideal.

I'm not a lawyer, but if you've been using it commercially, I believe you have the trademark. Theoretically, you could sue them for infringement, although I don't know whether you could claim damages or just get them to stop using it.

In terms of terraflops, AWS likely has the largest GPU compute capacity out there, and they could use idle GPU capacity to train their models, and Anthorpic has some top tier researchers building it. Give it time.

It's not that simple. You couldn't just use any random GPU in the cloud, you need to connect them using the right kind of interconnect (Mellanox bough by NVidia) which is very expensive.

As companies reduce their tech expenditure, there will be plenty of excess capacity to collocate and properly connect by high-speed switches. There is also a lot of research (not sure it by Amazon as well) going into better distributes model training.

Indeed. The AWS p4 instances are wired together by the next generation NVLink/NLink internconnect (600 GB/s bidirectional bandwidth) and also the Elastic Fabric Adapters (400 GB/s) which is likely Mellanox under the covers.

EFA is not InfiniBand. It has similar goals (low latency via kernel bypass), but is implemented using AWS Nitro NICs and regular Ethernet networking.

Other providers (Azure and Oracle) do offer InfiniBand, but that creates a problem with "islands" of compute which can't be easily expanded. There are also more security considerations when using IB in a multi-tenant environment which makes it not such a slam-dunk advantage as you might first assume.

You're right, EFA is custom nitro NICs. Although Mellanox (now Nvidia ConnectX) is not solely Inifiband either - Nvidia offers them with Ethernet upto 400GbE as well, same speed as EFA says on the label.

I hope this puts pressure on openai to improve their SLAs.

This is nowhere near what OpenAI is doing. OpenAI to this is like AWS to Hetzner.

A more disruptive and differentiated business model for AWS will be to offer their LLMs or open source LLMs as a Managed Service like RDS.

"Sign up to learn more"

Translation: this product line doesn't even exist yet. We used GPT-4 to generate our Titan/Bedrock intro pages.

Heh, doesn't Amazon know that Titan as a code name is cursed?

Got some examples?

I would like to see some benchmarking against OpenA and Anthropic

That would happen only if the benchmarks were flattering.

Anthropic is one of the providers on the marketplace they have.

Does anyone knows if this is eligible to (aws activate)credits?

"Sign up to learn more", no pricing, a few early partners to provide glowing "testimony".

Reply on news.ycombinator.com