Together’s $20M seed funding to build open-source AI and cloud platform | Hacker News Reader

Together’s $20M seed funding to build open-source AI and cloud platform

together.xyz

73 pointstim_sw22 comments

This is the company behind RedPajama, which is a VERY big deal.

It's the most realistic open source attempt at recreating the Facebook LLaMA LLM, from scratch, in a way that supports commercial usage.

They released their full 2.6TB training dataset last month, and it's significant: https://simonwillison.net/2023/Apr/17/redpajama-data/

They've also started releasing new, commercially-usable openly licensed LLM models trained on that data. You can try one of those out here: https://huggingface.co/togethercomputer/RedPajama-INCITE-Ins...

I would also check out their 3B model. I tested it on launch with LoRA fine-tuning and found it to be surprisingly capable despite its size. I think a lot of people are skipping past testing it because it only has 3B params.

Edit: https://huggingface.co/togethercomputer/RedPajama-INCITE-Bas...

Well, MPT-7B is also commercially usable and openly licensed: https://www.mosaicml.com/blog/mpt-7b

Yeah it's really promising. It's partially trained on that RedPajama data.

So is https://github.com/Lightning-AI/lit-llama

I've noticed a big trend by commercial entities claiming to be "Open Source" but actually charging for licenses. Just because your code is in a public repo on github doesn't mean it's open source. If it's not an open source license it's not open.

This is especially true of AI projects.

This is the org behind RedPajama. So, to date, they have been open.

Open source is overrated, in my opinion. I’d rather a healthy commercial entity driving development and supporting it, with visible source that I can modify as a big plus.

The whole OSI open source thing is nice, but it’s not going to matter to me much either way. I’m more worried that the open source project is no longer maintained or unsupported or development is slow and it gets obsolete. Your values, and thus your opinion, may be different.

Big tech seem to really dislike this model.

E.g. the whole tantrum Amazon had over the Elastic Search relicensing or Google briefly banning SSPL from their code hosting platform.

Since you mention it in connection to Together/RedPajama, are you saying that what they are publishing isn't actually open source? Most of the stuff I see them publish is licensed under Apache-2.0 which is a FOSS license, so I find it weird to mention it here.

Vipul's razor... https://razor.sourceforge.net/

He's well known for producing open source.

Didn't OpenAI promise the same thing when it first came out? I'm not fully invested in the space yet...so if that was completely different than RedPajama I apologize.

The homepage reads like PR speak to me.

Does someone here know concretely what the product will likely be? Is it a replicate/modal/banana competitor, an OpenAI/Anthropic competitor, a Stability AI competitor, or something else

$20M for a seed round? What’s their moat? How will they earn money?

Their outlined "mission" seems to be:

> to empower innovation and creativity by providing leading open-source generative AI models and a cloud platform that makes AI accessible to anyone, anywhere.

Since it's generally hard to earn money on the first part, I'm guessing they expect to make money on the latter part, assuming "anyone, anywhere" meaning people that can do business with US entities + people who can afford their service.

How is this different from let’s say huggingface?

IMO, this sounds more like an excuse to cash out the $20M check

How is it different from Stability AI and other platforms ?

Different investors.

Stability AI is also no longer open. They seem to be commercializing SDXL and it's not open - "yet".

> Together’s $20M seed funding to build open-source

This is a huge contrast. Stability AI’s bread and butter is proprietary, not open source. Together seem to have a track record of open sourcing both code and models.

To be fair, Stability AI have open sourced StableLM, which is welcome. But overall Stability exaggerate and lie about their openness.

It doesn't sound like it's trying to build a single model, it wants to build a network/community of them. One of which is their own. Probably sharing/coordinating training resources and other support systems? The things that are usually out of financial/technical scope for most smaller OSS AI projects. I could see smaller companies with open models spending to be a part of that or other financially backed OSS projects.

Maybe it's more hands-on than just a hosted service like huggingface? I don't really know the market, so just speculating.

No mention of their alignment strategy, if any. I feel like alignment is even more important in open models since it’ll be much easier to train a “but evil” version, right?

Reply on news.ycombinator.com