Don't build AI products the way everyone else is doing it

My takeaway is to avoid relying too heavily on LLMs both in terms of the scope tasks given to them as well as relying too heavily on any specific LLM. I think this is correct for many reasons. Firstly, you probably don't want to compete directly with ChatGPT, even if you are using OpenAI under the hood, because ChatGPT will likely end up being the better tool for very abstract interaction in the long run. For instance, if you are building an app that uses OpenAI to book hotels and flights by chatting with a bot, chances are someday either ChatGPT or something by Microsoft or Google will do that and make your puny little business totally obsolete. Secondly, relying too heavily on SDKs like the OpenAI one is, in my opinion, a waste of time. You are better off with the flexibility of making direct calls to their REST API.

However, should you be adding compilers to your toolchain? IMO, any time you add a compiler, you are not only liable to add a bunch of unnecessary complexity but you're making yourself dependent upon some tool. What's particulry bad about the author's example is that it's arguably completely unnecessary for the task at hand. What's so bad about React or Svelte that you want to use a component cross-compiler? That's a cool compiler, but it sounds like a complete waste of time and another thing to learn for building web apps. I think every tool has its place, but just "add a compiler, bruh" is terrible advice for the target audience of this blog post.

IMO, the final message of the article should be to create the most efficient toolchain for what you want to achieve. Throwing tools at a task doesn't necessarily add value, nor does doing what everyone else is doing necessarily add value; and either can be counterproductive in not just working on LLM app integration but software engineering in general.

Kudos to the author for sharing their insight, though.

I'm not so sure about the "train your own model" advice. This sounds like a good way to set your product up for quick obsolescence. It might differentiate you for a short period of time, but within 6-12 months (if that), either OpenAI or one of its competitors with billions in funding is going to release a new model that blows yours out of the water, and your "differentiated model" is now a steaming pile of tech debt.

Trying to compete on models as a small startup seems like a huge distraction. It's like building your own database rather than just using Postgres or MySQL. Yes, you need a moat and a product that is difficult to copy in some way, but it should be something you can realistically be the best at given your resources.

To my mind an "x product" is rarely the framing that will lead to value being added for customers. E.g. a web3 product, an observability product, a machine vision product, an AI product.

Like all decent startup ideas the obviously crucial thing is to start with a real user need rather than wanting to use an emerging technology and fit it to a problem. Developing a UI for a technology where expectations are inflated is not going to result in a user need being met. Instead, the best startups will naturally start by solving a real problem.

Not to hate on LLMs, since they are neat, but I think most people I know offline hate interacting with chat bots as products. This is regardless of quality, bots are rarely as good as speaking with a real human being. For instance, I recently moved house and had to interact with customer support bots for energy / water utilities and an ISP, and they were universally terrible. So starting with "gpt is cool" and building a customized chatbot is to my mind not going to solve a real user need or result in a sustainable business.

So when you've taken 6-12 months to ship and everybody already iterated twice by directly using a hosted model and is building a real customer base you are only at v0.1 with your first customers who are telling you they actually wanted something else and now you have go and not just massage some prompts but recode your compiler and tool chain and everything else up and down the stack.

Perhaps if you already know your customers and requirements really really well it can make a lot of sense but I'd be very sceptical about "given how easy it is to do, why are you not validating your concept early with a fully general / expensive / hosted model". Premature optimisation being root of evil type stuff.

With that said, I disagree with the sentiment of the author. If you're a developer who's only used the ChatGPT web UI, you should 100% play with and create "AI wrapper" tech. It's not until you find the limits of the best models that you start to see how and where LLMs can be used within a traditional software stack.

Even the author's company seems to have followed this path, first building an LLM-based prototype that "sort of" worked to convert Figma -> code, and then discovering all the gaps in the process.

Therefore, my advice is to try and build your "AI-based trading card grading system" (or w/e your heart desires) with e.g. GPT-4-Vision and then figure out how to make the product actually work as a product (just like builder.io).

One simple example are e-mail clients. Somebody asks for a decision or clarification. The AI could extract those questions and just offer some radio buttons, like:

  Accept suggested appointment times: [Friday 10:00] [Monday 11:30] [suggest other]
  George whats to know if you are able to present the draft: [yes] [no]

Human resources could use AI to screen job applications and let an AI resarch additional information about the applicant on the internet, and then create standardized database entries (which may be very flawed).

I think those kind of applications are the interesting ones. Not another ChatGPT extension/plugin.

Having said that, the actual recommendations the article offers are pretty reasonable:

- Do as much as you can with code

- For the parts you can't do with code, use specialized AI to solve it

Which is pretty reasonable? But also not particularly novel.

I was hoping the article would go into more depth on how to make an AI product that is actually useful and good. As far as I can tell, there have been a lot of attempts (e.g. the recent humane launch), but not a whole lot of successes yet.

Unfortunately, most products I've seen so far feel like solutions in search of problems. I personally think the path companies should be taking right now is to identify the most tedious and repetitive parts of using the product and looking for ways that can be reliably simplified with AI.

Isn't that building AI products _exactly_ the way everyone else is doing it? There are things in the world the internet doesn't know much about, like how to interpret sensor data. There are lots of transducers in the world, and the internet knows jack about most of them.

Adding AI and then doing a poor job isn't necessarily creating a lot of value. So, if you follow the author's advice, you might end up spending a lot of money on creating your own models. And they might not even be that good and differentiate you negatively.

A lot of companies want to add AI not just because it looks cool but because they see their competitors doing the same and don't want to differentiate negatively.

And those models don't have to be LLMs. It's still a valid approach to use a smaller BERT model as a text classifier.

It's interesting to me that there are apparently companies that won't let OpenAI see their data, but will let a random startup see it. What's going on with that? Does OpenAI have a lax privacy policy or something?

* Never build your own model unless you have proven your model, and you have expertise to build it. Generic models will take you long way before cost/quality becomes an issue. Just getting all the data to train an LLM will be pain. 1000s of smartest people are spending n Billions to improve upon it. Don't compete with them. and if downstream you believe open source or your own is better use it then.

* Privacy is overrated. Enterprises are happy to use Google Docs, Office 365 exchange and cloud and ChatGPT itself. Unless you are in a domain where you know it will be a concern, trust Azure/OpenAI or Google.

* Let it be an AI startup. It should solve some problem but if VC and customer want to hear AI and Generative, that's what you are. Don't try to bring sanity in hand feeding you.

I love speed and frequency of shipping but sometimes thinking about things just a bit, but not too much doesn't always hurt.

Sometimes simple is using a standard to keep the innovation points for the insights to implement.

Otherwise innovation points can be burnt on infrastructure and maintaining it instead of building that insight that arrives.

Finding a sweetspot between too little, and too much tooling is akin to someone starting with vanilla javascript to learn the value of libraries, and then frameworks, in that order rather than just jump into frameworks.

Woe is me, it takes minutes to go from user-designed mockup to real, high-quality code? Unacceptable, I tell you!

But seriously, if there are speed improvements that you can make and are on the multiple-orders-of-magnitude then I do get it, those improvements are game-changing. But also, I think we're racing too quickly with expectations here; where minutes is unacceptable now when it used to take a human days? I mean, minutes is still pretty good! IMO.

pluggable llms is not something most people look in a product - its rare case of enterprises that are afraid of everything atm, also finetuning should be considered for costs not really range (i have not seen examples of finetuning that couldnt be achieved via prompting in llms, if someone has any, would be great to hear)

selfdriving cars are actually going for e2e systems now, there is still infra/architecture and perhaps even different models that comm with each other, but regardless system behaves as e2e solution that learns internal hierarchies of all the signals/tokens

large supersmart llms is where the game is currently at, they will get smaller, faster and more efficient we already are on this curve - if you're building llms i think you will fall behind

outer leafs of your org fueled by ai - while yes, this model is good for already established companies yet its also super important to think "ai first", ai much like any prev technology can be attached to legacy solutions making them better or can be shaped into a completely new unimaginable before solution... i wouldnt give vanilla advice that discourages the latter

> We then used the location of the images as the output data and the screenshot of the webpage as the input data. And now we have exactly what we need — a source image and coordinates of where all the sub-images are to train this AI model."

I don't quite understand this part. How does this lead to a model that can generate code from a UI?

At the end of the day, having app in AppStore is OK as long as you can accumulate something the platform company cannot access (social network, driver network, etc). OpenAI's thing is too early, but similar thinking might be applicable there too.

Meanwhile, we used to sit around the office while waiting on compilers, after which we could see if recent changes actually worked.

Now?

"5 minutes of a spinning cursor for my design specification to result in usable software?! Ridiculous!"

This article acts like the risk was something the creators cared about

all they wanted was some paid subscribers for a couple months, or some salaries paid by VCs for the next 18 months

in which case, mission accomplished for everyone

Build a moat y'all - or be prepared to potentially shut down!

> Instead of a whole toolchain of specialized models, all connected with normal code — such as models for computer vision to find and identify objects, predictive decision-making, anticipating the actions of others, or natural language processing for understanding voice commands — all of these specialized models are combined with tons of just normal code and logic that creates the end result — a car that can drive itself.

Or, as I like to say it: what we now call "AI" actually refers to the "dumb" part (which does not mean easy or simple!) of the system. When we speak of an intelligent human driver, we do not mean that they are able to differentiate between a stop sign and a pigeon, or understand when their partner asks them to "please stop by the bakery on the way home" -- we mean that they know what decision to take based on this data in order to have the best trip possible. That is, we refer to the part done with "tons of normal code", as the article puts it.

Needless to say, I am not impressed by the predictions of "AI singularity" and whatever other nonsense AI evangelists try to make us believe.

This led to a broader realization: the song wouldn't exist without my initial concept and the nuanced curation involved in its completion. It's not merely that AI executed 90% of the work; it's that my 10% contribution leveraged these advanced tools to achieve a 90% outcome, a testament to the power of technology in amplifying human creativity.

In a world where websites, businesses, and SaaS tools can be launched in mere minutes, it's becoming increasingly clear that ideas and the ability to effectively harness technology will be paramount. This shift raises fascinating questions about the future of creativity and the evolving role of the human in the creative process.

My key message is this: "So what if your business heavily relies on OpenAI models?" The unique prompts you craft hold intrinsic value. They don't diminish the time, expertise, and knowledge you invest in shaping the results. Take designing a 3D chair using an AI system, for instance: achieving optimal results hinges on your ability to precisely describe what you need, a skill that itself depends on your understanding and knowledge of design. In this context, delving into classics and broadening your educational horizons is more crucial than ever. It equips you with the nuanced articulation needed to harness AI's potential fully.

P.S. An AI model assisted me in crafting this comment, but the experiences and insights I've shared are my own, as is the majority of the words in this text. The advantage I gain from AI is the better articulation of my ideas. This tool is akin to a dictionary, a grammar checking tool, or a system that translates my native tongue into English.

No thanks. I have an actual job & customer needs to tend to. I am about 80% of the way through integrating with the OAI assistant API.

The real secret is to already have a viable business that AI can subsequently improve. Making AI the business is a joke of a model to me. You'd have an easier time pitching javascript frameworks in our shop.

Our current application of AI is a 1:1 mapping between an OAI assistant thread and the comment chain for a given GitHub issue. In this context of use, latency is absolutely not a problem. We can spend 10 minutes looking for an answer and it would still feel entirely natural from the perspective of our employees and customers.

And then came along "You need to integrate Blockchain in your business processes."

And now is the time for "make your products smart with AI" season.

It’s fundamental value.

It’s who is creating value that cannot be destroyed. Who owns the house is determined by who builds the foundation first, and that means those that control the ecosystems.

All others will play, survive, rent, and buy inside of those ecosystems.

If you’re not building fundamental value, you are an intermediary, which may be huge companies, but ultimately companies built on others. If you don’t own the API and the customer, you’re a renter. And renters can get evicted.

Those opportunities may still be worth chasing, but we shouldn’t get confused or over complicate what’s going on or we risk investing and building straw houses when brick was available.

Nothing wrong with that. Respect to success. But let’s keep fundamental value in mind, as it’s the most important thing for first generation technology companies.