Hacker News Reader

I wouldn't really consider a wrapper around the OpenAI API to be an open source copilot. Am I misunderstanding this project?

True, GitHub copilot is also using gpt3.5 at the moment so it's actually kind of the same

You understand it fully

You do have a valid point that the LLM is closed aka OpenAI's API. We were thorn about it ourselves but decided to release with OpenAI because of it's ease of use for devs. We have already tested Llama-2 and seeing more and more pull for it, especially by enterprise who have can't use OpenAI because of privacy and IP issues. Thus, OSS LLMs are next on our roadmap :)

Rest of the stack is fully open-source and it's much more than a wrapper. Maybe the impression comes from the aspect that we've made getting started dead simple with the minimal Copilot example. The framework covers a lot of ground so you can build your copilot easily: backend, adding a knowledge base & dynamic data, front-end, monitoring & evaluation.

This is pretty much v0.1 release. What else would you love to see?

I would still like to have a model, open source at minimum, better yet with AGPL or so, that I train locally on my own code and also use locally exclusively on my own machine and that does not require me to have a Nvidia GPU and can simply be trained on CPU.

Does this exist?

There are significant safety risks with this approach.

Current foundation AI models have safeguards in place to avoid generating content that could be used for disinformation, violence, and terrorism.

While not perfect, these safeguards have meant that we've been able to use ChatGPT over the last couple months, without a major backlash due to the technology being used for destructive purposes and it potentially being closed (as has happened many times before).

Considering governments murdered literally a million times more people than civilizations over the past hundred years, it's ironic they alone get the models without safeguards in place.

The fundamental disconnect is that we as technologists believe we are entitled to technology. That technology should be open. And, for the most part, it is.

However, if you look at the most sensitive technologies, whether it's nuclear weapons or until recently cryptography, they're never open. The risk of them falling into the wrong hands is too high.

AI seems to us like any other software, as people who may understand it and its limitations. However, it's also a tool for unfathomable destruction -- not now, but in the coming years.

And so restrictions are almost certain to happen, and according to many, justified. You're not going to have access to "un-censored AI" in the coming years. That's unfortunately reality.

That is the thing though, those who historically pose the greatest risk to mankind (the heads of governments) do and will continue to be the ones with the greatest access and use of the most dangerous technology.

To put it another way, we're all concerned (at least in 1st would countries) about terrorists and hackers being the greatest threat to people's lives. So we're comfortable giving the government sole control of weapons - even though statistically that's not an accurate representation of the danger graph.

I don't know if it's good marketing or propaganda, but I find it ironic that "only government can be trusted".

Upvoted for at least being honest that gaurdrails are for brand safety, ie OpenAI not being embarrased by their bots output and having to pull the plug.

But that’s not snake oil. Seriously though, these are also my table stakes requirements for trying any of this stuff. I’m fine waiting a couple years for it, and I’m also fine if it never happens. I just want to know if it does :)

OSS Llama2 based LLMs upcoming which can be also run on a CPU so you have full privacy (inference, not train).

Why would you like to train it on your own code?

I think an ever-present all-remembering model of all code I have ever written (pointing it to the directories of my own projects) would be quite nice. For example I have example repos, where I collect how to do stuff with specific programming languages. And then have that model suggest me some blocks of code that I can be sure, that are not violating any license other than my own.

You could maybe throw something like this together, but it would be unusably slow

Do you mean the training of it would be slow? Or its usage after training?

What is the relationship to: OpenCopilot – Open source AI copilot for your own SaaS product (github.com/openchatai) | 127 points by gharbat 1 day ago | 31 comments | https://news.ycombinator.com/item?id=37203196

None, we've been building OpenCopilot for the last few months and it seems they beat us with the launch by a few days. I do hope they do well cus we believe a world where copilots can be built with open-source is a much better one vs built on a closed MS, etc. stack.

Related question: Why would you (and they) give your product a Microsoft name and open yourself up to a trademark fight and probable name change when, with just a little effort and imagination (and maybe ChatGPT), you could've called it anything else?

Why a "Microsoft name"? Are you talking about GitHub copilot? Because that's far from the first "copilot" product

I'm confused as to what this is? Is it just a wrapper to gpt API that parses the response? And it looks like it manages the convo so you don't have to send over entire history?

I have a somewhat related project that wraps the gpt API and let's you interact in a codebase, where the LLM can request access to certain files, propose new files and edits

https://github.com/breeko/j-dev

Sounds a lot like aider

The amount of cynical comments everywhere always get me.

Anyway, congrats, it will be really useful for building POCs and mvps quickly. I'll definetely give it a try and feedback once I test it!

Thanks for the support! :)

Would love to hear your feedback on what's good, what sucks and what to improve.

> Our team has been in the AI space since 2018 and built numerous LLM apps & copilots.

could you please elaborate?

Sure! Taivo here - CTO & co-founder of OpenCopilot.

The most recent LLM app we've built has been the Ready Player Me copilot (which you can see linked in the main repo as well, and which was an inspiration for actually making something open-source based on what we had learned into). The problem it solves is onboarding developers onto their platform, and is currently live for a subset of their users.

We've built a ton of copilots that we haven't published. Some have been for work like founder copilots for ourselves and a few friends, sales outreach copilots, etc. Some have been for fun: I wanted to see if I could create a copilot that coaches me to do things faster a la https://patrickcollison.com/fast - so I did. As a PoC I created a copilot that explains the rules of an obscure sport I like (floorball).

In addition, with the same team, since the beginning of this year we've made a bunch of experiments that I'm sure many others have tried: * "chat with your knowledge base" style bots: for tech support, for developer docs, for internal HR use, and probably some I can't remember * an LLM-based quiz app that dynamically makes a quiz based on your Spotify history * an automated engineer (a much more basic version of smol-ai/developer) * a website builder where you mostly uses a chat UI for making the website * making AutoGPT better (higher quality, faster, more straightforward to use) * we also briefly considered making an OSS LLM before Llama 1 came out and kickstarted the OSS LLM wild ride we've seen since then

We killed almost all of these experiments within a few weeks after release, because none found early traction; we'd probably been wrong about the problem these were solving.

About my AI background: I built a 25-person computer vision team from zero at an identity verification unicorn. After that tried and failed to build a computer vision labelling tool startup. Before that, built robot perception software at Starship robots and unsuccessfully tried to make a contribution to AI safety with my thesis on active reinforcement learning in 2017.

About Johannes (OP) & the rest of the team: in 2018 him and part of the current team tried to make Sidekik: something like character.ai, but training models from scratch - which eventually didn't work out. From 2020 (after a pivot) the same team was building Sentinel, a deepfake-detection startup - which again did not work out.

FYI: Sidekik's original website: https://web.archive.org/web/20181008082142/https://sidekik.a...

Yeah, we started Sidekik in 2018 where we were enabling people to create AI versions of themselves, including famous people such as Steve Jobs, Elon Musk, etc: http://web.archive.org/web/20191211212908/https://sidekik.ai...

It was way before all of it was cool and back then we didn't train LLMs but just LMs with LSTMs lol. Got to around 10K users but the technology was too early so we pivoted to Sentinel: https://thesentinel.ai/ Scaled that up to the largest deepfake detection provider in the world but eventually it didn't work out which is a longer story.

Can you share anything additional about your Fast Copilot? That’s such an interesting topic that I had no idea existed!

It's embarrassingly simple and of course has infinite room for improvement, but I figured it might be interesting for you to check anyway: https://github.com/taivop/speed-copilot

It's built with OpenCopilot; the bulk of the work is in the prompt (you could probably copy-paste it into ChatGPT and get not-too-bad resultS). The RAG part is not great right now since the documents are not chunked in a reasonable way. But it has been somewhat valuable for me as well.

Let me know if you tried it out and whether it helps you do anything faster ;)

Here is a bigger question: In your background explanations you mention AI projects and apps but seems like you guys are CEO and CTO of NFTPort. Any reason as to why you are not disclosing that?

Any plans to add Langchain support? Another thing for OSS LLMs could be adding support for the like of Replicate or Baseten as well.

We actually use Langchain under the hood for some things, and are in the process of figuring out which abstractions make sense to put on the SDK interface. For example, we already use the `langchain.schema.Document` class as the main document dataclass.

With making it possible to bring any LLM, we're considering adding support for bringing anything that implements the Langchain LLM interface.

In general, we're still trying to figure out how much customizability to give to the user and how (Langchain is a great option here) -- power should not come at the expense of ease of use.

What is the point of this if no RAG?

RAG comes out of the box - see here: https://docs.opencopilot.dev/improve/knowledge-base

We were hoping to make dynamic RAG part of the very first release but didn't - we'll be adding that soon.

Which LLM are you using?

Looks like gpt4

For the time being, it looks like you can choose between gpt-3.5-turbo-16k and gpt-4.

https://github.com/opencopilotdev/opencopilot/blob/d7aa8270d...

Yep, currently gpt-3.5-16k or gpt-4. We wrote the example prompts in a relatively Llama-compatible way though (we actually started building this onto Llama 1 before switching to OpenAI as default), and make few assumptions about the LLM so it's easy to switch out. Mostly this is waiting behind us adding an option to pass in any LLM, and we're planning to add support for this.

Generally, we leave the LLM up to the user -- if OpenAI or Google is a no-go, then you probably are anyway in the territory of self-hosting or even self-training your LLM, which means you're fine setting up your own inference endpoints as well.

Show HN: OpenCopilot – Build and embed open-source AI copilots into your product