williamzeng0

Unfortunately it doesn't, we typically don't expect to handle high load with this cache and actually disable it in production with another envvar.

Sometimes caching can actually be slower for certain functions, because just performing that operation is faster than pickle.load/pickle.dump.

I think the marginal gain would be a lot less for Jupyter notebooks, but I've definitely rerun individual cells and wasted time there before.

I think it could help if you forget to save the output of a function within a single cell like this:

1. print(f(x)) # -> check what happened 2. out = f(x) # -> turns out we want to save this, so we have to wait again

This is great, thanks for making it!

In our benchmarks, open source models still have a long way to go (just breaking past the GPT 3.5 barrier now). We're excited to move to open source models when they're ready, but right now they simply aren't there yet.

Even when they are, it'll still be a while before you can run such a powerful model on your laptop. Thanks for the feedback!

Nice to see this open source :)

I'm really excited about E2B. Axilla looks great too! :)

We have that! You can directly append to our system prompt here to customize Sweep https://github.com/sweepai/sweep/blob/main/sweep.yaml

Appreciate the correction, and apologies for the lack of clarity!

Definitely our fault on the part of this. We also deprecated the client recently, because we want to focus entirely on the GitHub issues.

Check out our discord for help! There's a couple of people trying it now :D. Happy to answer questions when we have the time, and I'll point you to the person who set it up themselves.

Haha yep, you can link Sweep a doc and it will read it though! This helps a lot for newer frameworks.

Apologies, during the launch yesterday we wrote a bug. It can be fixed by invalidating the cache (push any commit to the main branch). Sorry about that!

Sorry I think you misunderstood.

Sweep’s logic is fully open and it’s self hostable, but we’ve been focusing on the capabilities of Sweep(not on self hosting) so we haven’t provided docs.

Because Sweep runs entirely in GitHub, it’s easy to install but annoying to self host. You’d need to setup modal and create a new github app.

Definitely doable, some of our community members were able to do it.

Haha, we need good senior devs to review Sweep’s PRs :)

Thanks for the suggestion, we’re actually going to spend the weekend hacking at a user configurable step that handles different types of tests.

I’m personally a fan of letting users insert their own instructions per repository, and Sweep can use that for each PR.

You could do that, but we have a non-commercial license so you'd have to look out for that.

We have had users in our discord set it up themselves, so it's possible. If you go through us we retain your repo in a hashed redis cache (no plaintext code stored) and store some (~50 line) snippets in our logs for debugging. We don't train on any of your code. You can find our privacy policy here: https://docs.sweep.dev/privacy

Here you go, https://docs.sweep.dev/privacy.

There's also an interesting discussion here about how it works: https://news.ycombinator.com/reply?id=36990160

We do end up doing a GPT based rewrite. The initial description is really valuable too though, and we want to keep that throughout the workflow. It's kind of similar to a spelling correction or query intent system. If it's high confidence you can override their query, but ideally you use the original one too.

We use some simple ranking heuristics detailed here: https://docs.sweep.dev/blogs/building-code-search

One thing we also do is match any files mentioned in the issue. So if you mention sweepai/api.py, we'll find that and add it to the fetched files. There's still more work to be done here, so look out for those!

Likely file name based scoring, and other rules + finetuned retrieval models (opt-in of course)

Haha, Sweep also handles small features and writing documentation! There's a huge scope of software work besides fixing bugs, and to be fair, most of the work in bug fixing is not in the actual coding but thinking about the problem.

Cool! Sweep actually already does this before the PR is shown to the user. I agree it might help to expose some of the Sweep generated reviews (which we did before)

That would be me, completely happy to take the blame here :) We manually update the content here so the code works just fine for now.

Something I'm a big fan of is making a small successful ticket (for example migrate the functions in one file) and then applying it map-reduce style across the entire codebase. This could help a lot, and by definition addresses repetitive work.

We have this (to some degree) with issue templates, where you can pre-populate some text and fill in the rest. We're also thinking about good ways to offload that work to Sweep.

I didn't mention this point, but we actually do that during the modification. We ask the LLM to extract the necessary subcontext from the main context. It doesn't increase the costs much, but it does help performance because the unnecessary context is stripped away.

Yes Sweep does! It's through file comments and PR comments. We also handle failing GitHub actions.

Modal is great, it's been able to handle us chunking 10k files/second. Most of the costs come from embedding(couple hundred to embed tens of thousands of repos a month). Our chunker was in the tens of dollars as well.

The developer experience is also great, so we highly recommend it :)

These are nice PRs, also github.3dstreet.org is super cool! I'm glad it's passing the GitHub actions. Do you have any workflows that would be more helpful to you?

Thank you! We're working on integrating external browsing using another agent. For now we do have link processing, so if you drop a publicly accessible link in the issue, Sweep will actually gather context from that link.

You can give Sweep docs about a framework and it should help a lot.

Yep! You can leave a comment on the file just like you would review a PR. There's an example here: https://github.com/sweepai/landing-page/pull/226

That's a good question! We tried using Anthropic 100k before (Claude 1.3 was a lot worse), and I think that it's really important to figure out how to be context efficient, at least for GPT4.

My stance is with models ignoring long contexts(https://arxiv.org/pdf/2307.03172.pdf), we'll have this problem for a long time. I could be wrong though.

Also we did try function calling, but it doesn't allow for a chain of thought step. This made the plan/code way worse. Cool to see you found the same!

That's completely right, the testimonials will look really strange if the names have 3+ words in them. That's why we're targeting really strong developers to review Sweep's PRs. An experienced dev(like you) will be able to read the code, think "hey this needs tests and edge cases" and then request changes instead of merging it.