DeepWiki: Understand Any Codebase | Hacker News Reader

DeepWiki: Understand Any Codebase

aitidbits.ai

231 pointschildishnemo53 comments

It's a pity that there is no clear way to send takedown requests. We didn't ask for deceptive garbage to be generated as documentation for LibreOffice, but here it is and newbies are discovering it: https://deepwiki.com/LibreOffice/core/2-build-system (spoiler: LibreOffice has never used Buck as a build system)

Out of curiosity, how come LibreOffice has .buckversion, BUCK, .buckconfig, etc? This commit[1] does seem to indicate using Buck to build at one point, though it is 10 years old.

[1]: https://github.com/LibreOffice/core/commit/1fd41f43eb73c373c...

Indeed and the BUCK file was updated 2 months ago. It definitely does use BUCK. Maybe it's not the main build system, but it's not hard to see why someone or something unfamiliar with the project would think that.

Really there should be a comment in the BUCK file explaining what it is.

Ok, I submitted a patch with a comment.

It seems like it uses Make to build the main thing and then Buck is used to publish Java API to Maven Central, rather than building. Which is kind of a side thing, I guess code organization wise it would be better if Buck was not included on the root level? But Buck requires it? Although not a huge deal, but explains at least how LLM thought it was equivalent in importance.

From my experience using deepwiki what it generates is not deceptive garbage

I sent them a politely worded threat and they responded right away opting me out:

> Hello, I am writing you as an author of Open Source software seeking to protect my security and that of my users.

> What I would like to know is: how may I prevent deepwiki from indexing my projects, specifically those in the ----- GitHub organization? If you consider yourselves to have implicit legal permission to train on my projects and write about them, know that I hereby explicitly and permanently revoke that permission.

> Since you likely believe that I lack the authority to get you to stop, I will add this:

> To the extent allowed by law I will consider any incorrect information you publish about my projects to be libelous and, given this notice, made with your intention. LLMs have no will to act, so publishing misinformation about my project, at such time as that happens, could only be the result of human will.

> Kind regards, > Conrad Buck

Just because you "consider" incorrect information to be libelous does not mean that it is. While it's true that LLMs have no will to act, the use of an LLM to publish information that ends up being incorrect does not imply that the user of the LLM intended to post incorrect information.

In that universe, there is no accountability at all. There is no way the system of law would allow that.

I admit that in my message I was careful not to make an accusation. I only stated my disposition towards their actions.

IANAL, but I don't believe that lack of intent to cause harm means that relief can't be granted. You could still take legal action if they refused to remove the information. To my knowledge, a question that still hasn't been thoroughly tested from a legal perspective is to what degree users of LLMs should be reasonably expected to be aware of the potentially for false information and to what degree continued use despite that knowledge constitutes willful negligence.

Humans can make mistakes too when compiling information and when mistakes are done unintentionally without the intent of causing harm, I believe liability is typically limited. I would expect the same would be true of LLM use. As long as the user of the LLM has taken reasonable precautions to ensure accuracy, I think liability probably should be limited in most cases. In the case of DeepWiki getting something wrong, I think the case for significant reputational damage is pretty weak.

It's reasonable precautions where it seems to me they are likely to be what I would consider partially to wholly negligent. The lack of an ability to opt out, for example.

Agreed that there should be an ability to opt out.

Yuck, based on tone alone I would ignore an email worded this way.

Sometimes you have to get angry to set boundaries.

Anyone human who puts their name to their words is free to write about my work.

If their wiki was just a wiki with human and AI contributorship that would have been a much better product and I probably would have been fine with it.

The DeepWiki tool itself seems pretty neat. It has a pretty good go at collecting documentation from across the codebase and organising it in one place. It has a pretty good guess at coming up with documentation which isn't there.

It strikes me as an example of automated code assistance that's e.g. more useful than "the item under the cursor has type <X>, here's its documentation".

There are things which benefit from being automatically described, and there are things where "the map is not the territory", and you do want someone to have come up with a map.

> "Treat it like a patient senior engineer."

I trust that LLMs are patient (you can ask them stupid questions without consequence).

I do not trust LLMs to act as 'senior'. (i.e. Unless you ask it to, it won't push back against dumb ideas, or suggest better ideas that would achieve what you're trying to do. -- And if you just ask it to 'push back', it's going to push back more than necessary).

> It strikes me as an example of automated code assistance that's e.g. more useful than "the item under the cursor has type <X>, here's its documentation". ... I trust that LLMs are patient (you can ask them stupid questions without consequence).

DeepWiki does add tremendous value already: I maintain open source projects and frequently direct volunteers to use DeepWiki to explore those (fairly convoluted) codebases. But ... I've caught DeepWiki hallucinating pretty convincingly far more than once just because a struct / a package / a function was named for something it wasn't doing anymore / wasn't doing it by the book (think: RFCs, docs, specifications etc). This isn't a criticism more than it is of the refactoring practices by the maintainers themselves. "Code readability" and tests (part of "gyms" for Agents), then I imagine, are going to come in clutch for OSS projects that want to benefit from a constant stream of productive external contributions.

I'm pretty dubious that the value it adds is in fact positive.

I first thought to look at DeepWiki for a large project--LLVM, specifically--and the results ranged from incomplete to just plain incorrect (see https://news.ycombinator.com/item?id=45020628 for more details). Okay, to be fair, maybe looking at a multimillion line codebase is just too much for an AI tool, although it's arguably one of the use cases they're gunning for.

For a smaller project, I rummaged through compiler-explorer since I once poked around in that codebase. And when looking through its description of the property files (specifically https://deepwiki.com/compiler-explorer/compiler-explorer/3.3...), I noticed that it has some very subtly incorrect description of what they do, the kind of mistake that's likely to boomerang on you only a month or so later.

I then decided to see the quality of the ask-a-question system. At this point, I happened to be poking around CLP from COIN-OR trying to gauge how accurate it was about the simplex details, and I noticed it mentioned pivot tolerance here: https://deepwiki.com/coin-or/Clp/2.4-factorization-and-linea... . Playing a newbie, and given that it doesn't really explain pivot tolerance, I asked it to explain it in detail. Now the correct answer is essentially as follows:

> When you are factoring a matrix A, the algorithm is essentially dividing a column by a particular value in that column (the "pivot"), and then adding that column to every other column in the matrix. To minimize the accumulation of error in the calculation (i.e., for "numerical stability"), you want that value to be as large (in magnitude, i.e., ignoring sign) as possible. When you are working with sparse matrices, you have another problem, though: you want to avoid converting a zero entry into a nonzero entry ("fill-in"), and the pivots that minimize the amount of nonzero entries you create aren't necessarily the ones that minimize numerical error, and thus there's a tension between these two goals. The pivot tolerance factor is the ratio of the smallest-magnitude value you are willing to accept to the largest-magnitude value, which controls how much numerical stability you are willing to sacrifice for the goal of minimizing fill-in. [And there's a natural follow-up question, what value is reasonable here, for which I have yet to find a satisfying answer.]

This was DeepWiki's attempt at answering the question and some follow-ups: https://deepwiki.com/search/what-does-pivot-tolerance-mean_7... . It doesn't really attempt to explain what it does beyond a handwave towards "numerical stability" (there's no discussion of sparsity, for example), instead wanting to focus on how it's adaptive. It also has the general feeling of a student trying to pad their paper to meet a minimum length requirement. Finally, it doesn't answer the question "what is the default value" (I think the answer is 0.1, though I don't know the codebase well enough to confirm that the places where it's setting that are the actual initial values).

At pretty much every step of the process, DeepWiki has given me answers that are distinctly worse than what I would have found just traipsing through the code myself, and given the need to confirm whatever an AI says for correctness given their propensity for hallucination, it's hard for me to say that on those occasions where it was correct that it would have been faster. I do realize that navigation of large code bases is a skill that seems to be in short supply, but I have yet to see evidence that AI is good at covering for that.

I'm trying it out with a repo that has no code comments or documentation. Let's see how it does. :)

Edit: its been like over 10 minutes, and still nothing, thats interesting, I did choose a Lingo source project so it probably gave up by now.

I hesitate to dump on Deepwiki, because what it does is to some degree impressive and timesaving (especially the system diagrams).

But for my libs (that aren't super popular, but OTOH have a few million downloads per year) it generates documentation that is incorrect, and this is not good for users.

The Elkjs project uses this and im not really sure i like it. [1]

Its a bit hard to find stuff. I was looking to find the structure of the main configuration json object and couldnt find it in the deepwiki.

I found it on the “non ai created” doc page of the main Elk project[2] (Elkjs is a JS implementation of Elk)

But yes this is of course just one data point.

[1]https://deepwiki.com/kieler/elkjs/5-usage-guide

[2]https://eclipse.dev/elk/documentation/tooldevelopers/graphda...

"Uses it" sounds strong.. I don't see any link to it from https://github.com/kieler/elkjs?

Annoyingly, anyone can just.. request a deepwiki for any GitHub repo. That one exists doesn't mean that it's endorsed or reviewed by the project.

They just kind of barged in, welcome or not. Just another SEO slop-spammer.

Hmm you are right, there is no direct link.

I seem to have found it via a google search.

So in the end people will believe that these are the official docs…

A couple of years ago the OCaml and Julia languages already had to deal with a content farm that created wikis for them, filled them with LLM-generated, blatantly wrong or stupidly low quality content, and SEOed its way above actual learning materials. Cue in the newbies to these languages being incredibly confused.

This at least tries to generate the text out of the actual project, but I'm pessimistic and think it'll cause similar confusion.

So I decided to look at some open source repos I know decently well. The only one that seems to have a wiki is LLVM (https://deepwiki.com/llvm/llvm-project).

Thoughts on the overview page: Okay, weird subset of the top-level directories. The high-level compilation pipeline diagram is... wrong? Like, Clang-AST is definitely part of clang frontend, and you get to the optimization pipeline, which clearly fucks up the flow through vectorization and instruction selection (completely omitting GlobalISel as well too, for that matter). The choice of backends to highlight is weird, and at the end of the day, it manages to completely omit some of the most important passes in LLVM (like InstCombine).

Drilling down into the other pages... I mean at no point does it even discuss or give an idea of what LLVM IR. There's nothing about the pass manager, nothing about expected canonicalization of passes. It's got a weird fixation about some things (like the role of TargetLowering), but manages to elide pretty much any detail that is actually useful. The role of TableGen in several components is completely missing--and FWIW, understanding TableGen and its error messages is probably the single hardest part of putting together an LLVM backend, precisely the thing you'd want it to focus on.

If I had to guess, it's overly fixated on things that happen to be very large files--I think everything it decided to focus on in a single page happens to be a 30kloc file or something. But that means it also misses the things that are so gargantuan they're split into multiple files--Clang codegen is ~100kloc and InstCombine is ~40kloc but since they're in several 4-5kloc files instead of a large 26kloc file (SLPVectorizer) or 62kloc file (X86ISelLowering), they're simply not considered important and ignored.

That’s a very intriguing observation.

(I haven’t read how it works but…) I wonder if removing file sizes, commit counts, and other numerical metadata would have a significant impact on the output. Or if all of the files were glommed into one large input with path+filename markers?

Yeah this is my experience too. For projects I know well, the diagrams are not engineering quality.

Nonetheless I still think it's impressive considering that LLVM codebase is one of the most complex ones to be found in the wild.

Ok, yeah, this feels like a reasonable use case for AI. I generated a DeepWiki from one of my repos, and it's pretty informative. It goes into way too much depth on some trivial details, and glosses over more important stuff in places, but overall it seems to have produced a pretty detailed summary of what the package does, and why it does many things the way that it does.

Deepwiki was instrumental in our refactor of a large codebase away from playwright to pure CDP @ browser-use. Huge props to the team that built it, I regularly refer to it as one of the few strictly net positive AI coding tools.

The auto-overviews and diagrams are great, but where it truly shines is the "deep research" follow-up questions system at the bottom. It's much better than using OpenAI deep research of perplexity to ask questions about complex codebases like puppeteer/playwright/chromium/etc.

I want to respond to our last interaction here.

This is Cordyceps [0]. My Chrome extension port of the typescript port of Browser Use needs some love but it is contained there.

I wanted to post the port of a MCP with pure CDP libraryI found to use Chrome.debugger therefore removing Playwright but the typescript and monorepo tooling is out of date. Hopefully I'll get it done tomorrow.

[0] https://github.com/adam-s/cordyceps

Nice work! You should reach out to one of the other team members on discord / X about it and make sure they see. (I am on pat leave right now and trying not to open Slack)

My next step will be record / replay of user workflows. I started looking at rrweb and did a search on Github for rrweb and chrome.runtime to look at how people use it in a Chrome extension. workflow-use appeared on the first page of search results. I'll delve into that tomorrow.

Enjoy your time off!

(I just opened up the workflow_use and noticed the AGPL license. Given the situation, it is best I don't even open or look at the code. I'm going to have to figure this one out myself. I did watched the video, very cool stuff.)

workflow-use is currently just an early implementation of some of the ideas for repeatable workflows, there's not much secret sauce to worry about from an IP standpoint yet. you could also use deepwiki to safely paraphrase it / get an overview of the architecture.

It is still a dangerous grey area. If Browser Use is acquired by Mega_Corp for the $10B its IP is worth, I would have to prove that my product isn't a derivative of Workflow Use, otherwise, I, or a company that contracts me, am obligated by law to link to Mega_Corp. If there is prior art as you say, I will be able to find and use that as long as it uses a less restrictive license like MIT.

deepwiki is awesome and my first time learning about it

Isn’t this supposed to be a short technical blog? Why does it seem like they’re a salesman and it’s a sales pitch?

> "We are generating more code than ever. With LLMs like Claude already writing most of Anthropic’s code, the challenge is no longer producing code, it is understanding it."

The first sentence already is obviously AI generated, and reading through it it, it is obviously completely written by AI to the point of it being distracting.

I understand the author probably feels that AI is better at writing than they are, but I would heavily recommend they use their own voice.

I’ve personally started to try to think about the points someone prompted an AI to generate some text (the actual thoughts of the author) so that I can more easily skim past the AI generated slop such as: "… you’ll get the env setup, required services, and dependency graph with citations to README, Dockerfile, and scripts, so you can hit the ground running".

Though I agree with you, both of the sentences you cited (the first two in the piece) have mistakes in their English and wouldn’t be written by AI.

very good review (and yes: deepwiki is just amazing!!!)

I would love code could be opensource: I just saw now a couple of attempts

* https://github.com/AsyncFuncAI/deepwiki-open

* https://github.com/AIDotNet/OpenDeepWiki

with several stars

I really want to like deepwiki, but just looking at the diagrams of repos, they are too handwavy to be useful.

They are a conceptual overview and don’t seem tied down enough to the actual implementation details of a particular project.

Perhaps this could be the improved.

> Suppose you find a clever mechanism in another repository, such as an authentication flow or a clever way to persist state locally. In that case, you can ask DeepWiki to provide a Markdown cheat sheet: a breakdown of how it works, which files define it, and what it depends on. You can then drop that summary directly into Claude Code or Cursor as structured context and ask it to implement it in your project.

Bonus if the LLM was trained on the original repository.

Then it would be that much more clear that you're just laundering open source code.

It would be nice if it could also read github issues etc if they were available, so it could have more context about the decisions that were made. For large projects/those with a lot of issues this might be a bit impossible though

Who is paying for those tokens? I had a long conversation with "Devin" and I must have burnt up a large number of tokens. In any case, thank you "Devin"

> my go-to tool for understanding unfamiliar codebases

Somehow voices are a little easier to pay attention to than text to me.

I had done a show HN for https://gitpodcast.com earlier, created with a similar goal in mind.

neat system

tried it out on Mixxx; https://deepwiki.com/mixxxdj/mixxx

don't know how to zoom the diagrams on mobile tho, n they can easily almost disappear from view when panning around

the prompt box could do with a way to move it out of the way of the bottom couple of cm in portrait, or from covering more than a quarter of the screen in landscape orientation

I'm wondering about the "any" part. How does it perform on big codebases? Like ~20 repositories with lots of classes.

What if I don't trust this third party with my code? are there any open source/ local way to run this ?

This is my way to do it:

1. Archive whole repository into single text file with Repopack: https://github.com/yamadashy/repomix

2. To reduce token, compress the file with LLMLingua-2: https://github.com/microsoft/LLMLingua

(fewer token = more context can be given to LLM = LLM better understands your repository)

3. Copy & Paste the compressed archive text contents as a context, into ChatGPT’s input field as-is, or local LLMs.

4. Ask the LLM for documentation generation. Like, “this is a repository source code. given context, generate a ‘table-of-content’.” Then you will get a ToC. If it looks good, you can ask for generating first chapter. And keeps going until you finish whole documentation.

If you are trying to document Typescript/Javascript codebase, You may use bundlers like esbuild for step 2, which will beneficial for token reducing.

If you interested in step 2’s LLMLingua-2, check out my Typescript port that can be ran without no installation at: https://atjsh.github.io/llmlingua-2-js/

Is deepwiki related to wikis in any way, or just grifting on the name?

Yeah, it's not a wiki at all However, in my experience, many people just know the term 'wiki' as a short form for Wikipedia - and for many, Wikipedia is the only encyclopedia they know. So, I guess that the Deepwiki authors see it as some kind of deep/specialised encyclopedia.

I just use context7, they launched api recently. It's my goto solution for coding agent docs.

Gemini and chatgpt and github copilot subscriptions also provide similar functionality.

I recently received an AI-slop bug report for a small open source project (PureLB) that I maintain, and the slop was generated by DeepWiki. It was very incorrect, but I didn't know what "DeepWiki" was so I wasted about an hour. If DeepWiki is causing garbage bug reports even on tiny projects like mine, I can't imagine how much maintainer time it's wasting over all.

for anyone curious: https://gitlab.com/purelb/purelb/-/issues/147

Reply on news.ycombinator.com