Show HN: Visualize Data Structures in VS Code | Hacker News Reader

Show HN: Visualize Data Structures in VS Code

github.com

509 pointsGehinnn61 comments

This reminds me a bit of `ddd` [0] which has been around ~forever. It really helped me visualize some object relationships in ThoughtTreasure [1] back in school.

A main difference seems to be you have to tell vscode-debug-visualizer how/what to show in the code, whereas with ddd you either point-and-click or else run gdb/ddd commands like `graph display _variablename_`. I guess you could probably script the latter, but I like how vscode-debug-visualizer reifies what you're visualizing into the code itself instead of into a debugging session.

[0] https://www.gnu.org/software/ddd/ [1] https://en.wikipedia.org/wiki/ThoughtTreasure

I too enjoyed using ddd back in the day. A few weeks ago I needed to visualize a complicated data structure in a Rust program that I'm working on and sadly it completely failed (http://db48x.net/reposurgeon/Screenshot%20from%202020-08-04%...). I spent some time trying to figure out how it was supposed to work (my guess is that GDB's pretty-printer is getting in the way), but didn't have much luck. I think we're going to have to reimplement some of this stuff. I ended up writing some code to dump the data structure out to graphviz. (http://db48x.net/reposurgeon/pathmap-diagrams/generated-path...)

Then this extension might be able to help you! You can visualize graphviz dot data that is stored in a rust string.

I don't really need an editor extension in order to visualize graphviz data; I already have graphviz.

All visualizations (i.e. json ~> ui transformers) are available as standalone library too. A playground can be found at [1]. The entire project is open source and can be easily extended!

[1] https://hediet.github.io/visualization/

Here's my modest plea to authors of extensions like this: please consider creating a "standalone" version, too. The AST viewer in the GIF is a good example of a tool that doesn't need the whole VSCode framework around it; it can be implemented in a single "ASTViewer.html" that can be double clicked or dragged and dropped into the user's browser, which is probably already open.

EDIT: I see comments from the author below, where there's some version of this tool online, but it goes for other authors, too, and the ability to run it from a truly standalone, single-file viewer.

I refactored all the visualization out into a standalone framework. I share your view ;) Except that it should not be a single file. That is technically neither feasible nor required.

https://hediet.github.io/visualization/ https://github.com/hediet/visualization

What makes it infeasible?

Well, maybe it actually is easier than I thought to bundle everything into a single html file (I thought you couldn't embed fonts into html, but apparently you can [1]).

Anyways, in case of this application, all visualizations together are like 20MB (the monaco editor is a significant part of it). Having them split up and only loading things one demand significantly increases startup time.

Also, base64-encoding all the binary resources is a little bit inefficient.

[1] https://www.mediaevent.de/font-in-css-einbetten/

This is the approach I’m taking with https://rx-store.github.io/rx-store/

We use http://Nx.dev to split up example apps and libs into separate packages. I have a “visualizer” package, a “devtools” (for use in browser), will potentially add an electron app etc later

The app I'm making called Repla (https://repla.app) is designed for this purpose. To be able to take the things typically done in the "preview" side of VS Code and break them out into their own window that's usable with any text editor.

> Here's my modest plea to authors of extensions like this

Not trying to be snarky or anything but... the source code is available. If there's something that you really need, please consider contributing. That goes for any open-source project.

But they have a valid point that many things could be more useful if they were thought more general.

I feel like software development could be significantly more efficient if all developers would work together and not against each other, or just with much more foresight. We have so many programs/code/libraries that do almost the same stuff. But then, we are already super efficient compared to what we could do 50 years ago.

For what it's worth, I wasn't going to respond, but the more I read back this reply, the more it chafes—maybe because I've bristled about similar replies in the past and I've tried many times to write a blog post about what I see happening here. It reminds me of a different but commonplace phenomenon that has led to the case where I no longer bother filing bugs against projects who host their bugtracker on GitHub because of the high-ish likelihood that it will get a reply in the form of a comment responding solely to an imagined subtext in my message—where the subtext is that my bug report is actually a support request. I attribute this to the bugtrackers in many projects being run so poorly that folks are even allowed to file support requests there (which is frustratingly common on GitHub) and that maybe because the reply originates from someone who lives in a world where all communication happens through subtext.

So intentional snarkiness or not, there's something in your comment that just doesn't comport with mine and the message in it. If I were directly addressing the author of this tool and, say, asking that it be changed or repackaged to solve some problem for me, then your reply would be on-target. But I'm not even using this extension, and that's not what my comment was. It was not a support request. Besides what I wrote, there was no subtext to be found, but if a person insisted on trying to root out some latent meaning from the words, then let it have been an observation about current approaches to modularity or the principle of least authority or the principle of least power.

https://www.w3.org/DesignIssues/Principles.html

[Edited - point taken ]

I apologize for having jumped the gun here. I guess your reply was appropriate for a 'Show HN' thread, since these threads are feedback requests.

Not sure how often I'll use this in day-to-day coding. Just don't find myself encountering that many of these types of issues on a daily basis.

However, I am 100% for sure going to break this out when it comes time to do Advent of Code this year. Always spend tons of times debugging code where this will be awesome.

Advent of Code really might be a great use-case for this extension!

Anyways, I think the table, the plain text and the planned text-diff visualizer might be handy for day-to-day debugging, as they fix some issues with the watch window in VS Code (mainly displaying multiline-strings and lists of homogeneous objects or just being able to update the watched expression with a shortcut).

I'm both excited and saddened by the fact that AoC is getting close! Time flies...

I'm a bit late to the party here, but I need to say that posts like this are why I keep coming back to hacker news.

Automatic visualization of data is such a powerful and underutilized tool in many disciplines, especially ours. A picture is worth 1000 words, and the same could be said for many styles of graphics.

I often find myself writing adhoc/bespoke visualizations for complex data, and the change in perspective can make an intractable problem almost easy, especially for more visual thinkers (so much so that I often wonder why I needed a graphic in the first place?!). To have this right in the debugger is an order of magnitude improvement and I'm excited about this tool, and I hope I find some time soon to learn it.

I know tools with aspects of this exist (ddd, etc..), but since vscode is my current editor of choice, I'm quite excited about this. Great job!

Great, but I feel those eye candies with e.g. growing and reshaping lists will get old fast. What would be more useful for work in the long run is a rather tacit - not distracting - hint of what's going on, unambiguous but not trying to switch attention.

If you're fast stepping through code trying to confirm it working or finding a bug, you're going to be annoyed by the time it takes the visualization to finish on each step.

That is an emphasis of the approach I was taking here: http://symbolflux.com/projects/avd

Using it feels very different from a traditional debugger where you need to control every step: it's often sufficient to just observe passively and note when incorrect patterns surface.

I really like your projects! Especially your ideas in Tiled Text. I think we should think more outside the box and question our current way to write software.

Thanks! I hope to work on a successor to Tiled Text some day. I have a better sense of how much work is involved now, though. My current approach is to do a smaller project[0] with the potential to help fund work on more ambitious projects like Tiled Text and Lucidity.

[0] https://diskatlas.com/

That's really well done; I wish I had that in the program I'm working on. I ended up traversing my data structure and making a graphviz file, then doing it again after the manipulation I was testing in order to verify that it worked correctly. That works, but it really doesn't scale very well. Plus, the being able to see the individual steps of the algorithm would make it much more clear and obvious.

As an example, http://db48x.net/reposurgeon/pathmap-diagrams/generated-path... is one of the graphs I generated.

Have you published the server and the protocol anywhere? I'd use it even though I would have to write the client side myself. For my project it would even be desirable to write the client side twice; I'm porting a program from one language to another, and it would be amazing to be able to compare the implementations visually.

I am thinking about publishing the source etc soon. I have a mailing list for the project here. I’ve never actually sent anything out on it but I will announce the open sourcing there if you are interested. More info at http://symbolflux.com/luciditystory.html

debug visualizations in fields like computer vision, 3D modeling, or really anything that has a direct visual representation very naturally pull themselves into these debug transformations much more readily than maybe regular data structures and algorithms do. The more complex the transformation the more useful the debugger visualization becomes. It probably doesn't need to be turned on all the time though, it may be an expensive proposition for sure, but when you need to bring out the big guns--by all means visualize away.

I built one of these things when working on a very complex graph algorithm a few years ago. It took a month, but without it I wouldn't have been able to spot my bug. I am glad this exists and is general.

The reason this one feels distracting to me is the animation, and the use of force-directed graph layout. Graph layout is hard but I hope they can implement Sugiyama or something instead.

Instead of animation, they could do something like showing deleted nodes in red and new nodes in green, a bit like online chess.

Maybe I'll send a PR their way..

I just used Vis.js for graph visualization without much tweaking. There is also a graphviz visualizer that is not optimized for being stable [1] vs [2]. I would gladly accept any improvements here! Please reach out if I can help you.

Implementing custom visualizations is super easy! See [3] for a very simple example. If you implement a visualization there, it will be available for all people that use the visualization framework (e.g. extensions for other IDEs), not only this extension.

[1] https://hediet.github.io/visualization/?darkTheme=1&state=XQ...

[2] https://hediet.github.io/visualization/?darkTheme=1&state=XQ...

[3] https://github.com/hediet/visualization/blob/33de54d69624e1d...

Offtopic: There was a compiler a decade ago named "Orange" or something with Orange? Orange-cpp Orange c++ or something similar for C++ that I've been trying to find that visualized data structures. I can't find it, I may be forgetting the correct citrus. Does anyone remember?

It might be the LEMON library ( https://lemon.cs.elte.hu/trac/lemon ), it does have some options to export graphs to Postscript.

I am using vscodium instead of vscode so the "Install" link on the marketplace page doesn't work and searching for the extension doesn't show it. Any ideas on how I can install this?

According to the build script https://github.com/hediet/vscode-debug-visualizer/blob/maste... it'd be

    yarn run package

and then

    codium --install-extension $PWD/whatever.vsix

so long as you have https://github.com/microsoft/vscode-vsce in your PATH

Wow, this seems like a great tool to help teach basic data structures. I'll be excited to try this out with my son.

Can't wait to try this!

Also see Glamorous Toolkit [0] where visualization is a core part of the development process.

[0] https://gtoolkit.com

This is amazing !! So much better than looking variable in the watch. Please add ruby support to this

Ruby might work out of the box if it can produce JSON strings that are not obfuscated by the debug adapter in vscode!

This looks extremely helpful for when one is digging deep into nested structures, I'm definitely going to have to give it a go. Looking forward to the C# support too.

I was just saying today how I wish I could have a graphical representation of things like hashmaps and collections while I'm debugging!

Then this pops up on front page!

This would be amazing in Python when working with dataframes.

What are dataframes?

Think of dataframes (from Pandas library) as excel tables on steroids.

Underneath dataframes are a collection of Numpy arrays with ton of extra methods on top.

Essentially an optimized array with lots of built in subroutines for statistical and numerical analysis.

they're basically tabular data where columns are homogenous in datatype, and where you can have different datatypes among the different columns. The rows are the data points or samples. They're very often thought of as lists of multidimensional data where the dimensions correspond to the columns, and the rows to the samples of the data.

This guy gives a good overview. https://github.com/mobileink/data.frame/wiki/What-is-a-Data-...

If you just use perspective (which this library uses for grid views), support for dataframes is native.

Interesting to see that decent-sized list of languages without Ruby in it. Is there some technical limitation preventing ruby support?

It should work out of the box, but nobody confirmed yet that it works with ruby ;)

In ruby if you have the right json schema set up it works. I'm not sure I completely grok yet how the data extractors would work on the ruby side yet though.

https://i.imgur.com/84mgPdF.jpg

Data Extractors usually need support from the extension. But you can configure how to construct the expression that is sent to the debugger to be evaluated. By default, it is `x => x`, but you could do `x => extract_json(x)`.

Thanks, that works. I can put something simple in my settings.json like this

  "debugVisualizer.debugAdapterConfigurations": {
    "Ruby": {
      "expressionTemplate": "DataExtractor.to_table(${expr})",
    }
   },

..and something simple in the debugee like this..

  table_example = 
    {"id"=> "1", "label"=> "One"},
    {"id"=> "2", "label"=> "Two"},
    {"id"=> "3", "label"=> "Three"}
  
  # Quick hash to visual table helper
  class DataExtractor
    def self.to_table(hash)
      meta_hash = {
        "kind" => {"table"=> true},
        "rows"=> hash
      }
      meta_hash.to_json
    end
  end

..and we get a decent hash or ActiveRecord visualizer. Thanks for the hint.

Data Extractors would also allow to offer alternative extraction options (like how visualization data should be constructed from the given value). The user can then select an alternative data extractor.

The `expressionTemplate` does not allow to do that yet.

If somebody is interested in implementing data extractors for Ruby, I can offer help!

Can you point us to where the existing extractors are implemented in the codebase?

In case of javascript, the data extractor runtime is injected into the debuggee at [1]. You can find the data extractor runtime implementation at [2].

[1] https://github.com/hediet/vscode-debug-visualizer/blob/c420f... [2] https://github.com/hediet/vscode-debug-visualizer/blob/maste...

This is cool, could definitely be useful for working with complex data structures. Does anyone know if there's a Light Table-like (http://lighttable.com/) plugin for VS Code?

It's not coming close, but a step towards light table: https://github.com/hediet/vscode-realtime-debugging

Also, there is https://marketplace.visualstudio.com/items?itemName=WallabyJ....

I myself want to explore further ideas in that direction. I think more interactive debugging/development with smaller feedback loops really increases productivity.

Fantastic work. This looks immediately useful. I'm often inspecting data structures with D3, but the ability to do this on the server side is a game changer. Can't wait to try it out.

I have a very large json structure that generates a Media Live channel in AWS. When I want add/change something it's almost impossible. I can see this tool helping me.

This is such a good idea. Why is it 2020 and programming tools don’t give us this sort of visual insight into data structures?

Look up smalltalk if you want to see what was possible in the 80s.

did everyone abandon sublime text? Doesn't seem like a lot of people are writing plugins for it.

This is SO COOL! Keep up the great work!

Powershell support would be nice.

Reply on news.ycombinator.com