Hacker News Reader

You should consider integrating this visualization in a few third party applications with a significant database component.

I'm thinking web frameworks (like WordPress) and CRMs (like SalesForce). Easy to comprehend visualization of business data is a lot more valuable than visualization of internal program data structures. If you can populate structures from 3rd party sources then you could have a valuable product.

I can see how a business would be interested in visualization of contact growth or website structure.

> Easy to comprehend visualization of business data is a lot more valuable than visualization of internal program data structures

I agree that must be true in an absolute sense, but part of the reason I'm making this is that I find visualization of internal program data to be valuable. I have other things I want to use this to help me build. But it's definitely a gamble whether I'll be able to convince many other developers that it's valuable :)

Thanks for the suggestion though. I do think it's interesting, but if I understand correctly, the volumes of data involved would require different sorts of visualization approaches, whereas I've spent lots of time specializing mine for high-level program state. For instance, the visualizations are specialized for common data structure types, and can be stepped backwards and forwards through, like in a stepping debugger. So if nothing else, I'd be very reluctant to pivot without really giving this a shot.

Let me know if I've missed anything about the kind of data you're thinking of—maybe it's more suitable than I think.

Well, I'm not sure it would be as difficult as you think to get the capability.

There are plenty of ORM libraries. I am familiar with python, so sqlalchemy etc for me, but most languages have ORM libraries now.

It could be as simple as some glue to pipe data from the 3rd party into native objects (via ORM) and then use your visualization efforts directly.

Your visualization looks smooth. It may require limiting visualization to certain fields, rather than every attribute as you would want debugging.

In my opinion, though, you've done the hard part.

And volume wise it seems you already have the ability to handle relatively large complex tree structures (I think your examples showed 500+ nodes). If you can hide/show sections dynamically then that should be sufficient for some very large structures.

maybe try to see if your tool makes it easy to understand viruses, malwares, ransomwares, etc.. Usually there is a time-race to understand how they work in order to find protections quickly. Also the code path is usually convoluted to make them hard to understand. A visualisation could be helpful and have a lot of value for clients

Very interesting idea... I'll look into this. I know nothing about the domain, so if you had any additional resource(s) that might be useful, I'd be glad to see 'em :) Thanks for the suggestion!

Cool project, with beautiful presentation!

I'm working on something similar, with similar motivation. I often need application-specific diagnostics and I haven't had a framework for building them. So I decided to make one.

I'm approaching LuaJIT diagnostics in a really similar way to you. I instrument the VM to snapshot key data structures (e.g. JIT code) and log them to a file. Then I make a UI that visualizes them as a graph. My snapshots are raw C data structures straight out of memory with only a subset of references crawled. The UI decodes the objects the same way as GDB i.e. using DWARF debug information.

LuaJIT example: https://github.com/raptorjit/raptorjit/pull/63#issuecomment-...

My main project for the tooling (early days): https://github.com/studio/studio

Anyway. Maybe our paths will cross further down the line. Looks to me like your work is more targeted, more polished, and that you are likely to find an excellent niche e.g. Redis as another commenter suggested. I am more interested in breadth i.e. object graphs are just one of many problems I am interested in and so I'm always looking for 80/20 solutions so that I can move on to the next thing.

End braindump. Once again - great work, and great presentation! Open an issue on the Studio repo if you want to share your work there some time btw :)

Hey Luke, that's very cool—first thing I've come across that seems to value visualizing that same kinds of things I think are important.

You are correct that for the moment I am attempting to be more targeted and find a good niche. My first priority is to generate some passive income so I can relax a little and work on things at a reasonable pace. From there, I have some more grand ideas for this project, though I will probably remain focused on abstractions of program state. I'm curious to see what other useful things you find to visualize—I liked the execution time/trace trees. I could definitely use that.

Thanks for the comment—feel free to shoot me an e-mail if you ever feel inclined: westoncb[at Google's email service].

Get it in front of users now. You're probably wasting your time on a tonne of things that aren't relevant.

Haha yeah... You're probably right, but I feel like it's still not quite actually usable, and just around the corner it will be. Maybe that's just how the trap feels though ;)

not probably, they are right. I fall into this trap all the time, adding these complex amazing features, that, once launched nobody uses, nobody ever asks about. The 1-2 people who find out about them are amazed, the 99% don't give a damn.

Get something working and throw it out there. Then find out what actual paying customers want, then go from there.

Although that isn't cast iron advice as early take up customers tend to be less risk adverse (they are paying for an unknown software) and perhaps more technical than others, so you run the risk of making a very technical program based on their feedback, which then when you get popular alienates users because unless they were with you from the beginning, they have no idea what half the features do, because obviously you didn't document things well enough.

Or so has been my experience, but perhaps I am an odd case, still creating desktop applications and don't give trial versions. You pay the money you can have me contactable all day and get your features added. If you haven't paid any money, then I don't care. I'll take word of mouth sales every day of the week

I'm in love with this, I have nothing constructive to add but you should be really proud of this work. Reminded me of this video: https://www.youtube.com/watch?v=8pTEmbeENF4

Think you're on to something that this talk points out very well.

Thanks! That's funny, before starting at a programming job a few years ago my soon-to-be-employer (at a tiny YC startup) said to me, "you can be like our own little Bret Victor!" I should re-watch that one though since I only have a vague recollection of it. I enjoyed his "Inventing on Principle" talk quite a bit.

That video reminds me of a comic I sketched out years ago. It's about this rag tag armed group of hackers who specialize in intelligence espionage in the near future. There's a payload specialist, security specialist, data structure specialist, etc. Much of the drama unfolds in VR space where the hackers can be seen frantically querying/manipulating data structures directly by hand whilst evading detection. It's supposed to be educational as well, explaining CS topics through stories. Think the movie hackers plus GitS, but with attention to accurate portrayal of CS knowledge. It's basically my vision of what the future could be like.

As a developer, I would love to have something like this. If you can make the visualization easily traversable using simple text queries, that would make it even more useful. Heck, there is value in simply being able to query your runtime data structures anytime. So monitored data structure is in itself a very cool idea.

I highly recommend coming up with a small enough use case to ship sooner so that you can build something that users want and pay for. Also, if you can document the client/server API, other devs will write monitored data structures for you.

> If you can make the visualization easily traversable using simple text queries ...

That's a cool idea. I've been working on something similar, but that takes kind of a different angle which I like. What I have at the moment is a way of specifying filters for which properties should be displayed or not.

Yes, small enough use case... unfortunately the smallest one I can think of is MASSIVE :P

I will document the API and probably open source the client before long. Clients pretty much just send serialized Operation objects as JSON. Operations are like add/remove to/from whatever location, using a shared addressing systems for all data structures.

Redis provides data structures to be stored inside its key-value database. It also allows you to setup hooks to listen to whenever something changes. You can potentially use it to connect client/server. Wrappers around client objects will sync with Redis whenever anything changes, and your visualization tool will listen to changes in Redis to update. The advantage with this is that some aspects of your system will benefit from years of testing and thought that went into building Redis.

Oh dang, cool! I will definitely look into that. Want me to send you a message or something if I end up doing it? My email is in the contact section on my webpage.

Very interesting concept. Excellent job. As others mentioned, it can be extended in to other attractive domains than code visualization. Developer tools don't tend to be as financially rewarding.

Reminded me of a conversation with my friend over a year ago about using VR for data visualization. His son had sold a VR startup, another of his friend was working on DNA/RNA visualization with VR, and he was exploring dashboard data visualization with VR.

> it can be extended in to other attractive domains than code visualization

I'm glad people are pointing this out to me because I hadn't really given it any thought. My first instinct is that the data being visualized here is somewhat unique in how its structured—but if nothing else, once I get graph/state machine visuals in, I could see that probably being useful elsewhere.

With a lot of the dashboard kind of stuff though (which I admittedly haven't had so much exposure to), it seems like line graphs etc. are more appropriate from what I can tell.

Very slick. I see the value as a developer looking at my in memory data. Not sure on the best monetization route but nice work. A way to import JSON might be a nice feature where it could be used by non-developers to visualize data and navigating as well. Good luck with whatever path you take.

Thanks Attila! That's reassuring to hear—I've mostly gotten responses that it's unclear how it could be useful, though that may be largely due to my unclear communication. (That also relates to my older video, since I haven't shown this one to many people yet.)

As for monetizing, I'd like to sell it as a cheap utility for programmers or CS teachers (those would be two different products). If I can make enough to visit the dentist regularly and maybe just supplement with part-time grocery store work, I'll consider it a success. Of course I think it has the potential for more than that if properly executed on...

While I watched the videos I could not help but be impressed, it looks really great. Just letting you know!

Thanks! Much appreciated.

You could make some fun tutorial videos with this; I'm thinking visualizations of how basic data structures work, tree balancing, linked lists, hash tables internals, perhaps sorting algorithms, git commands...

I like your thing. It makes visualization easy for people making videos about programming. Rationalizing code can be very social. Livestreaming your programming is a thing, right? Jonathan Blow does it all the time: https://www.twitch.tv/naysayer88

Your visualizations empower the livestreaming coder.

Also all of the YouTube channels that put up tens of programming videos every month. I'd love to watch coding tutorials and courses that use that kind of visualization.

Thanks muthdra. That is definitely something I've been thinking more about with this. Initially I was just thinking university professors might want it—but what about open source project maintainers who want to introduce new programmers to their code? It would be easy for them to get visualizations of their core parts of their programs.

I haven't seen too much livestreaming, but I'll check out Jonathan Blow's now!

Nice! It would be interesting to visualize a React hierarchy this way. Is your display client using some sort of "open" protocol that you could hook other things?

That would cool!

> Is your display client using some sort of "open" protocol that you could hook other things?

Pretty much, except it's technically the server which does the rendering etc., and the clients send over information about data structures (as JSON 'operations' which say thing like add/remove to/from some position in a data structure). It probably would not be very hard to write a client which observes React programs and sends that info over.

You should re-record the entire intro video and leave out all the 'um' and 'uh's. It makes the product seem far less professional.

It's fine. What makes it seem unprofessional is the low sound quality.

Amazing job, man. You should be proud.

this looks awesome - beautiful work so far

I bet you could make money on this now, today, by creating a code consulting service, where people would come to you with "that one bug they could never fix". They would send you their code, you would go through the debugging with them, and provide them with a video of your application showing how the data moved around and explaining to them the bug. Like debugging as a service. This wouldn't scale, unless you brought on a team of consultants who were all able to use your buggy debugger, but it would be lucrative.