Diagram Maker: Open sourcing IoT visualization | Hacker News Reader

Diagram Maker: Open sourcing IoT visualization

aws.amazon.com

299 pointske4qqq60 comments

Shameless plug: If you're looking to develop a web app with diagramming functionality and need considerably more features, I make GoJS: https://gojs.net/latest/index.html

In addition to basic stuff like zooming and moving nodes GoJS has undo management, data binding, templates, lots of built in tools and layouts and a large showcase of customizations of each, animations, palettes, overviews, etc. It's not free, but if you're looking to use a library to buy developer time, I think it's a much better deal than this offering.

How much pushback/confusion do you get on the naming of this, given that Golang seems to have taken over all the other go<add stuff here> names?

Surprisingly little. Though for a small time the Go releases were almost the same version numbers as GoJS so I'd have a mini heart attack when go-lang made HN frontpage.

(for us, Go stands for Graph Object)

In that case you should rename the project GObjectJS to avoid further confusion :-)

I used to think gevent was a Google made library about event handling. Turn out the g stands for greenlet.

I agree gojs is confusing as it seems to be "go-lang implementation in javascript" but I believe no one can claim the ownership over the word "go", as it's an normal English word as well...

That’s pretty cool. :)

About five years ago I built something similar but with a single focus of building interactive network diagrams.

We would link individual nodes to backed APIs that would get triggered by certain network events and cause the nodes to animate (e.g. traffic flow, ACL rule hit etc). We would bind buttons on the diagram to send requests to external APIs to trigger requests etc. We also had things like packet counters and traffic flow diagrams on the canvas that would show additional detail.

We also repurposed it later to visually build networks of VMs: we would lay out the nodes and networks on the diagram, configure OS, network address settings etc and then hit a button to convert the diagram to an Ansible playbook that would make it all real.

That was a really fun project - I should dig out that code and play with Terraform some time. :)

Thanks, I'm after product like that right now. How does it compare with PlumbJS and JointJS? Are there any more competing libraries that you are aware of?

My use case is complicated as I'm playing in someone elses sandbox with constrained JS and DOM API's (that's what Salesforce calls it when they liberate you of software).

Hi, The discussed topic is in the area of my interests and strictly connected with the challenges in the company I work in! I've been working with the GoJS library for many years, creating advanced diagram apps. Multiple comparisons with other libraries made me assume that the performance and the number of features spoke for GoJS. Actually we are just launching a product that is based on GoJS. If you would like to see what we made out of it, feel free joining the webinar: https://multiplayertool.synergycodes.com/

Thanks for this!

Does anyone know if there is a python equivalent?

This seems very specific to their IoT products, so it's weird they gave it such a generic name, like "Diagram Maker", which makes it seem like a competitor for something more generally capable like diagrams.net/draw.io.

This tool is not limited to AWS IoT products even though that is where this tool was born. You can use this tool for rendering any graph like structure like diagrams, flowcharts, etc.

See more details here: https://awslabs.github.io/diagram-maker/why/features.html#da....

Every third word in this article is IoT though.

Having a declarative API is really nice. Are you aware if anyone else doing this?

Why would you use this over diagrams.net/draw.io, which integrates with far more tools (Google Apps, Atlassian tools, Sphinx) and can take text input (Mermaid), and embed editable copies in other file formats like .svg/.png?

The target audience of this tool consists of application developers that want to create new applications that require their end users to author graph like structures. They can use this tool and avoid re-implementing basic interactive features like drag & drop, panel management, zooming, undo/redo, etc. Examples of such applications within AWS would be AWS IoT events & AWS IoT Things graph (existing customers), AWS Step functions, AWS Cloudformation (not using the library as of now). Examples of such applications outside AWS would be: https://www.stackery.io/, https://www.voiceflow.com/, https://parabola.io/, etc.

The new AWS glue builder interface uses this as well

It'd be great being able to program an echo routine with this. What's available there is lacking in functionality.

Tools like this can be very useful in supporting operational use cases in addition to documentation. The tricky part with cloud is how to effectively visualize infrastructure that is composed of a variety of services and attributes. If you have a VPC in each of two regions with dynamodb, s3, ecs, alb, third party providers with access via cross account roles, etc etc it gets tricky to sensibly lay out and demonstrate relationships between the indivdiual components.

Hopefully this will get traction and they will start to build in some of this intelligence and perspectives into the application (presumably as a plugin).

Right. When a tool like this can actually "compose" a dataflow using cloud native components (lamdbas, messaging, storage) in real time, this has potential to be super powerful.

Think of, for example, Apache NiFi [1] but instead of the components being constrained to a single JVM execution, the components are realized as AWS or Azure services. That "operational view" of your architecture would be super neat, especially when statistics about each component can be visualized in real time.

[1] https://nifi.apache.org

That's honestly eerily close to what is in my head with what we're building for Refinery. It's a rough nut to crack but we're trying!

https://refinery.io

Neat. Thanks for the link!

Whoa this is pretty cool!

So, like Yahoo Pipes for the 2020's?

It is impossible to express all of the relevant information about an architecture in a single view. That's why a diagramming tool that natively supports multiple views/perspectives is a must-have. Anything less is just a toy, frankly.

> It is impossible to express all of the relevant information about an architecture in a single view.

Yet lots of people have been doing it, successfully, for decades with pen and paper (equivalent). I think you need to constrain your use case from "all architectures" to whatever it is you are trying to describe.

While I totally agree, I think we rely on people just making sense of the diagram based on context.

Say you're diagraming the architecture of the aforementioned app and you have your VPCs and your ELBs and compute and subnets and IGW and tidy little boxes containing them all to indicate this is everything in prod AWS account, us-west-1. Then you need to add an S3 bucket and dynamo db table. Most people would just drop a bucket icon outside of the VPC and label it...maybe dynamodb sits inside the region box and s3 kind of straddles it. Good enough conceptually to talk through the design.

Now imagine you want to automate the process of generating that view (or the associated resource graph) automatically. That's where it gets tricky. Tons of service-specific rules start to come into play. Kind of a mess.

Furthermore, you have your security groups, roles, permissions, and policies. Plus you have build-time concerns (CF templates, CodeBuild builds, images, etc.). You simply cannot fit that into one view and have it make any kind of sense.

And those are just static relations. For interactions and data flows, you need sequence diagrams.

I wrote about this last month: https://blog.ilograph.com/posts/fixing-aws-architecure-diagr...

This is good stuff!!!

Link to Github repo: https://github.com/awslabs/diagram-maker

For people who are curious to see some of the functionality, there are a few demos linked from the documentation here:

https://awslabs.github.io/diagram-maker/explore/demos.html

Given that AWS continues to spin out more and more purpose-built technology as needs come up, I wonder whether AWS's long-term strategy is to:

- Continue to spin out products to support common business use cases.

- Expose commonly used functionality from products via this WYSIWYG tool.

- Allow drag and drop programming by connecting services.

- Lambdas are the primary way that a business bridges gaps between OOB functionality and business requirements, but are still connected up via this WYSWYG tool once created.

It would be a while before it was useful to FAANG, but maybe mom and pop businesses could cheaply partner with AWS experts to ship custom software?

I'm thinking the same. If this happens, it will be amazing.

I'm Building it now. I imagine hundreds of others are as well.

Looks promising, but it's hard to tell without a more full-featured demo.

I'm comparing this to Node-RED [0] which is pretty robust and has a large ecosystem. Ultimately, I think the value in diagramming platforms is not just in the software, but the community around it that creates plugins/extensions for different use-cases.

[0] https://nodered.org/

I was hoping something better than Mermaid. Mermaid is great, but esp. its Gantt chart is too simplistic.

Hey wiradikusuma, thanks for the reference to Mermaid. I've been looking for a week for something like this to let us drop diagrams into our internal website. This looks like it will do about everything we want.

Your point about Gantt carts may be right for you, but for our simple timelines it's good. The real winner for me was the state diagrams. Thanks!

Have you considered PlantUML[0]? There's plenty of examples on planttext[1].

It's my go-to tool whenever I need to document something visually, whenever I have a meeting with my peers and I want to get them quickly up to speed with the process I have in mind, I just create a quick sequence diagram and we can hit the ground running.

[0] https://plantuml.com/state-diagram [1] https://www.planttext.com/

I've been using a product Websequencediagrams, to create the sequence pictures. There is issues around the licensing, and the company wants to sell a a service. I'm in a lab well behind secure firewalls, so a non starter.

There is also the desire to have documents have more types of pictures/ drawings. A Swiss Army knife of drawing tools. In our case it's ok if some of the styles are not as powerful.

Thanks for the link!

If you go to the Github repository, it's more understandable and it's actually pretty nice. I wish my company had a use case for this, so I could play with it.

For instance, with Diagram Maker, application developers can enhance the experience for end customers by enabling them to intuitively and visually build cloud resources required by cloud services such as Workflow Engines (AWS Step Functions) or Infrastructure as Code (AWS CloudFormation) to get the relationships and hierarchies working.

Does this mean that Step Functions and CloudFormation will adopt this library? Both already have similar visualizations.

From the screenshots this looked like it was built with [0] or [1] but from their repo it's actually completely homegrown.

[0] https://github.com/uber/react-digraph [1] https://github.com/wbkd/react-flow

Similar project in Java: https://github.com/miho/VWorkflows

that sure is an abuse of json

Sorry, dog. Gonna need a markup mode.

If you want something not-Amazon, try Whimsical.com or miro.com (I adore Whimsical though)

Somewhat surprised they didn't build this on top of something like JointJS..

For those without experience why is this a surprise?

JointJS documentation makes me wanna cry.

re-inventing graphviz

I've tried so many of these diagramming tools (mermaid, draw.io, figma, lucid chart). But none of them seem to ever match the effectiveness and simplicity of PowerPoint. Why reinvent the wheel so many times?

PowerPoint is anything but simple.

If PowerPoint works for you, that's great! I have no problem with it. But it doesn't work for me. There is a true need in this area for better diagramming tools. Plantuml and graphviz were good ideas, for which we still lack a killer execution/implementation.

I thought the same, even Visio is more complex then PowerPoint- PowerPoint has highest productivity level for me personally in this regards. Funny to say that.

draw.io is pretty simple though and very easy to open up and use.

I use a mac and am not a big fan of opening up office products for simple/quick things

I gotta add excalidraw to this list aswell, so powerful to be able to sketch up diagrams fast, and the collaborative aspect works perfectly.

Take a look on Heimer app.[0]

[0] https://github.com/juzzlin/Heimer

Omnigraffle is the only one that has been more productive than Powerpoint for me.

I typically use a whiteboard, dry erase markers, and a camera (usually my phone's) and then upload images. It is fast, even works via webcam, and I get the same level of control I'm used to having in-person.

Try Miro. It's nice. And you can collaborate easily.

I use yEd [0] for live editing in meetings. It has auto-layout. Not intuitive but faster to use than everything else. The web version is easier.

[0] https://www.yworks.com/products/yed

I wouldn't put Diagram Maker in the same bucket as these tools. These tools can be used directly by an End user for drawing diagrams via drag & drop or via markdown. Diagram Maker on the other hand empowers application developers to embed a tool where end users can create diagrams by drag & drop within their applications.

Some of the reasons why an application developer might prefer using Diagram Maker over using draw.io's embed mode in their application: * You want to customize the editor to match your application's styling. * You want the editor to impose your application's custom semantics, for example: your application doesn't allow cycles.

Reply on news.ycombinator.com