Cyclotron: A web application for constructing dashboards
cyclotron.io
cyclotron.io
A dashboard framework has been tried many many times. Here are some old ones just on the top of my head:
* Shindig aka OpenSocial aka Google Gadgets
* J2EE portlets
* OpenUSS
I'm not sure why Shindig aka Google Gadgets failed but one of the complaint we had on a product that I helped build that used Shindig is that business users did not want the complexity of customizing a dashboard. That is hard coded panels were good enough.
I guess just like many tech stuff (slack/irc, wiki) the timing and execution just wasn't right. I think now that might change given the plethora of devops, data driven biz, and generally improvement in tech awareness.
It would be nice if Cyclotron had some sort of spec (particularly language agnostic) but I guess that failed hard for the above.
For my own small company we use Kibana (the old one that doesn't require NodeJS), Grafana, and Jenkins.
Yes we use Jenkins as a dashboard. Jenkins is actually shockingly a good place for a dashboard because every time we build/deploy or kick off some kind of job we see the current status of stuff. I'm not sure why cloudbees hasn't taken advantage of this. I honestly think it is the best place to put devopsy like dashboards.
The other nice thing about Jenkins besides it actually kicking things off is the notification. IMO notification is almost more important and more useful than dashboards. The two should be not be far apart but for some reason in most systems they are.
Easy enough to 'just get the thing done', but as complex as you want it to be.
I suppose you are confused because of my mentioning building product based on shindig. That was a previous employment many moons ago. I now am a many hat wearing owner of a SaaS.
While I'm speaking of Grafana I find Kibana for more useful probably because we have a lower level of traffic than big boys but still produce an atrocious amount of logging. Logs are damn useful. In some ways metrics are actually a subset of event aggregation (ie logs) so I think Kibana is more useful at first.
The incoming events have a spec, like Riemann, and is compatible with Graphite. Godot2 also has it's own mini query lang [1], which allows for easy selection of events for the dash, e.g.
service =~ 'health/machine/disk/%' and status = 'ok'
Basically you create plugins and make sure they are compatible with the dashboard view plugin.
The nice thing about jenkins is the contributed plugin packaging system. I'm not sure how Cyclotron plans on doing that stuff or if it will ever.
I think the biggest issue is no one in our organisation has a clear idea what a 'dashboard' is exactly and how people are supposed to use them.
For some people a dashboard is a static page that gets refreshed periodically. I.E. "Lets show the current line speed in big bold 48pt font size and auto-refresh the webpage every 15 seconds."
For other people a dashboard is something interactive (drill down, selectively trend values, modify trend scales on the fly etc).
So there has always been this tension between the two use cases and we always end up with these halfhearted things (usually built using some BI package like COGNOS) which are the worst of both worlds.
At the same time a dashboard has never really been in high demand. So I think people like my manager have really failed to articulate the value a dashboard would supply.
The most promising thing I have seen recently is "Coresight" by OSISOFT (we use their PI plant historian system pretty extensively). They gave a demo that integrated control system type information (Conveyor status's, Flow meter readings, Tank levels etc) with location derived GIS data to show realtime updating maps etc which I thought was pretty cool.
What do most people use for this kind of use case, assuming changing of the values needs to be scriptable?
There's always some arbitrary software mentioned that is often barely related even tangentially and expect someone else to reply with the info.
If you want to know how it differs, YOU look it up and if the outcome of that research is interesting enough YOU post it. Don't expect other people to do it for you, or worse, make it so that the creator of something that might not even be the one linking it feels like they have to justify why they open sourced something.
Is Grafana "barely related even tangentially"? No it is quite related.
Have the creators of Cyclotron considered how it compares to existing tools? Probably, and they should come forward and be open about it.
No software developer is obligated to do anything for any of us. It's on US to figure things out and share them as we see fit. Nobody cares if you are willing to invest serious time looking into it or not. Asking a leading question and linking to something most of us have already seen is pointless. Defending it when a good argument was presented is also pointless. I hazard these discourses are linked to age.
Further, there are always new people looking to learn and grow. They shouldn't be looked down upon simply because others got there first.
What makes you think the question was a leading question rather than a genuine one? There isn't enough there to definitively determine the intent in any case, so it's up to whoever responds to decide which way of reading the question they will respond to, and that's an opportunity for the larger software community and for the developers of the projects in question to nurture thier communities. Just because it doesn't have value to you doesn't mean it doesn't have value to others.
It's not the most articulate way of starting the conversation but it is effective.
Quite frankly, if you want to know it takes five minutes to head over to the respective docs and get a rough idea, as I just did. If you're super-motivated you can even then share your conclusions and tell us "How does it compare: like this, this, and that" (which I won't).
But just shooting a question away like that is not helpful and comes across as lazy.
And lazy? Since when is asking questions is lazy?
People are reading a lot of intent from what was a very plain and straightforward question. What I think would be valuable would be if someone shared why the mere presence of such questions is somehow detrimental to the point that they can't simply be ignored, which I don't think has yet been explained.
Would you read a scientific paper on well-established topic X if it omitted a comparison to related work?
X, here, being dashboards.
To me, it's the spirit of your post that drives many away from software development. This holier-than-thou attitude of how dare someone ask a reasonable question of people who might be able to easily answer, or direct to an answer.
Someone asking a question like this isn't saying, "Hey, next person that comes by, do some research for me" it's saying "Would someone that has already done this research mind sharing their findings?"
Progress in knowledge base communities is driven primarily by building on existing knowledge. The more readily knowledge is shared, the faster progress will be. So having a culture where anyone seeking answers must redo the groundwork that has already been done by others in order to "earn" the answer is an impediment to progress within that community.
¯\_(ツ)_/¯
Grafana is very hard to extend with new visualizations. It uses "flot", a JavaScript library that hasn't seen a commit in Github in 2 and a half years. The documentation is poor.
There is definitely a need for dashboard software that allows for more complex custom visualizations -- I'll be looking closely at this.
Full disclosure, I work for Grafana, but seemed relevant to mention.
Here's examples of the final result [0], it's better than I could have hoped for.
Grafana is
> Go 33.9% TypeScript 26.4% HTML 15.0% JavaScript 14.2% CSS 8.5% Shell 1.6% Other 0.4%
Cyclotron is
> CoffeeScript 45.1% JavaScript 30.5% HTML 15.4% CSS 8.9% Nginx 0.1%
Both are using Angular 1, installed via bower ([Grafana is on 1.5.8](https://github.com/grafana/grafana/blob/master/bower.json), [Cyclotron is on 1.4.9](https://github.com/ExpediaInceCommercePlatform/cyclotron/blo...)
Additionally, when Cyclotron started, it supported pulling data from a handful of different data sources, whereas Grafana was focused on Graphite. Grafana has definitely come a long way in the last year in this regard though.
Where did you see that it uses Postgres? I think it stores its data in json files.
Seriously now I wonder how Cyclotron compares.
Another open source option: Metabase http://www.metabase.com/
We support most SQL databases and a bunch of others, plus upcoming support for Google Analytics.
https://github.com/SmashingDashboard/smashing/wiki
You can store models in SQLite:
https://github.com/SmashingDashboard/smashing/wiki/How-To%3A...
http://img.pandawhale.com/post-73940-Solving-Imaginary-Scali...
(e.g. regular sql database/redis/sqlite/etc)
I just want to pay for a hosted version that isn't going to be $1k monthly and require me to build "plugins" to do something that is akin to an API call.
In the startup space, there is also Mode [2] which has reasonable starting pricing at $25 per user per month.
I've used all three, and my personal favorite is Power BI. They are all great though.
[0] https://powerbi.microsoft.com/en-us/pricing/?b=1
Sorry for a shameless plug, but you might check something that I built and launched today [1] as I felt exactly the same pain point.
One key is getting decent render performance on low-spec hardware (i.e. a Rasberry Pi). Not something you usually think about when building/evaluating a web app, but those low-spec "boxes" crawl on most modern web-sites.
Yet there shouldn't be any reason why a dashboard couldn't be rendered on them!
I wonder how Cyclotron performs with it's JS heavy front-end on those types of dedicated dashboard devices?
So while most framework models are oriented around their own ORM, a dashboard is probably better off modeling a set of translators, and providing tools for adding new translators. Translators can take a random API, and convert into a normalized data type the rest of the framework can understand. And a good community will have a lot of these translators available.
Additionally, the dashboard framework can provide a variety of standard visualizations for the translators to send normalized data. Time series, histogram, pie chart, text, etc. And provide an intelligent layout engine for arranging a collection of these on a screen. Dashing, for example, allows a user to re-arrange dashlets.
tl;dr -- it _is_ a webapp, but the more limited scope allows for better tooling and your valuable engineers can focus the more productive uses of their time, like writing new visualizations or, more likely, new translators.
* I'd default to the "light" theme, especially if you're presenting it against dark background. It will highlight the content better. * You could use some whitespace between the widget titles and the content. The text is sliding into the graphs. * Might be nice to use more than two colors for the graph content, or at least make one of them different from black, which is also the color used used for the tables and titles.
[shameless plug] We're also building an open-source dashboards framework https://github.com/kittoframework/kitto
https://github.com/square/cube
Those can be embedded into dashboards
Edit. To be clear I mean forums where business intelligence, data science, dashboard folks hang out. I work with Tableau and they have a big community, I am looking for sites less tableau centric...
http://www.cyclotron.io/faq.html:
"Can I Use Cyclotron to Host Dashboards on the Internet?
It is not recommended to deploy Cyclotron as-is in a public setting. Cyclotron was designed to be used in a corporate intranet environment, rather than for public dashboards. In addition, it's possible to create malicious dashboards using the extensibility features of Cyclotron. Allowing untrustworthy users to create dashboards is not recommended."
Not something I'd trust as a primary datastore.
Looks good though :D
It's very simple. It would mean my soapbox wouldn't feel so tall when I speak, and that when I take other peoples code, I might, some day, actually have to give something back as a result. As opposed to just taking it, profiting off it (like ~80% of people who 'eschew' the GPL because they can't make as much money as possible off it) and never doing anything in return.
As I'm sure you can understand, those consequences would be utterly devastating. For me. I hope you can fully empathize with my plight, so we can reach a mutual understanding. Preferably in the form of you giving me things with no strings attached.
This right here - though I would extend it to also mention those who take it, extend it, profit off of it, then next-to-never (and many times never) returning any of those changes back to the community.
If we're lucky, they contribute something else back (and those contributions are welcomed, mostly) - but never the changes which allowed them to grow and expand well beyond anyone's expectations. We all know who you are.
I understand the want to see my code used by "the big guys" (or the "up-and-comers" who become "the big guys"); it does look good on one's CV. At the same time, I also understand Stallman's reasonings and message; he has personal experiences which led to the creation of the GPL and his philosophy on software - and he has been around long enough to see how the corporate world really works when it comes to the lifecycle of software. From the birth of an application or system, to its use and adoption, to its death.
Sometimes, its the latter end-of-life phase that we seem to forget about. So often as developers we have to "re-invent the wheel" when had old proprietary software source code been released at EOL, a base to build something better would be in place. Unfortunately, then as now, software has been looked on as an "asset" to be traded and sold for profit, despite the fact that it also depreciates faster than a new car. Yet, like an old Vega, some desperately cling onto that old software hoping to profit "someday, you'll see!".
And in many cases, that source code, and even the binaries - suffers the ignoble fate of "bit rot", sitting in a warehouse on an old hard drive (if we're lucky - if not, it's on some old nearly unreadable 7-bit paper tape literally rotting away, in a format used by a computer system that hasn't been turned on - if one even exists and hasn't been scrapped for gold - in over 50 years).
Original symbolics LISP? Some of the code behind the Apollo program (heck, and that was government funded!)? Cray Research operating systems, compilers, libraries, etc - for the old Cray supercomputers? This list could be extended nearly forever.
The sad thing is, we have lost man-centuries of work in the form of software, due to want-of-profit and bit-rot - and Stallman knows this - we should all know this. Furthermore, even when the code does survive (in whatever form), inevitably, the systems that can run it do not (in general); in some cases, it has become impossible to run code written for certain machines because no examples of those machines, nor their designs (to create an emulator) exist any longer. All we can do is guess at what the program should do (assuming we have at least documentation of the mnemonics and opcodes for the machine language binary to reverse engineer - if you don't have that, nor the real hardware, the binary code might as well be noise).
Crazy enough - in some cases we have the machine and the software extant - but only one operating copy of that machine! For example, there's a company in Texas which uses an old IBM accounting machine to this day; it uses punched-cards for it's data input/output, plus it has a printer. It's "programs" are hard-wired plugboards. A computer history museum delegation has begged them to donate the system to a museum for preservation, as it is the last operating example of such a system. But the company has an attitude of "if it ain't broke, why fix it" - and you can't fault them for that. Even so, one day it will break, and likely whomever is in charge will decide to sell it for scrap rather than donate it (due to likely amount of precious metals in the machine) - and that machine, plus all of its programs - will be lost forever.
Heck - we treat our automobiles and automotive history with greater respect, and that has had arguably a much lesser impact over the last 100 years, than what computer technology has had over the last 20.
/end ranting (and book)
In other words, if you GPL license your code, and someone else uses it, they need to "pass it on" - in other words, their payment to you (and the community) for using your code is for their changes to be passed back to you (and the community).
Code paid for with more code - imagine that.
The only restriction is against those who seek to profit (almost) exclusively off the source code, in perpetuity. They want their cake, and to eat as well. The MIT and other similar licenses allow this - they get to take the code, extend it, profit off of it, and never give back the changes that made the system better (and arguably forked it).
And if you're lucky, you get mentioned down at the bottom of a copyright/license agreement in 4pt font that's barely readable.
<snip> The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. </snip>
https://www.mskcc.org/blog/how-new-particle-accelerator-will...
Megatron, anyone ?