Show HN: Linux server monitoring web dashboard | Hacker News Reader

Show HN: Linux server monitoring web dashboard

github.com

175 pointsafaqurk68 comments

I believe the memory assessment will always show basically no free memory due to Linux's memory management and caching. It would make more sense to subtract out the cache to see what's really used and what's just being (smartly) cached.

You are right. I'll have to make that adjustment.

Just linking for anyone wondering. http://www.linuxatemyram.com

Nice interface, but it's a bit strange to me that it uses shell exec for every kind of measure.

e.g. # uptime.php <?php echo (int) (shell_exec('cat /proc/uptime')/(60*60));

Yea, that's my #1 concern now that I know its worth pursuing as a more well developed product. I'm hope to step away from PHP totally for the next iteration. This was more-so a proof-of-concept for the sake of feedback.

I was thinking about not forking processes to read a file, not to avoid the use of php, if you're comfortable with that.

What ideas do you have about reading the stats? You'll have to use a server side language anyway (or do you plan to read data from a third party service with a restful api?)

edit: if you want to have a zero-install tool I'd suggest to go with python plus bottle (a one file webframework), so that you would be able to clone and use it from any directory.

Yes, seems incredibly insecure approach, although the interface is beautiful.

Be careful of any externally controllable strings which might allow XSS and give panel access to an attacker.

More of a problem for admin dashboards which have two-way control, which this one must if it can do on-demand refresh.

That's why I prefer munin static pages.

Very cool.

How difficult would it be to create widgets that show application-level stats like number of visitors on site or conversion rates (e.g. calculated from google analytics).

I'm tired of having to loging to google analytics, lulu.com, and gumroad in three different tabs....

Take a look at Dashing[0] by Shopify, it can be a base for what you want. There is already a Google Analytics plugin[1]. For Lulu.com, they are saying they are not accepting new developers[2] but if you have an API key, you may have luck building a widget for Dashing.

[0] http://shopify.github.io/dashing/

[1] https://github.com/Shopify/dashing/wiki/Additional-Widgets

[2] http://developer.lulu.com

Dashing is gorgeous and surprisingly easy to write widgets for.. Hardest part was getting the layouts right.

As of now it is a very simple application. I am going to now start planning the roadmap for this as a product. Things like an API, integration with web services, etc. I'd say a few months down the road, your requests will be very viable.

Not much different than conky...except the web service part.

http://conky.sourceforge.net/

Can I use conky to monitor a remote server? I'd like that.

Of course! ssh -X. Obviously requires conky (and a minimal X server) on the server.

Eh, I was hoping to do it without running x on the remote server.

no offense, I think that's effectively 'no'. :)

no one is running an X server on their production servers, perhaps unless somehow related to their actual purpose (screenshots, ui testing, etc.)

P.S. I agree this is pointless. The best solution in my opinion is simply ssh and [h]top or glances.

You could use the conky-cli build (without the X dependency). You could then run it and pipe the output into some file that you serve statically. Hmmm may spend some time investigating.

To follow up with this. You could add `conky -c json.conky | while read json; do echo $json > /opt/http/monitor/info.json; done` to your start. Then just static serve the `info.json` file and it should update at whatever interval you set for conky. No dynamic execution though (hard part is that lack of interaction).

Neat, I'd like this to monitor my Pi... Just because I can.

I was actually just thinking the same thing. This would make a great status page for a headless RaspberryPi.

I made similar dashboard app a year ago, written in python with extensible and configurable metrics https://github.com/Eyjafjallajokull/aboco also supports application-specific stats.

you sure you don't want to make a go at turning this into a full blown product? Imagine a more modern version of webmin. Very nice.

Webmin still works and is actively maintained. There even are good-looking themes nowadays. It would be a huge endeavour to re-implement all that came into webmin in the past ten years.

But Webmin not satisfies the NIH-Syndrom ;-)

Exactly my intention: get feedback and then iterate towards a more viable product. I have quite a few more ideas for the interface and options. Definitely going to pursue it.

Better idea: output the output of ohai and then handle the display in JS.

Could you add installation/setup instructions?

There's no installation, just git clone and move to your www directory.

$ git clone https://github.com/afaqurk/linux-dash.git

$ mv linux-dash/ /var/www/

That assumes you have a web server configured to serve PHP out of /var/www.

At the very least you would want to put some access control in front of this.

> That assumes you have a web server configured to serve PHP out of /var/www.

The example was for Apache webserver.

> At the very least you would want to put some access control in front of this.

You're right, access control would be in order. Though I was merely suggesting the fastest way to try it out.

Minor typo on the header: Dashboad

Looks really nice. Was this built for Debian-based distros?

Thanks for the correction. Yes, this version was built for Debian distros. Working on more comprehensive support for the next iteration, which will have an API.

True using PHP would also have server dependant issues as well as security issues. I feel PHP is a web framework and would suggest using a application framework for this.

I think you should remove the animation on the numbers: it prevents the user from reading quickly valuable info (and isn't really useful anyway).

Still, the interface is really cool!

Very Cool. Would be nice to show Apache access logs as well (careful parsing those:).

This looks very nice!

However, multi-server support would make this even more amazing.

Pretty awesome!

The interface looks very nice but no thanks, I'm not going to install PHP on my servers to have it. I'd totally set it up if it weren't for PHP. The risk is just too great to ignore.

Some server side code is required to obtain the data. Out of curiosity, what would it have to be written in to make you feel safe?

In anything that is not PHP. Bash would have been perfect for this use case, for example. The correct way to do this is to obtain the data separately and merely show it on the interface. I haven't audited the code, so I don't know what's being passed to the shell commands, but the fact that an attacker MIGHT be able to influence the commands being executed (see register_globals, etc (yes, I know this has been disabled by default in recent versions of PHP)) is enough for me to completely write it off as an unnecesary risk.

By collecting the data periodically (i.e a cron job) you eliminate most possible vulnerabilities.

First you say anything not PHP then you don't mention one. While I agree that off-request collection may be wiser, I fail to see anything mentioned better than PHP unless you're seriously mentioning Bash to write a webapp like this. Mentioning an old security concern that is no longer an issue seems like an easy way to discount every possible language.

I said that a bash script would be perfect for collecting data periodically and saving it to a file, I wasn't suggesting that you use it to write a web app. I thought that was clear; apologies if it wasn't.

However, you say that this is an easy way to discount every possible language: can you point me to a vulnerability on the same level as register_globals in any other language?

Server monitoring applications, by their nature, have to interact with the system by issuing the commands that will collect data. The most common solution is to separate the application in to two parts:

Back-end: a daemon process that runs at an interval, collecting data and writing it to a data store.

Front-end: connects to the data store, processes statistics, and formats for display.

By constructing your application in this way, the data store acts as a firewall between the system calls required to collect the data and the interface used to view the data. Your daemon application needn't listen on any network ports if it is running locally.

Your web application still needs to be secure, but you don't have the additional risk of many calls to system/shell exec commands that are a honey pot for exploitation.

Have you even looked at the (trivial amount) of PHP source code? Where is PHP going to be attacked? Perhaps I am missing something obvious but it seems like PHP is running some linux commands and parsing the results, not sure how this would be different/safer in another language?

Also you can put it behind http auth or restrict the vhost by IP etc

It's not how much PHP is running, it's that PHP is running _at all_. It makes me feel very, very uneasy that every time that web interface is hit PHP executes a shell command. There's something inherently wrong about that, to me.

Unless the script accepts parameters, which it doesn't, there's nothing to worry about.

Like I said in a previous reply, see register_globals and company. Sure, that was some time ago. Sure, the defaults are better now. But at some point in time the people in charge of PHP thought "yes, this is a good idea, let's do it".

...

It's not like the attitude has changed, though. There are many, many things deeply wrong with PHP when it comes to security. PHP is supposed to cater to unexperienced programmers. An unexperienced programmer might see "mysql_escape_string" and think that it will escape strings, making them suitable for use in SQL queries. The programmer will think the code is secure. WRONG. Because you have to use mysql_REAL_escape_string.

Also, look at the `e` flag in preg_replace. WHAT THE FUCK. Like, seriously. What. Why. There are no words to describe how gobsmacked I am.

And FOUR people in the PHP committee (or whatever it's called) voted __AGAINST__ deprecating it. FOUR. [1]

The point is that I can't audit (and would rather not waste my time doing so) this PHP code. The fact that it uses shell_execute when a HTTP request demands it is enough of a red flag.

[1] https://wiki.php.net/rfc/remove_preg_replace_eval_modifier

But you can totally audit the PHP code. There honestly isn't that much of it, and absolutely none of it takes user input. I'd be way more concerned if this was a Rails or Django app, because then there would be lots of library code to worry about.

It doesn't take input from the user, but it does use untrusted input in a way that allows XSS. See https://news.ycombinator.com/item?id=7128442 .

> WRONG. Because you have to use mysql_REAL_escape_string.

Using mysql_real_escape_string is almost a sign you're doing something wrong. You should be using prepared statements with PDO or mysqli.

> The point is that I can't audit (and would rather not waste my time doing so) this PHP code.

I wasn't going to bother, but this post is pretty high up on the front page. There's some XSS issues with the JSON output, the Content-Type header isn't set to 'application/json' so PHP decides to set it to 'text/html'. Now anyone that controls ipecho.net[0] or can execute commands as any user on the server[1] can XSS users of the panel.

If you'd like to confirm, go to /sh/ps.php and notice where the page breaks due to strings in the JSON being interpreted as HTML.

[0] https://github.com/afaqurk/linux-dash/blob/master/sh/ip.php#...

[1] https://github.com/afaqurk/linux-dash/blob/master/sh/ps.php#...

But the XSS issues have nothing to do with the language choice. Python, ruby and any other langauge do nothing by default to protect you against such things either.

I agree this is a poor choice of code, and an attack vector, but the language used here is not to blame.

I was responding to the parent's unwillingness to audit the code, not so much about technical issues with PHP.

The biggest security issue that I've noticed with PHP is more cultural: Developers are far more likely to write ad-hoc pages with subtle security issues than use well-tested frameworks and libraries because it seems easier.

I would never expect to see Ruby or Python code that generates a JSON array like this [0], but I'm not at all surprised when I see it in PHP. It's too easy and tempting to do the wrong thing.

[0]: https://github.com/afaqurk/linux-dash/blob/master/sh/users.p...

I agree; PHP's problem is more cultural than technical. But that doesn't mean there aren't boatloads of technical problems that can only be dealt with with unnecessary memorization (see PHP's `==`). Yes, people will tell you that "this is by design", people will tell you that you're only supposed to use `===`... I hope you see the pattern here: if the language gets in your way for no good reason then it's wrong.

> Using mysql_real_escape_string is almost a sign you're doing something wrong. You should be using prepared statements with PDO or mysqli.

I agree, you should be using prepared statements, but using mysql_real_escape_string is not inherently wrong. This is a theme with PHP: if you want to do thing X, you have three different ways, each with different naming conventions, different side effects, different APIs, and at least one of them will subtly lead you to shoot yourself in the foot.

Of course an experienced programmer can work around this issues, but I think this is not very relevant to the discussion.

>There's some XSS issues with the JSON output, the Content-Type header isn't set to 'application/json' so PHP decides to set it to 'text/html'. Now anyone that controls ipecho.net[0] or can execute commands as any user on the server[1] can XSS users of the panel.

This is the kind of thing I expected to happen. It's funny how PHP is supposed to be accessible to everyone but can only be used correctly by experts.

I know PHP started as a templating engine, and it isn't a very good one at that: all of the templating engines I have used escape HTML by default. PHP is the only one that doesn't. I don't know, draw your own conclusions.

What you are talking about are core language decisions. Good developers have known how to work around register globals for years now, and it's not been a problem in any modern web app I've touched for the last 8 years (off the top of my head).

The language being used here has absolutely nothing to do with the output. You could substitute the PHP shell_exec calls for something similar in python, ruby or any other language. You should audit anything that hits your server. Using PHP doesn't meant it's more susceptible to vulnerabilities.

>Good developers have known how to work around register globals for years now, and it's not been a problem in any modern web app I've touched for the last 8 years (off the top of my head).

A good programming language lets a good developer to focus on writing code, not wrestling with bullshit decisions. YES, good programmers can write good and safe PHP code. But in the same way, bad or unexperienced programmers WILL and DO write awful code. The most recent example is OP's code. I've clicked on a few links that people replying to me have posted, and honestly, I'm glad I didn't spend any more time looking at it.

>The language being used here has absolutely nothing to do with the output. You could substitute the PHP shell_exec calls for something similar in python, ruby or any other language.

Like I said in previous comments, substituting PHP by Python or Ruby wouldn't improve the situation very much; I agree with that. The correct way to do it is to separate the data collection from the web interface. I think that should be common sense, and I fear that PHP is eroding that by making it very easy to do it "the lazy way" (fuck it, we'll use shell_exec on the web, what's the worst that could happen right?).

I had the same attitude, so I forked it and rewrote the PHP endpoints as a Python script, which I execute outside of the context of the application via a cronjob.

https://github.com/arbuckle/linux-dash

Thanks, that's how it should be done. I don't care if the collection script is written in Python, Ruby, Node JS or Brainfuck; it shouldn't execute shell commands when it receives a HTTP request.

Checkout https://github.com/scoutapp/server_metrics (Ruby - gem install server_metrics). Grabs all the key server + process metrics, handles Linux memory cache, etc.

Requires a distro w/ "proc" functionality for the most metrics.

Thanks, this architecture makes more sense imnho. Between this and the other thread on monitoring solutions, I wonder if it would be worth it to write a snippet that takes data from sar [ed:sar/sysstat or perhaps conky] and dumps out json that works with this dash...

"The risk is just too great to ignore."

Can you elaborate on this? I'm looking for you to clarify which other dynamic languages would be considered less risky in this specific application and why.

Other dynamic languages have similar risks but at least they don't have a history[1,2] of making completely stupid decisions when it comes to security.

However, I'd rather not have a program execute a shell command when it receives a PHP request. Written in PHP or not. It's just wrong.

[1] http://www.php.net/manual/en/security.globals.php

[2] https://bugs.php.net/bug.php?id=47796

Register Global has been depreciated since two versions ago and even completely removed since the last version. Follow your own posted link [1] and it says it right on top with a big ass banner.

>Other dynamic languages have similar risks but at least they don't have a history[1,2] of making completely stupid decisions when it comes to security.

Yes like Ruby and Active Records SQL injection debacle where Github was pwned and the developers din't seem to care?

I can't argue that PHP is better than other languages, but holding it to f*ck ups from over 5+ years ago, most of which have been fixed, is a low.

If you can find something vulnerable within this open source coded, then I happy to hear you out and would prefer that than your unregistered paranoia.

Also, sorry for being harsh but this kind of attitude just deters people from contributing to open source unless they follow the 'elite' practices of the hivemind.

>Register Global has been depreciated since two versions ago and even completely removed since the last version. Follow your own posted link [1] and it says it right on top with a big ass banner.

Well, the vast majority of people would not consult the documentation for register_globals when it was the default. Sure, it's deprecated now, but... what the hell, who thought it was a good idea in the first place? Oh yeah, the same people who built PHP as a hack designed for a very specific purpose that grew out of proportion too quickly.

> Register Global has been depreciated since two versions ago and even completely removed since the last version. Follow your own posted link [1] and it says it right on top with a big ass banner.

The Active Record stuff was a vulnerability in a LIBRARY. The register_globals stuff was a vulnerability in THE CORE LANGUAGE. Those are two very, very different things. Also, it's a bug rather than a design decision.

>Also, sorry for being harsh but this kind of attitude just deters people from contributing to open source unless they follow the 'elite' practices of the hivemind.

I encourage as many people as I can to contribute to open source. I also consider myself an experienced software developer, so I feel like it is my responsibility to educate people and prevent them from shooting themselves in the foot. PHP makes it very easy to do this.

Genuinely curious, how would you implement something like a cron web interface? Certainly it would require you to have a program execute a shell command upon some HTTP request. Granted this project focuses on displaying data but there are many, many serious web libraries that execute commands on the server depending on what the person is trying to do.

Think of all the libraries that are just abstraction layers to imagemagick or ffmpeg. Do you really think there no legitimite use case for executing commands via some web program?

The libraries that are abstraction layers to imagemagick or ffmpeg are, in fact, that: libraries. There is a very critical distinction between a library/module and a command-line wrapper. Your programming language probably has a module that exposes the imagemagick API. However, this doesn't mean that it uses shell commands for anything; it just exposes the functions to your programming language of choice.

I wrote the backend and processing system for a website[1] that deals with converting files, and incidentally uses imagemagick and ffmpeg, amongst other things. You'll notice that everything that that calls external programs is handled very carefully. One example of this careful handling is that all of the commands that can be executed are in a single file[2], and very easy to think about. Compare that to OP's software, where there are multiple commands scattered across different files. Also, although this is a bit harder to see superficially, but no HTTP request triggers any external program call directly. This is by design.

Coming back to your question,

>Genuinely curious, how would you implement something like a cron web interface? Certainly it would require you to have a program execute a shell command upon some HTTP request.

You set up a cron job[3] that gathers the data and stores it somewhere like a database, or even a file. The idea here is that you have a strictly one-way flow of data, which prevents a large number of attacks (register_globals is the easiest)

[1] https://github.com/MediaCrush/MediaCrush

[2] https://github.com/MediaCrush/MediaCrush/blob/master/mediacr...

[3] Or a daemon, whatever; the point is that the website can only communicate with it through a specific channel with a very small attack surface (i.e a UNIX socket where the messages are very limited), whereas doing it on the script that generates the website exposes a very large attack surface

Reply on news.ycombinator.com