Is there anything similar to glances that's native, super light. Don't have python in all production server and I wonder how weighty it would be when things are getting haywire .. or am I wrong and glances is lightweight enough?
1,051 karma · joined March 10, 2015
Always get inspired by the brilliant insightful things people do and a little sad by what little I know.
But I didn't see (perhaps I missed it) about tracing inside a single instance where I want the individual method call to be traced to know which method call was expensive within that instance.
Like hangout > chat, let's drop the existing feature of read receipt that is even more important in the remote/work world in a product geared toward business.
Let's do an integration of task & files in chat in gmail but not what's suppose to be the primary app the standalone chat.
Let's take 3 release to decide yes there is some usefulness to wireless scanning and we can put it as an advance option.
It's not just dropping produces, it's actually degradation of function as if just to bump up the changelog size.
This is just one example that I vividly recall. The simplest of operation downloading of an update, only apple can make it as big a nightmare as it was. Now they do a lot of things fabulously but when the fail they just fall on their face and ignore it or they would fail where I can't even imagine how to fail this bad.
The latest blunder (again download), my iPhone lost it's wifi after a fall and wow! behold you can't download an update over cellular. Apple will make you save your money by not using your expensive cellular data after all it's so much more expensive than your $800 phone.
1. Detail response/time taken for every single endpoint. So I want a histogram and not just an average. I have seen often you get the top X endpoints (for me mostly requests) in tools and top X by global measure. At my work the number of unique requests are a lot and the good response time for them varies a lot from 50ms to 1s easily. A global threshold is useless. There's also the fact that users have different cost for same endpoint but that I don't have a super good solution.
2. Some level of traces for requests that crosses a threshold. That means the agent has to keep the threshold for every endpoint but is that so expensive? I think I saw some ideas (in some commercial product) where they collect data and drop them if the request ends up being fast enough. I think that's a very good approach. So I want specifics when something is slow and against a per endpoint threshold. Perhaps I want all requests slower than 99 percentile, perhaps I keep 20% for > 90 percentile etc.
3. Now the per end point statistics has to be kept for significant period, months not days otherwise how would you see the change. I have things slowed down due to code change, due to usage increase, due to query plan getting whacky. You have different performance due to different level of usage (concurrency). No one can probably afford per second resolution for a year sure but if my 9am spikes are averaged out how would I know that this is an old problem or the spike actually worsened by 20% in the last two month? I think you can get away with reducing resolution a lot if you keep a histogram and not just average. But I don't think anyone optimizes for that. And you also need to keep those important traces for quite a long time.
4. I work in Java and use unstructured log and have been trying to figure out how to parse that reliably for debugging so that we don't spend hours to grep the logs. I just realized recently the most common queries are easily parsable for me users, servers/app-instance, codeLine (java loggers prints the class:line) that narrows things down so much for me that I can afford to just export those and grep if I need to. But most log tools seems like either grep & super structured. Also someone else mentioned cardinality because while the rest can be fine, user is definitely high cardinality so everything might break down there.
I think for my problems, at my scale Million of user not Million concurrent user I think these are nicely solvable given the capabilities the tools today have (even if they are not doing it exactly like that) but is is unscalable at large or this is just not needed if you do 'something` which I/we are not doing?
Do they even follow any of their original advice or Google basically keep doing over engineered stuff fixed by adding another set of over engineered staff?
Let's talk gmail. I just refreshed the window and it did close to 400 request, ~8MB download which translates to nearly 40MB resource. And it keeps making more requests even when I'm not doing anything.
And a refresh of Google.com the search page did 33 request and nearly a MB download.
And they are preaching the world about optimizing the web?
I like the writeup though I need to do a second pass to fully review everything. The biggest thing that just suddenly hit me was 5 TB/day is really 60MB/s.
Browsers are a universal medium both for content and UI for app and that's great. But they are kind of the most wasteful creation ever. The most obscene apps of yesteryears would do a lot more with a lot less. An example of that is gmail taking 600MB memory (according to chrome) far more than a desktop client that didn't depend of the server doing all the heavy lifting would have taken. It would have all my mails locally, would actually be much faster changing pages (if done correctly).
But I digress. I think the complexity comes from (apart from over engineering syndrome) trying to deny that there's two very distinct use cases of web. One is nicely formatted semi static content and fairy dynamic stateful applications. I think designing a set of features (even modes in browsers that a page needs to declare for) can simplify things a lot.
The second thought I had was a brand new layout model/engine that does away with all the crafts and define some powerful primitive that works in a more well defined way that doesn't take a Google to properly implemented. Now I was thinking of two options of how to get that accepted.
1. A page can declare itself using the new primitives (now there can be two primitives for doc/app or this simpler one might handle both?) and the engine is small enough that it's worth having two in the browser. And the saving from some pages specially as it grows over time specially for well maintained/popular pages use the faster version the overall browser efficiency increases even if you can't remove the old engine.
2. Can there be a translation layer that can take a old layout page and rewrite the css/layout into the new primitive. It's perhaps slower than the current engine and perhaps on changes to content/viewport/scroll retranslate a lot but perhaps allow one to drop the old layout engine a lot faster.
Complete bonkers? I am almost certain it is but I don't know enough to tell and I have been meaning to ask to a crowd who can answer.
As I said don't know what the answer is. I definitely want someone using password manager be able to use them. Using a password manager is a conscious choice with setups involved. Browser's default autocomplete is often not. I don't know how to separate these two out.
Two points about a potential trial. 1) Since it's a runtime tool to actually see what it can detect I assume I will actually have to generate some attacks myself to actually see it in affect? It also makes false positive testing a little harder.
The reporting and such is on the cloud I presume? Are there some documentation on what happens at the agent level and what gets send to the cloud?
On a side note just read about sqreen. Anyone with a clue to how useful such a product is in reality. I find application security related products very hard to evaluate from the docs without heavy usage and trial.