Hacker News Reader

Comment by overcast | Hacker News Reader

Can we aggressively throttle memory usage? Why does it require 300-500MB of ram PER TAB?

It probably doesn't. Measuring memory usage is hard on a modern OS.

I'd guess there's a lot of shared memory between tabs

    Measuring memory usage is hard on a modern OS.

Not in the slightest. OS's have to allocate virtual memory to each process or that process with SIG-FAULT when it touches invalid memory.

Tracking per-process memory usage is one of the most trivial things OS/Pref tools do.

Measuring it is easy. Defining it is what's hard. If three processes each have the same 10MB executable mapped into memory, plus 10MB of heap data, how much memory is each process using? There's no easy answer to that question.

    Defining it is what's hard.
    how much memory is each process using?

20MB.

    Oh but they are sharing the pages the executable is mapped into.
    So how can you can you say they are each using that much RAM?

Because that 10MB of RAM won't be freed until the last process exits. Each process is using 20MB of physical pages. If only 1 was running it would use 20MB.

The semantics that each process is sharing 5 HUGEPAGES of heap isn't really important as even if 1 was running alone it uses that much memory.

If you are sharing that much data with executables that you have this problem there is some low hanging fruit for memory optimizations you can do.

     Oh but you aren't *actually* using 60MB of Heap, only 40MB

Indeed. And in cases where these 3 children have the same parent you look at the ParentMem + 40MB (child binary+individual heaps) to report a better picture.

Shared/unshared semantics are not hard. Have you ever read /proc/$PID/maps ? Because it clearly labels what is/isn't shared physically. When you are optimizing for memory usage you rarely look at shared readonly pages as the kernel put them there for you.

   Defining it is what's hard.

Injecting an interpretation before taking a measurement is not sound science or engineering.

Measurements inform Interpretation. Not vice versa.

Sure, shared/unshared semantics are not hard. How do you map that to a generic concept of "usage"? There's no right way to do it.

It doesn't really make sense to ask "how much memory is this process using?" Instead, you want to ask questions like "how much memory are all of these processes using?" or "how much memory would be freed if I terminate these processes?" or "how much extra memory will be used if I start another process?"

What's hard is understanding which question you want to ask in any given situation.

     There's no right way to do it.
     What's hard is understanding which question you want to 
     ask in any given situation.

I don't see the issue. You admit that answering all of these questions is easy. The hard part is lawyering which question to ask.

This is not a data reporting problem.

This is a data interpretation problem.

A good reporting tool should not be interpreting (or forcing an opinion) on the end user.

You believe this problem is hard because you are forcing an interpretation to exist before you take a measurement. When you should not be interpreting until after measurement. Injecting an interpretation into a measurement process fundamentally changes that measurement.

Measurements inform your interpretation. Not the opposite.

TL;DR You put the cart in front of the horse.

"This is a data interpretation problem."

That's what I said. "Defining it is what's hard."

You seem to understand what I'm saying, and agree with me, but insist on arguing for some reason I cannot comprehend.

     You seem to understand what I'm saying
     and agree with me

Literally the opposite. Your initial statement was

      Measuring memory usage is hard on a modern OS.

Which you've agreed was false as /proc/$PID/maps is easy to read. Then said:

      Defining it is what's hard

But these statements are not mutually inclusive nor does the later imply the former.

I stated measuring is literally the simplest thing. You said ah but what you measuring. That isn't dependent on the tools, or OS. So why did you state it was?

Nooooo... somebody else made that initial statement. My initial statement was "Measuring it is easy. Defining it is what's hard."

Are you just arguing with me because you didn't realize I was somebody else?

Yeah, it depends on what exactly you're asking for. But one can provide different visualizations for different questions.

As a typical computer user, this is what I care about:

- who the frak is eating all my RAM?

- which processes should I kill to get some memory back?

Whatever the weird sharing things are going on, the memory is ultimately linear. So show me a large rectangle subdivided into sections, each one representing memory associated with a set of processes. One section for "this memory is used by Chrome.exe". One for "this memory is used by Chrome.exe, X.exe and Y.exe". One for "this memory is free".

When I click on a section, it should highlight all the processes that share this memory. When I click on a process, it should highlight all the sections this process uses or shares with others.

This way, I could quickly see who's wasting my RAM and how many processes do I need to kill to get some chunk of memory back.

> Measuring memory usage is hard ...

Exactly, I usually find that chrome is eating a little more than what it's displayed in task manager in windows ;) I have no idea how it can achieve that to be honest. Restarting it seems to bring back the 5-15% missing ram in my case.

If it doesn't, then why does it grind to a halt eventually on both Mac and Windows? Firefox suffers from the same nonsense as well. In fact, Safari seems to be the ONLY thing I can keep open for more than a day without a process restart.

You're doing something wrong. I use Chrome on Windows as my main now and I usually have more than 100 tabs open. I just switched a few months ago from Firefox on Linux, also usually with 100 tabs or more open. Both are very stable. Chrome is worse with handling tons of tabs, but I certainly don't have daily crashes.

People who have extremely unstable/inconsistent experiences with these platforms usually have a rogue extension or some other problem lurking beneath the surface.

Also consider occasionally VACUUMing your browser's DBs, and on Firefox, taking advantage of the profile reset option.

For every person that has my symptoms, there is someone to counter it. I'm not going to get into an argument about how I'm using my browser incorrectly. Every computer / browser combo I've ever used exhibits the same thing, and I'm pretty religious with keeping things trim and clean.

Using a lot of extensions?

Could be the sites themselves.

Why is it that whenever somebody has a problem that less than 100% of people experience, the response is always "you must be doing it wrong" rather than "I must be lucky"?

Because the quantity isn't just less than 100%, it's a small minority. Chrome and Firefox are among the most widely-used pieces of software in the world. I'm not saying they don't have weird bugs, but they don't crash daily for most users.

While the OP may not be doing anything "wrong", he is almost definitely doing something unusual, or has deeper issues like malware or malfunctioning hardware.

If the ratio were inverted and most users were experiencing daily crashes from normal use, I would consider myself lucky. That's just not been my experience with Chrome/Firefox.

Bugs which only affect a small minority of users don't exist, nor do unusual but perfectly legitimate usage patterns?

You probably want to re-read my post, as I said neither of these things.

You're right. I was confused because in your other comment you just said they were doing something wrong, and I didn't realize you'd reversed course on that.

I would suspect a majority of the world does not keep dozens of tabs open, on an computer that never shuts down. You have to remember which forum you're on here. Any software issue, usually rears its head relatively quickly from one of our machines around here.

That's the use case I'm most familiar with. I've done this on many different machines for many years and not had anything like a crash once a day. I stand by my belief that your negative experiences are likely caused by a rare collision of variables and not representative of the typical experience among similar use cases. As I said in my post, it's totally legitimate that such an atypical experience may be caused by real bugs, but it's still an unfair generalization to say that Chrome and Firefox crash daily.

Does Chrome only use RAM if it thinks it is available? (i.e. if it starts detecting that other process are running and eating up RAM, will it use less?)

Nope - it's fairly happy to choke everything else or even crash (tabs or the browser) when RAM gets too scarce. It does seem to have some capability to restrict usage, but it's not thorough enough to avoid performance/crashing issues.

Not in my experience.

With 4 GB ram, Chrome will happily give ~2GB to any tab that requests it in a couple of seconds, leaving me with an unresponsive Windows aggressively paging memory to disk, only to give whatever it finds... back to chrome

It does try to reduce what it can in low memory situations (less "pre" optimizations like prefetching, prerendering, even dns prefetches and stuff, V8 gets more eager to throw out "potentially unneeded" code, tabs get suspended and written to disk, heuristics can get shut off, etc...), but it can't change what the page does.

If a browser tab wants to store 300MB of data in RAM, or wants to throw a shitton of stuff at the DOM, Chrome needs to oblige.

Firefox on Linux used to many years ago, it even included swap space as available memory.

This is design issue, inefficient data structures, lazy programming etc.

Im affraid tabs taking 2MB(two megabytes for a page with javascript, not 20, not 200, two) like in the good old Opera 12.16 wont come back :(

I agree, the Presto engine was (and still is) super efficient.

We may see something similar come out of the Vivaldi project which is headed by folk who led the Opera <= v12 development who know very well what was lost when Opera moved to Chromium based web engine.

No. Vivaldi like new Opera is just a skinning project. They made UI slightly better, added some missing functionality (mostry tabs and gestures), but they do not touch core engine.

Case in point fresh Vivaldi on startup makes 4 connections to Google just to say hi, and will keep sending google delicious pieces of data throughout the day, not to mention hardcoded google DNS in case yours is too slow. How about spamming my local network with chromecast UDP garbage? or enumerating shares on startup? WTF would you even do that?

Ps I use Vivaldi :/.

I suspect that you know of this already, but here it is anyway: https://otter-browser.org/

Personally I still have Opera 12.18 installed on every computer as it still is the most swiss-army-knife browser out there for countless reasons. Most of the webpages can be made to work properly in it by adjusting the site preferences.

One thing I am yet to see any of the new browsers be capable of is complete popup blocking. Opera 12.x excelled at this and a complete blackout of popups is 100% achieveable.

Reply on news.ycombinator.com