Ask HN: Why do langs with alleged “automatic memory management manage tiny part | Hacker News Reader

Ask HN: Why do langs with alleged “automatic memory management manage tiny part

Java boasts it has automatic memory management. But you have to close files after using them. That's manual memory management!

View on HN

Closing files manually is manual resource management, not memory management. C++ uses its memory management for resource management, but Java doesn't. Java improved resource management with "try with resources", but generally resource management really isn't an issue for most Java developers. In modern frameworks like Spring you rarely have to manage resources yourself. You let the framework manage them for you, and the framework will provide them to you via injection.

Your way of saying it isn't wrong, but I think I would say instead that C++ uses destructors both for memory management and management of other resources.

If we're being pedantic here, destructors don't manage memory at all, they only free resources. Memory is freed by the caller of the destructor, either automatically if the memory was allocated on the stack, or manually by the delete operator if the memory was allocated on the heap.

Memory is a resource, too, and C++-style destructors do manage it, sometimes.

Destructors do not free the memory of the object being destroyed, but they can free memory that its fields point to.

I want a language where the garbage collector not only frees inaccessible dynamically allocated blocks, but also frees all resources when they're inaccessible. Is it possible to design such a generalized GC?

In c# you can implement IDisposable. The garbage collector will call a dispose method allowing you piggyback on the garbage collector for just about anything.

But it's rarely a good idea to piggyback. Most resources are much more limited or time sensitive than memory. So you should free them immediately.

Even when your resource -is- memory it sometimes is a very very very bad idea to free via garbage collection.

What is even the point of garbage collection? By the time you manage all the resources of a 100k line application you're going to have to do as much work as if you wrote it in C.

Well the point of gc is to allow the algorithm to focus on the problem domain. and not worry about mechanical implementation details of a computer. Everything has a trade off. An obvious win still must pay a cost elsewhere. In the case of GC it's simply not a good trade off for many resources. Sometimes it cannot be used even if you were willing to accept a trade.

Managing resources by hand is managing the mechanical implementation details of a computer. So in order to have garbage collection that manages more than just a sliver of memory so small that it counts as a rounding error by the same you get to the size of real world projects, you need a custom operating system and possibly even custom hardware too.

Your statement is false for many real world projects. (For some, you are correct. Those projects should not be written in a garbage-collected language, or should be written to run on a custom OS and custom hardware.)

When GC'd objects represent resources like open connections and files, use-after-free safely and reliably raises an exception to handle. Without GC, use-after-free on pretty much any object is so catastrophic that even correct code starts doing unpredictable things.

GC can do that, for a loose definition of when they’re inaccessible (it will do it when, but not as soon as)

Reference counting almost gives you as soon as, but it has trouble handling circular references. That is fixable, but not, AFAIK, in a way that’s both reasonably performance, frees resources as soon as they become unreachable, and isn’t too restricting.

What fix are you referring to? The ones I know of are algorithms for detecting when the object graph becomes disconnected. Unfortunately that turns out to be very hard in practice and you often have to scan a large portion of the heap. So it doesn't offer any major advantage over reference counting + mark & sweep gc. Which is kind of the worst of both worlds. :/ Ref counting overhead + potentially long gc pause times.

Ones that are either not reasonably performant, don’t free resources as soon as they become unreachable, or are too restricting.

The fix you refer to is one, but it falls under ‘not reasonably performant’.

One could also make it illegal to have cycles (easy way to do that: make it illegal to update any references). For many, many, that will fall under ‘too restricting’, though.

Alternatively, allow updates to references, but give every allocated object a ‘bit’ that indicates whether, from it, a reference can be reached that contains such an updated reference that points to an object created later (maintaining that property is doable, but may not be ‘reasonably performant’)

You can use that later: when the reference count of an object goes to zero, and the object has a reference to an object O with that bit set, check whether the graph reachable from O can be reached from elsewhere. Again, that’s doable, but likely not performant.

It somewhat depends on how common circular references are in real world code, though. I’m not aware of any papers researching that.

Java's GC does it. When a reference becomes unreachable, it will automatically free the resource. That's not the issue. The issue is that GCs are not predictable. All current implementations either have issues with circular references, or they run infrequently and may even need several runs to clean up complex structures. So it may take a while until an unused resource becomes available again, and you don't know when.

The "file" abstraction covers many different types of storage: spinning and solid state disks, in-memory structures, whatever your cloud actually uses, optical medial like DVD's, and even magnetic tape. Typically, these are managed by the underlying operating system's IO interface not Java itself.

Historically, operating systems use buffers for IO and closing the file includes flushing the buffers of in-flight data (in addition to things like deallocating pointers to files at the OS level). Without explicit instructions the operating system doesn't know when a file is no longer in use.

A programming language/environment could open and close files on every read and every write automatically. But it would entail overhead for operating system calls to allocate/deallocate nodes in the file table. For some types of programs this might be ok. For others it might be problematic, particularly those with moderate or strong dependence on IO.

So in order to have complete automatic resource management you need to build a custom OS and possibly even custom hardware.

How would your new OS/hardware know when the "use" of the resource is complete ? Maybe the dev wanted to read just 10 bytes from a 1 GB file, maybe 100 bytes... maybe its a telnet connection which is never supposed to be closed and just kept in a pool ?

For a very small benefit of "automatic closing of resources" this looks like a huge trade-off.

Disagree that it is a small benefit. Resource management becomes so unwieldly that there's little difference between a 100k line application in Java and a 100k line application in C with regards to memory/resource management. Are you saying that moving beyond C is a "small benefit"?

And it's easy to determine when the use of the resource is complete: when there are no active pointers to the resource (the circular reference problem I don't think can happen with external resources like files).

Some languages (Python, Ruby, probably dozens of others) have constructs along these lines

  with f = open("data.txt")
    print f.readline()

The file is automatically closed at the end of the block. Those languages also have explicit open/close/seek calls to let the developer implement more complex cases.

Btw, the os closes any open file when the program ends.

Yes, you're correct that os's have some kind of rudimentary reference counting for files, but it's very inchoate and not designed as the primary method of handling file references.

There's nothing wrong with Java, but there's been a lot of effort over the last two decades put into designing alternative languages for the JVM. Scala, Clojure, Groovy, and JRuby are examples (though none are primarily in the direction of automatic file handling).

I don't think that's necessary. The 'friendly' language could have a write(file, data) command implemented at a lower level in an 'unfriendly' language as:

  // try, catch, error handling, etc.
  // omitted for clarity
  friendly_write(file,data) {
      open(file)
      write(file, data)
      close(file)
  }

It would be an engineering tradeoff of code brevity for possibly lower IO performance. Sometimes it might be worth it. Other's it might not. Java's design reflects the engineering opinion in favor of finer grained control. This is consistent with it's heritage in the line of Algol inspired languages.

That's what the concept of "Finalizers" is for. They indicate that something should be done to an object just before the runtime garbage collects it.

https://en.wikipedia.org/wiki/Finalizer

Files specifically are a bit thorny because the OS generally places a limit on how many simultaneous open files a process can have, and you don't want to risk bumping in to that limit due to unpredictable garbage collection timing.

Reply on news.ycombinator.com