Hacker News Reader

Comment by andrewchambers | Hacker News Reader

This is 2015, 2MB files shouldn't be news worthy for a text editor. We seem to be bloating software faster than hardware improvements can offset it.

View on HN

Interestingly, Visual Studio disables all editor services (cold folding, intellisense, etc.) except syntax highlighting on XML files bigger than 10MB.

Code editors are surprisingly complex, it's not about simply putting lines on the screen. A good exercise in understanding the complexity is writing a simple syntax highlighting code viewer with native controls (not HTML) and see how much memory/CPU cycles your implementation uses for a 2MB file.

Tons of optimization needs to happen - for example, as far as I know very little information about lines that aren't visible is kept in memory in Visual Studio - so called "virtual lines." I'm sure other more mature code editors have this type of optimization as well. From a native perspective this type of optimization isn't a complete nightmare to implement (although it's hard if you consider code folding), however, imagine trying to do it in HTML. The browser is still going too do all the reflow etc. for the lines that aren't even visible, eating up CPU cycles and memory.

Even merely syntax highlighting is a problem. If you make one edit the entire file might be re-lexed: once again you need a unique solution for editors ("progressive lexing").

imagine trying to do it in HTML

The question on my mind is, why do it in HTML? Especially if you're going to wrap that browser in something that approximates a desktop app anyway.

Imagine if designers tried to do everything in PowerPoint for some unknown reason. "3D animation? Yeah, we can do it in PowerPoint with this incredible hack that makes slides sort of like faces on a mesh..."

That's how it is with HTML today.

"Imagine if designers tried to do everything in PowerPoint for some unknown reason."

A fair percentage of the employees where I work seem to have reached that point already - reports, training material, data for general discussion, GANTT charts with individual symbols that I guess must be manually moved every week. Gah!

Because of extensibility concerns and the sheer number of JavaScript developers? I'd agree if you say it's not a good compromise, but it is one nevertheless.

HTML is not the problem.

Light Table doesn't have any issues with large files, and is built on essentially the same stack (Electron + native HTML/JS/CSS for Atom vs Electron + ClojureScript compiled HTML/JS/CSS for Light Table).

Visual Studio Code doesn't have any issues with large files, and is also built on web technologies (no source available but I assume some variant of Electron + TypeScript?).

The problem is somewhere in Atom itself.

> Visual Studio Code

Virtual lines in action.

1. Open VS Code with a file that extends below the bottom of your screen.

2. Press F12 to open the dev tools.

3. Use the magnifying glass to inspect the final visible line of code (it should be the predecessor sibling of <div class="contentWidgets">).

4. Scroll the text editor up and down and watch as the lines are added and removed from the HTML.

Edit: Looked at Atom and, yep, all the lines are kept in the HTML all the time. They seem to be chunked into "tiles" but those tiles always exist in the document.

Edit2: Actually it seems as though there only ever exist 3 tiles (partially obscured by top, fully visible, partially obscured by bottom) containing a dynamic list of lines. So this isn't why Atom is slow, arguably this is a better approach than appending/removing single lines at a time (which is what VSCode does).

> Visual Studio Code

It is based on Visual Studio Online. Its text editor component is non-trivial and Microsoft spent a lot of money on it. The editor (codename "Monaco") is from Erich Gamma (ex-IBM/Rational), the "Design Patterns" book author from Switzerland. Microsoft bought the Monaco editor with him and its the single reason why TypeScript exists and why JavaScript ES6 looks like it looks ("class" syntax) and why he leads the Visual Studio team: http://en.wikipedia.org/wiki/Erich_Gamma

The Monaco editor (coded in TypeScript) will never be open source, its their competitive advantage. The same reason why HTML5's contentEditable API will never be fixed in IE/Chrome/Safari as a web office suite is their competitive advantage of Microsoft Office365 Web Word , Google Docs and Apple iCloud Pages. With a bug-free contentEditable API would render also Monaco editor and many other web based rich text editors obsolete: https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement...

I've actually done not 3D, but 2D animation in Powerpoint (as a kid, with access to an office suite, but not the internet and thereby more appropriate software). Each slide contains a frame of animation (drawn in Paint, of course), and slides advance automatically in milliseconds. You can attach an mp3 to play in the background for the soundtrack. Save it as a .pps and you have a cartoon that you can share with your friends.

That made me wonder whether the demoscene had ever done something with Powerpoint... and indeed it has:

http://www.pouet.net/prod.php?which=54670

http://www.pouet.net/prod.php?which=30319

Excel too:

http://www.pouet.net/prod.php?which=53021

Completely impractical and very silly, but that's what the demoscene is for...

This has nothing to do with bloat. While unimpressive not to be able to open files larger than 2MB, it's easy to see how and why such a limit came about, and as Atom seems to be styled as a text editor for programming, not really unreasonable, as very few human-created text documents get anywhere near 2MB.

Sorry, it's not obvious at all how the limit came about. That is pretty terrible.

Atom is a desktop/native wrapper around browser-like internals. The stuff inside is all just HTML/javascript/css running on node. There are some neat advantages, as well as just seeing how far a web interface for a desktop app can go.

The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. It was just a hard problem to solve, having an HTML-based editor deal with so much text at once.

> trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser.

I'd love to test that. Anyone willing to privide an example file? HTML with css perhaps, no js. Should be pretty small when zipped.

Assuming you don't need anything complex, it's pretty easy to generate it.

Here's a 100MB HTML file (compressed to 300KB): https://curl.io/get/yt7zbmfq/0eb1564f570236acf1d9f304855a7c7...

As a side note, VIM opens it in an instant /troll

The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser.

I remember opening HTML files in that size range (they were logs) in IE6 around a decade or more ago, on a machine with <1GB of RAM, and it handled them just fine - there was a noticeable delay, but it wasn't unusable. Even searching for strings worked.

It is obvious to me: I think they chose to load, parse, prettify, colour etc. and keep the entire file into an in-memory data structure, which is easy to reason about, and totally reasonable for "human sized" data. However, for arbitrary sized files, you need a mechanism for leaving most of the file in a raw state - that's harder (but not impossible, obviously).

So, it makes sense to me that they would assume only "human sized" data (and expect users to use another tool for looking at large files, "do one thing well" and all that) and opt for a slightly simpler programming model. Apparently, they decided that the assumption was wrong, and so changed the model.

very few human-created text documents get anywhere near 2MB.

True, but as a programmer i very often generate and need to view files greater than 2MB.

This.

Not to mention log files which commonly consume megabytes.

Nitpick inbound.

You shouldn't open log files with an editor unless your intent is to edit the file. There are a whole host of tools (grep, less, awk, sed, etc) useful for parsing and exploring log files.

You also very often do X, but that doesn't mean your programming text editor is the correct tool for doing X.

Insisting on all tools having an X because X is a useful thing is how you get the very bloat the GP is talking about.

On the other hand, virtually every other editor is the correct tool to do that task, yet this one is not. Are we supposed to accept that?

2MB is 0.025 percent of my ram, I shudder to think how much ram is actually being used by atom... At least the html looks pretty.

The biggest problem I see is that this is an artificial, arbitrary hard-coded limit:

https://github.com/atom/atom/commit/ec65a71d6dc8c3c44cb1b77a...

If I have 64GB of RAM, I expect my applications to be able to use all of it when asked to, and not just give up completely. It's fine to slow down, and giving a warning "Opening large files may be slow. Do you want to continue?" would be nice, but working with a 2097151-byte file and giving up if it's 2097152 bytes is really unpleasant.

Reply on news.ycombinator.com