Hacker News Reader

Comment by AriaMinaei | Hacker News Reader

Does everyone think this is good news?

I'm all for making the web faster/safer/better and all that. But I am worried about losing the web's "open by design" nature.

Much of what I've learned and am learning comes from me going to websites, opening the inspector and stepping through their code. It's educational. You learn things you may never read about in tutorials or books. And it's great because the author may have never intended for their code to be studied. But whether they like it or not, other people will learn from their code, and perhaps come up with [occasionally] better versions of it on their own.

This has helped the web development to evolve faster, and it's obvious how democratizing this "open-by-design" property is, and I think we should be concerned that it's being traded away for another (also essential) property.

Human beings cannot read asm.js code. And a bytecode format will be more or less the same. So, no matter how much faster and more flexible this format/standard is, it will still turn web apps into black boxes that no one can look into and learn from.

View on HN

> going to websites, opening the inspector and stepping through their code

Good news then, because even when loaded from the binary representation, browsers will apparently be able to show a canonical text format representation. (more info here: https://github.com/WebAssembly/design/blob/master/TextFormat...) Citation:

  > Will WebAssembly support View Source on the Web?
 
  Yes! WebAssembly defines a text format to be rendered when developers 
  view the source of a WebAssembly module in any developer tool.

WebAssembly isn't going to kill that notion; uglify already did. Nearly all codebases on big sites get run through a build process these days. The open web continues to live on through the open and free software movements.

I agree that the web is far from the full ideal of "open-by-design", but it's still the most significant platform providing it to some extent.

The problem of uglify can be mitigated if someone invented a binary format for JS that was interchangeable with the current text format. The format would reduce the code's size, while keeping it readable (you'd just have to run it through a binary-to-text decompressor).

I should also say that you can read uglified/obfuscated code. It just takes more patience. Example: if PS was written in JS and then obfuscated, how hard do you think it would be to find out how their color transformation algorithms are implemented?

You can't say the same thing for asm.js code though.

What's the difference between deobfuscating code, and decompiling it? Either way you end up with a weirdly-structured, canonicalized, symbolless mess.

The binary format that I mentioned/proposed would map one-to-one with a text format. They would be interchangeable. That means if you open a file in that binary format in a text editor (with the right plugin), you'd see almost exactly the original code. It's not decompilation. Only decompression.

I think you're confusing cost of parsing with cost of lexing. A 1:1 binary representation of JS (like HPACK is for HTTP headers) wouldn't decrease parsing time (the time it takes to turn a stream of recognized tokens into an AST) at all, which was the goal here.

Agreed, but the idea is still beneficial. If 1:1 binary representation for the data from the lexer doesn't yield much benefit, then a representation for the AST might.

And reading from that data could be much faster. And with a supposedly smaller file format, it probably wouldn't be so necessary for a minifier to rename symbols to single-letter ones to save a few more bytes.

[Edit: added the following paragraph]

This could yield a smaller file format (which is what people usually want from JS minifiers), without sacrificing the readability of the JS code.

Renaming symbols, and minification in general, is not really "necessary", it's just something people do to extract a few % more of performance. If they had that file format, they'd still have exactly the same reasons to rename symbols, so they'd still do it.

After all, if people cared about leaving their files readable, they'd just publish the non-minified version as well. Those who don't, won't avoid renaming either.

I agree that renaming symbols would still save a few more bytes, but I still think that with the right file format, symbol renaming and mangling would become an insignificant, unnecessary micro-optimization.

But that should only be tested in practice, so ... :)

This seems backwards. If I have JS source and the identifiers take up 10% of it, then symbol renaming and mangling can shrink my code size by something less than 10%.

If we shrink that other 90%, so that the formerly 10% is now 80%, renaming becomes more attractive.

This doesn't hold if people have a fixed problem with a fixed solution and fixed constraints... but problems and solutions grow.

> If we shrink that other 90%, so that the formerly 10% is now 80%, renaming becomes more attractive.

Convinced :)

I would say the same thing for asm.js. If you had enough time/patience you could still determine how some given logic worked in some sample asm.js code.

> if someone invented a binary format for JS that was interchangeable with the current text format

We already have gzip, although even with gzip minifying is useful, because gzip isn't content-aware. You can also enable source maps in production if you care about openness. The client doesn't have to fetch the maps/sources until developer tools is opened.

Well, gzip is general purpose. A binary format would be aware of JS' semantics and possibly yield much higher compression rates.

In fact, WASM's binary format is intended to be that way and apparently is yields more compression than using only gzip: https://github.com/WebAssembly/design/blob/master/BinaryEnco...

I'm apparently in the minority but I'm with the GP in that I've never been sure uglify and other minification was actually a great idea for this reason. The gains are marginal if you're using gzip compression (you are, right?); the loss of source readability is a big deal for the open nature.

Saying that the open web lives on through open/free software also seems dubious to me. Most SaaS don't have their offerings up on Github.

I wonder if we're about to discover yet again with the browser itself why the browser beat several cross-platform VMs.

As someone who has used uglify on occasion, sometimes you pull a trick and you don't want the competition to find out too easily. Make'm sweat for it.

And on the subject of Web Assembly, if asm.js is the inspiration, great thing will come for it, really a new era for the web. For example, things like that https://github.com/audiocogs/ogg.js, and to me that's just a tiny glimpse on what the possibilities will become.

> I've never been sure uglify and other minification was actually a great idea for this reason. The gains are marginal if you're using gzip

Smaller JS parses faster. gzip doesn't affect parse time.

WebAssembly will be more readable than asm.js, so it is an improvement over the current state of things,

https://github.com/WebAssembly/design/blob/master/FAQ.md#wil...

I do agree though that asm.js is less readable than handwritten JS. However, as others pointed out, when handwritten JS is minified, it also becomes quite unreadable.

Overall, I think WebAssembly will keep the web about as human-readable as it currently is.

Questions like this ignore the economics of minification. If you're a website and you have the option to ship a megabyte of unminified JavaScript or 150 KB of minified JavaScript, the minified JavaScript makes a ton of sense. Sure, gzip helps, but not all the way.

Same thing with WebAssembly: wishing WebAssembly didn't exist isn't going to make it go away. It exists because there is a real need for it: the need to securely and efficiently run high-performance C++ codebases in a web browser. Not everyone has this need, and not everyone will use it, but WebAssembly (and asm.js and Emscripten) solve a real problem for some people, and they simply cannot realistically target the web without this technology.

Well, I didn't really say I wish WebAssembly didn't exist (although the title of my comment somehow implied that).

I've written a more complete discussion here: https://news.ycombinator.com/item?id=9743859

The FAQ at https://github.com/WebAssembly/design/blob/master/FAQ.md has an item about that. The WebAssembly has a view source implementation that translates the bytecode into a textual representation that is easier to read than asm.js

Unfortunately, with minification, obfuscation, and transpilation, I think learning via view source has long been dead on the web. The loss of fidelity is simply too great.

That said, just as you can convert LLVM bitcode back and forth between binary and textual representation, you will be able to with WAsm as well.

Chrome's pretty print does wonders.

It's still not as good as raw source, but I have definitely analyzed obfuscated/compiled JS without incident.

Well, the bitcode is apparently an encoding of LLVM IR, which doesn't translate back to the original source code http://stackoverflow.com/questions/5180914/llvm-ir-back-to-h...

[edit: additional context: https://github.com/WebAssembly/design/blob/master/FAQ.md#wil...]

I think this argument is sort of a nostalgic one, especially with the advent of mobile apps. Mobile apps, their quality and responsiveness, are the standard on which software is judged. For the web to be a positive, competitive, open platform going forward, it's extremely important that user experiences (as in end user, not software developer js user) are the priority. As Brendan points out, parsing to turn something that's actually bytecode into bytecode becomes a hotspot, so this is a way of removing that hotspot. ASM.js has enabled webgl and canvas to have the ability to compete with native experiences... so progressing this out seems like the best way for the web to get better.

We should try to make the web faster. But it seems to me as though that in this process, there is no discussion about keeping the web readable. (Sourcemap's can be turned off.)

I don't think these two features, fast+readable, oppose each other. Not in the long run at least. I'm sure we can find a solution to keep the two together, as long as we consider both to be essential.

It could be addressed culturally. There's no need to mandate a single channel for distribution to humans (source) and machines (binary).

We could define a standard discovery mechanism for embedding a source URL, e.g. github repo + changeset, or PGP-signed javascript.

This would be even better than "View Source" (e.g. source provenance, change history and human reviews/comments), and the browser can still correlate runtime inspection with relevant source code.

A cultural solution, to me, sounds like the ultimate goal. But I observe that culture usually arises from the constraints that a society lives with.

I think many people would gladly opt-in to adopt the standard you proposed, but perhaps commercial projects won't have the incentives to do so.

Like the web itself, commercial companies will eventually come around. It can be the "new new" content marketing, since it's a small step from source-discovery to company-brand-discovery. Code becomes a proxy for author-reader connection, like prose.

We can culturally transform "view source" into a reputation metric that impacts visibility and revenue.

Look at the code snippets in the recent Bloomberg article, "What is Code", http://www.bloomberg.com/graphics/2015-paul-ford-what-is-cod.... Imagine a weekly NY Times list of Top 10 New Code, or Genius.com annotations, or CC-BY code comments which appear in search engine results, etc.

Society will increasingly care about coding, as writing code|prose influences thought, and more people will seek to understand the tools that make us as we make them.

Thanks for raising this important requirement.

Very interesting.

Are there any books/studies/examples of commercial companies doing something like this?

It's more of a vision than an observation :)

Web site easter eggs and job ads-in-code are a precursor, http://www.bbc.com/news/technology-25826678 & https://www.smartrecruiters.com/blog/the-5-most-creative-dev...

W3C has an Annotations Group working in this area, which includes commercial companies and educational institutions, http://www.w3.org/2014/04/annotation/report.html

What I find interesting about this discussion is the implicit realization that the freedom to inspect the source code is valuable. It has not always been the case that one could make such a statement and expect the majority of readers to instantly understand. What a long way we have come!

I suspect, though, that we still have not come far enough to be able to deal with this problem culturally in the mainstream. The technical solution you present would work exceptionally well along side licenses like the AGPL and would definitely be a boon to the free software community. However, are people as eager to provide freedom to others as they are to take advantage of it?

I choose to mention the AGPL on purpose because it goes a fair way beyond what you propose. It grants all 4 free software freedoms, not just the freedom to inspect code. I could well understand if someone felt that they were very keen on the one freedom to inspect code, but couldn't bring themselves to champion the other 3. The problem is that many, many other people feel exactly the same way about all of the 4 freedoms. Not only do they not see particular value, but they are actively opposed to it.

I think you would fight a losing battle trying to implement this in a standard. In the same way that I couldn't realistically expect to convince most people to use the AGPL in their web projects, despite the obvious benefits that I see, you probably could not convince enough of the powerful players to do this. As a programmer, once you live in a world where this freedom is ubiquitous, it is obvious and even jarring when the freedom is witheld from you. As a decision maker protecting corporate interests, it is merely a matter of tradeoffs and the value is not apparently clear. Certainly, it is not something that most people outside of the programmer community would want to encourage as the default course of action, even if they are willing to engage in it as an exceptional case.

Of course, I would love to be proven wrong ;-)

I view the situation slightly differently: we now have a few examples of business strategies which employ open-source as part of a freemium sales model or for strategic commoditization. Such businesses can identify the subsets of code which qualify for one or more of the software freedoms, i.e. they have already done the legal groundwork for source-controlled separation into different asset classes. A "marketing standard" for View Source would let them earn incremental returns on the already-cleared code.

Github would have a lot to gain from such a standard, as they are already in the code history and metadata business. It could be prototyped with a browser extension, prior to standardization. In time, businesses will care more about software supply chain transparency, for open and closed code. A "View Source" standard could optionally request authorization before the source is made visible, e.g. from a private Github repo. The value comes from the run-time coupling of source and behavior, and it can be derived from source anywhere on the licensing spectrum between proprietary and public domain.

> A "View Source" standard could optionally request authorization before the source is made visible, e.g. from a private Github repo.

Or they could make a protocol to select which version you want to see: the webm or the plain JS. No need to mix external websites like Github in the loop.

http://www.gnu.org/software/librejs

As others have already stated the web today can be quite hard to read. WebAssembly doesn't make that situtation worse. WebAssembly, however, is and will continue to be well defined and openly documented. I am confident tooling will come that will help humans understand this format.

It might be time for a WebDwarf debug format, though, to help us. Source maps aren't quite robust enough.

The web is hard to read because people have an incentive to use alternative languages/tooling to produce web content.

These alternatives wouldn't exist (or be so widely used) if the web provided the benefits that these tools give.

For example, look at how many people switched from CoffeeScript to ES6/7 once these languages started providing most of (and not all of) the benefits that CoffeeScript provides.

In all honesty, I don't think sourcemaps are the answer to the black-box problem.

I think you have a point, and (elsewhere here, among other places: https://brendaneich.com/2007/03/the-open-web-and-its-adversa...) I've argued that the Web's open-box nature matters. It's not just an accident or hindrance.

If I'm right, we won't see wasm overused such that the web becomes a sea of monolithic and opaque apps (as too many mobile OS homescreens are). Per the FAQ, wasm is also for hot kernel code and reusable libraries helping mostly-JS apps.

It's up to web devs to keep innovating ahead of browsers, to keep pushing for better HTML, CSS, JS, and beyond. One way to do this might be via wasm modules that extend the web. The https://extensiblewebmanifesto.org/ project continues.

I think the only way the web could remain mostly-JS apps, is if the benefits of writing mostly-JS apps would outweigh the benefits of writing, well, not-mostly-JS apps.

And I'm not sure if that's gonna be true.

For example, I'm sitting here, arguing for an open and readable web, and yet I can't wait for a rust-to-asm.js workflow to get stable enough so that I can move the perf-critical parts of my app to rust (you know, the part that could be learned from and hacked on by another developer).

Serious question: why could not those other devs learn from your Rust code, even decompile wasm back into it via their devtools and ye olde view source?

Optimizing compilers.

Sometimes you must build with -g. WebDWARF ;-).

Can you please clarify? I've seen WebDWARF mentioned twice now, but when I Google, I get a bunch of results about a shareware clone of Dreamweaver.

DWARF is a debugging data format for ELF files.[1]

WebDWARF then would be to WebASM, as DWARF is to ELF files.

[1]https://en.wikipedia.org/wiki/DWARF

Thank you!

I'm suggesting that when debugging optimized code, you'll use -g as on native platforms. I'm speculating further that with wasm, decompilers will rise to the view source challenge.

Because the source-map may not be available. And they might not know Rust. Or perhaps having to setup a Rust dev env to play with this code is too much work and it's just not worth it.

Anyway, I think I should explain my thoughts better instead of only pointing out a problem. I wrote this comment: https://news.ycombinator.com/item?id=9743859

I think it is perspective. I would prefer a web where I can use any language I want and not be forced to use only Javascript to create content. So in that sense this could give me more freedom.

For example I may chose to write my code in Rust and that imposes a burden on you (assuming you know only Javascript), but it does not curtail your freedom. Not as convenient as before for you since you now need to learn Rust. I think the freedom has spread to the person who is creating the content (code). The general consumer does not care, but the tinkerer/hacker now needs to put in more effort to parse the creation.

So long as the wasm blob can be read as Rust by a third party.

If it's a one-way Rust source -> wasm transformation, even a Rust user might have a hard time following what wasm spits out.

I agree that giving authors more freedom is a good thing, but as I argued, it also has serious downsides. But I think there are ways to keep the best of both worlds. I wrote a comment here with more detail: https://news.ycombinator.com/item?id=9743859

Not really. I happen to think that a browser is supposed to enable the browsing of HTML documents but every year the browser becomes more of the one takes all platform. Which means my hopes of seeing JavaScript go out of existence keep waning.

I'm glad I'm not the only one thinking this.

WebAssembly is in itself probably a good thing, but yeah, as is evident from the replies you're getting, for lots of people readable sources simply isn't a priority or even a goal.

Which is probably why we have the GPL. But unfortunately, it's not seeing much use on the web.

This is at least addressed in the comments, which point to this link: https://github.com/WebAssembly/design/blob/master/FAQ.md#wil...

AFAIK, WebAssembly code will come from places like LLVM IR. The compiler pipeline can simply choose to omit references to the original source code. Without the source, the WA code (whether in binary or text) will be just IR data. I haven't read IR code, but I'm guessing it's far past the AST stage, with many aggressive optimizations applied to it. It won't look anything like the original source.

"Open by design" doesn't have to mean plain tex. I learned large swaths of SSLv3 and TLS just using Ethereal and later WireShark to watch handshakes. WireShark handles HTTP/2. Byte code decompiles easily. I understand you concern, and I love view-source too. But I don't think this loses that.

Like it or not, the web needs a way to protect business logic, no matter how imperfect it may be. Besides, uglification has already rendered most deployed sites unreadable anyway - no serious business that I know of will deploy non-minified code.

Keep your important business logic server side if it needs be protected. No serious web business lets their propriety magic happen client side, obfuscated or not.

When I learned into computers there wasn't any open source and yet I was able to learn a lot.

In the demoscene it was very rewarding to be able to match other demos without having any help how the others have done it.

Now you just look at the code and be done with it.

did google closure compiler or other js obfuscation tools ruin education or open design?

i dont think view source has been instrumental to the web or education. yes there have been good moments and uses of it, we all have our anecdotes, but it's not a killer feature.

i would happily remove the ability to view source in favor of a faster, richer web site.

https://news.ycombinator.com/item?id=9735042

I think this feature is instrumental, or rather should be instrumental.

We are talking about the platform that will soon define the main medium of human expression. It's no less significant then everyday language. It should be open, by design.

i dont understand this. what does open mean? how open does "open" need to be, to be truly open?

there's 7 layers on the osi stack. there is no internet without them. do you need to know how all of them are designed to be "open"? even if you go on wikipedia and read the designs, you'll get an abstract view of them. but does that tell you enough? the implementations are all different.

focusing just on layer 7, the page you are viewing, do you need to see the actual html blocks, css, javascript to be "open"? of a sizable website, this front facing page is only represents a fraction of all the tools to produce the site. do you need to know how those hidden parts work too to be "open"?

i think the big distinction for me is HTTP vs Internet. if i click on a page and it includes a remote javascript file, i want to know that my computer is connecting to another server to get that file, but do i really have to be able to read it? my computer already connects to others over the internet in non-HTTP settings where I cant read what i'm getting.

The question of "How open?" should be answered in a design process where "As open as possible" is kept as a goal, along other goals such as "As fast as possible" and "As safe as possible", etc. They should be all considered as factors in the trade-off decisions.

Well, some of web apps are already blackboxes, due to use of languages which are compiled to javascript (JS output usually is completely unreadable) and tools like Google Closure Compiler. WebAssembly won't make much of a difference.

There is an interesting discussion going on here, and I'd like to share a few more thoughts for clarification.

There is a TL/DR at the bottom.

---

My main point is that we should see openness as a primary goal in the design of the web platform, alongside other primary goals such as performance and safety.

What do I mean by openness?

A web app is open, when I, a developer interested in how that app is implemented, can look at its code, and say "hey, this code handles that video, these functions manage those audio nodes that make the sounds I'm hearing, and this code decides what happens when someone clicks on that button," as opposed to "these are some ones and zeroes and I don't know what they do." [1]

What's the benefit of this openness?

I think the benefits are obvious, but for starters, when web apps are open, everyone can learn from everyone else's work. And they can build on each other's work. Web content evolves faster. And all of this is there by design. Authors don't have to opt into it. It's there and it's ubiquitous and it's just how things are. [2]

How does WASM endanger this openness?

WASM doesn't necessarily endanger anything. It's good. It allows devs to make faster apps, experiment with different semantics, and discover better ways to define the higher-level features of the web (as per the extensible web manifesto).

But it could also be bad. Here is a hypothetical example:

Imagine someone writes a graphics library for their app. It doesn't have the bloat of DOM. It's fast. And it allows him to make really cool apps.

But it's in WASM. It's not really a web app. Sure it has a URL and can't hijack your computer. But it's not HTML; others can't read it. It's not hackable. It's not mixable. It's not even indexable. Adblock can't tell its content from its ads. It's just a portable black box with a URL.

And devs have many incentives to write apps this way. Not all apps need to be indexed. And everyone could use the better performance.

So imagine a future where most web apps are made this way, each using a different rendering engine, written in a different language. I think it's clear why that could be bad. (Tell me if it's not.)

So, what do I propose?

I don't propose that we ditch ASM or WASM. These are steps in the right direction.

But we need to recognize that while safety/perf/better semantics are natural needs that drive the design of WASM, openness is not a natural need and it won't naturally affect WASM's design. Let me explain:

People need web apps to be safe; if web apps aren't safe, people won't open their browsers. People need web apps to be fast; if not, they'll have a reason to prefer native apps.

So, we have strong natural incentives to make the web safer/faster/etc. But we don't have any strong natural incentive to make it "open." So "openness," naturally, won't get enough attention.

But if we think that this openness should be a feature of the web, then we should treat it as a first-class feature, among the other features like perf and safety. Meaning that when making a decision about adding a functionality to the web platform, we should not only consider how it affects the web's safety and performance, but also what it does to the web's openness.

And we should communicate that to the community, especially when news like this come out. So that readers just don't assume that the web is only gonna get faster, but that it's openness is still gonna be a goal. This will also help create momentum and community feedback for this goal.

What does having openness as a goal mean in practice?

It probably means that committees would constantly look at what devs do in WASM land, and create standardized, high-level versions of them to be used by all developers [3], and also couple these features with great tooling and other benefits that are hard to replicate for WASM. [4]

This makes sure that developers have all the incentives to try to remain within high-level APIs and standard structures as much as possible (thus, keeping web apps readable/interoperable/etc), and only venture into WASM land when they absolutely need to.

I should conclude by saying that I realize this is probably what WASM's authors intend to do. But imho, it is just not being communicated very well. Many people don't notice that openness is a goal, some don't see its benefits. That's why we should communicate openness as a primary goal and write it in our documents, in huge black letters :)

>> TL/DR: Openness is important. There isn't much community momentum behind it (unlike perf and better semantics). There aren't natural incentives to focus on openness, so it might become an afterthought. Web apps might turn into high-performing black boxes. The black box part is bad. To prevent that, we should take openness as a primary goal, and communicate that with the community.

---

[1] WASM is more high-level than ones and zeros, but you get the point.

[2] Today's web apps aren't really open, per my duck-typed definition of "open." They're only open to some extent. It takes a significant amount of time and patience to understand how a certain feature is implemented in a typical web app. And I believe there is an opportunity to make that much easier.

[3] This is to some extent covered in the extensible web manifesto.

[4] It doesn't mean we should deliberately limit WASM's capabilities though. Let me know if I should explain better.

ps. I use compile-to-js languages exclusively. This discussion is definitely not a matter of taste or resistance to having to learn something new :)

Reply on news.ycombinator.com