Raw String Literals Removed From Java 12 as Feature Set Frozen | Hacker News Reader

Raw String Literals Removed From Java 12 as Feature Set Frozen

infoq.com

87 pointsjavinpaul56 comments

Java Unicode escapes are kind of a mess, and the raw string literals proposal only makes it worse.

Unicode escapes are processed in Java not just inside string literals, but everywhere in the source code. So, for example, the following program prints "Hello, world!" even though that line of code seems to be commented (\u000a is new line, so it ends the comment):

    public class Test {
      public static void main(String[] args) {
        // \u000a System.out.println("Hello, world!");
      }
    }

Moreover, a \u000a inside a string literal is the same as an actual newline, so the compiler doesn't accept it:

      Test2.java:3: error: unclosed string literal
        String s = "\u000a";
                          ^

But now with the raw string literal proposal, JEP 326[1], Unicode escape processing is disabled inside raw string literals, and \u0060 escapes (backticks) aren't considered backticks for the purposes of starting raw string literals.

So, with this proposal, Unicode escapes are in a worst-of-two-worlds middle way:

1) They can't be handled uniformly at a low level anymore, so a Java parser can't naively convert escapes while reading the source file, but

2) They must still be naively interpreted in unexpected places like comments and normal string literals, as shown in my examples.

What a mess.

[1] https://openjdk.java.net/jeps/326

Dear God

Why did they do it like this

Just why

Why did they do what?

probably this very big WTF:

    // \u000a no comment

What is the reason for Unicode escapes? It is something legacy from when Unicode was not universally accepted in all systems? Do other programming languages have this?

Several programming languages have arcane ways to write what today seem like very normal characters.

https://en.wikipedia.org/wiki/Digraphs_and_trigraphs

For example C has escape sequences for characters as basic as # and [.

Doesn't it have escape sequences for those because they're used in the language's syntax? Stands to reason, or is this even in string literals?

They are alternative ways of writing the same character, unlike escape sequences, which have a different meaning from the unescaped version of the same character.

For example, in C, I can write:

    int a<:10:>;

Or I can write:

    int a[10];

They are the same, because <: is exactly the same as [, and :> is exactly the same as ] (except… mercifully, digraphs are not interpreted in strings; trigraphs are another story). However, in a string, the escape sequence \" means something different from ". One is a character in the string, the other ends the string.

Digraphs and trigraphs exist because not all keyboards have the corresponding characters, like [ and ]. For example, look at a Swedish keyboard. (You can type [ and ], but not as easily, and maybe not at all on some older keyboards.)

Wow, that is similar to the C++ trigraphs, where using ?? followed by various characters inside a string literal (and any other place) does not work as expected

But those in C++ are deprecated, yay!

not just inside string literals

This is the most unusual part and just leaves me asking why!?!? Was it an error of omission, or a deliberate design decision, to essentially preprocess the whole file blindly with Unicode escape parsing instead of putting it only inside the string literal (and character constant) parser like just about every other language that has a similar escape system? The fact that \n behaves differently (and in the sane manner that those are accustomed to from C/C++) is the most bizarre part --- Unicode escaped could've been handled by the same parser, but they didn't.

Sibling comments mention trigraphs. Those are different, starting with ?? instead of \, so their "applicable to the whole file" behaviour is more reasonable. But \-escapes are, outside of preprocessor line continuations (where it's literally \ followed by a newline), customarily only interpreted inside character and string constants.

It was a deliberate decision. The Java language specification [1] says:

> The Java programming language specifies a standard way of transforming a program written in Unicode into ASCII that changes a program into a form that can be processed by ASCII-based tools. The transformation involves converting any Unicode escapes in the source text of the program to ASCII by adding an extra u - for example, \uxxxx becomes \uuxxxx - while simultaneously converting non-ASCII characters in the source text to Unicode escapes containing a single u each.

The Java language grammar is defined in terms of Unicode, but it was designed in the early days before Unicode was ubiquitous. In particular, this was long before UTF-8 took its current place as the de facto standard character encoding.

The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well. But they also wanted to preserve interoperability between systems using different character encodings.

It's certainly confusing that \u behaves differently from other backslash escape sequences, but I guess they figured that was less bad than introducing an entirely new escape character.

[1]: https://docs.oracle.com/javase/specs/jls/se11/html/jls-3.htm...

The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well.

In my experience (although not with Java; maybe its users would make more use of the feature, but I doubt it) even developers whose native language is not English but something very different like Chinese or Japanese will continue to use ASCII-only identifiers, and only write things like string constants or comments in their native language (which may mean using high bytes and a non-UTF8 multibyte encoding.) What the identifiers mean may be Romanised foreign words, but they're still written with [A-Za-z0-9_].

Sure, but this was in the 1990s, when "internationalization" was The Next Big Thing.

At the time, the Java team really didn't have any way of knowing for sure whether or not non-ASCII source code would catch on, because most existing languages didn't support it.

I have seen quite a lot of source in Japanese where the language allows it easily, much of it was written by a Japanese partner and their customers so it made sense for them.

> The designers of Java wanted to support arbitrary Unicode characters in source code -- not just in string literals, but in identifiers as well

That's bad! I recently noticed "my" C++ compiler allows you to write something like int mäin(){}. I didn't try it out but I guess it breaks with different file encodings.

Please stay with ASCII!

Due to how they were written, a lot of compilers don't really care about character encoding in the sense that any bytes >127 were simply considered identifier characters because their tokenisers picked out the special characters and "anything else" would be classified with the usual identifier range. This meant you could use whatever single or multibyte encodings you wanted, as long as any trailing bytes of a single character didn't conflict with the special characters; with this, UTF-8 naturally works too.

JEP325 seems like a nice little improvement. Just a linguistic trick but still lovely.

   int numLetters = switch (day) {
       case MONDAY, FRIDAY, SUNDAY -> 6;
       case TUESDAY                -> 7;
       case THURSDAY, SATURDAY     -> 8;
       case WEDNESDAY              -> 9;
   };

Switches can return values, and can have multiple cases on the same line.

I'm sure in 2 years when my organization gets around to switching to Java 12, I'll enjoy using it.

Java 12 is not a long term support release and will only be supported until September 2019 (so don't switch to it in 2 years :).

Interesting to see the faster pace releases to get feedback on features, but many major libraries are just getting Java 11 support.

This is mostly due to Java 9 breaking everything.

Post Java 11, I doubt you will see the same problems you did with going from 8 to 9 or 11

What happens if the value is not found? I would love it if you could throw an exception from a default block.

You need a default clause when your cases aren't exhaustive otherwise. The compiler will insert a default clause transparently if you switch over an enum: https://openjdk.java.net/jeps/325

The cases of a switch expression must be exhaustive; for any possible value there must be a matching switch label. In practice this normally means simply that a default clause is required; however, in the case of an enum switch expression that covers all known cases (and eventually, switch expressions over sealed types), a default clause can be inserted by the compiler that indicates that the enum definition has changed between compile-time and runtime. (This is what developers do by hand today, but having the compiler insert it is both less intrusive and likely to have a more descriptive error message than the ones written by hand.)

This is pretty much the behavior you'd expect if you're familiar with languages that support pattern-matching.

Introduced for the first time in Algo 68 as far as I know.

It’s only a preview feature in 12, so you need to give a command line switch to java for it to be enabled.

It’s also the first part of a much larger feature adding pattern matching to the Java language. Getting all the ducks in a row for this sort of thing is tricky because it touches so many other ares, but switch expressions can be delivered on their own quite nicely.

Adding patterns means looking at variable scopes (where is a variable bound in a pattern in scope?), record classes (because destructuring patterns should not be implemented by hand everywhere), and even constants and raw string literals (because being able to switch on regexps would be nice, and you need a good way to write that).

That sounds lovely. Fully functional pattern matching in Java would be a godsend.

I'm glad they're going to try to get multi-line strings right, but seriously, I (and many other people) have wanted this feature for 19 years.

This is not an exaggeration. I was writing code in Java to access SQL databases in 1999 and needed it to express long SQL strings.

One would think they could move a little faster.

I don't think I've ever wanted SQL embedded in my Java source over loading it as a resource. Loading it as a resource allows me to edit the SQL in a sql-aware text editor.

IntelliJ is aware of SQL in text strings, and even validates it against the database schema.

I like my SQL right next to the code that sets parameters, processes results, etc - mismatches are apparent (and also validated by IntelliJ).

Not just SQL, but any language. It's one of my favourite (and IMO very under-appreciated) features of the Jetbrains editors.

Intellij also automatically escapes your string and inputs newlines wherever necessary, so if you copy a long string, it's generally done for you (I've copied pieces of XML and it generally worked totally fine).

Since Java 12 isn’t an LTS, isn’t the effective delay for slipping a release just 6 months? Many shops are going to use 8, 11 and 17 (assuming it’s the next LTS), so it won’t even matter to them.

If that’s right, it’s exactly how the new release cadence is supposed to work—there’s no longer such a hard decision between delaying the release, leaving a feature to languish for several years, or releasing a feature without time to validate it.

Can someone explain to me why java has never considered as "import com.package.foo.bar as bar" feature? I really think fully qualified names can be disgusting when you have two classes of the same name.

What do you mean? You can do both:

- import com.package.foo.Bar -> use Bar.methodOfBar() in code

- import static com.package.foo.Bar -> use methodOfBar() in code

What does "import com.package.foo.bar as bar" add? Other than some kind of aliasing, as in you import Bar, but alias it as Baz, so then you use Baz.methodOfBar() ..?

I think he's saying:

    import java.util as jvutil;
    import com.betterlist as better;

    class Example {
      jvutil.List list = new better.List();
    }

I assume it is because Java developers rely very heavily on their IDEs. Most of these IDEs simply hide the import statements by default. So you assume the imports are irrelevant, unless explicitly spelled out - and clashes are rare enough to not be much of a problem in practice. If imports are renamed, you would have to actually care about the import statements.

> If imports are renamed, you would have to actually care about the import statements.

Not really. VS Code transparently handles aliasing of imports in JS. You still never need to look at them manually AND you get the benefit of cleaner naming.

I have difficulty seeing what this would add. As a Java developer, I know what "List" is, just like I know what "ArrayList" is. If someone instantiated new better.ArrayList(), I immediately have to check what the hell is the import (go to the top of the class, uncollapse the import statements, and see what is "better").

As with all code readability, I assume to write and use import aliases, that's very easy. The person who's gonna read it after you, they've got a problem. And that's why I'd personally be strongly against this in my code review.

> As a Java developer, I know what "List" is

Is that java.awt.List or java.util.List?

Using the example from your other reply:

import java.awt.List as awtList

import java.util.List as utilList

Granted, there's not a lot of scenarios where you have this sort of collision, but it does happen and using aliasing makes the code easier to read.

That's a real problem that I've encountered. There were three web services which I consumed. They used a lot of classes with the same name and I had to work with them from one class. I had to use fully qualified names everywhere. At one point I rewrote this code using Kotlin. Kotlin had this feature, you could write "import com.package.foo.bar" and you could use "bar.Something" in your code. My code was sane. But in some release they decided to remove that feature for some reason.

You can still write

  import com.package.foo.bar.Something as barSomething

according to the documentation: https://kotlinlang.org/docs/reference/packages.html

Although you'd have to do it for each class, so importing the whole package with an alias would be nice.

Can anyone recommend a good resource to get up to speed with Java's new features since 1.8?

I find the JEPs listed on the individual project pages quite helpful

- https://openjdk.java.net/projects/jdk9/

- https://openjdk.java.net/projects/jdk/10/

- https://openjdk.java.net/projects/jdk/11/

- https://openjdk.java.net/projects/jdk/12/

Adding new syntax to Java is tricky (unless certain other JVM languages starting with C ;) but I was really disappointed in the string proposal.

At the very least, have a default, optional string substitution feature. It ain't that hard and it's something users of raw strings will NEED anyways. What good is it to have snippets of raw strings that then need to be parsed again and split and re-glued again?

JavaScript backticks are much more useful out of the box. But that isn't the Java way... In Java, everyone gets to write their own string substitution library, littering the heap with completely unnecessary substrings.

After the disasters with generics and enums, do it right, for once, please?

> unless certain other JVM languages starting with C

Are you referring to Clojure or Ceylon? Or both, or neither?

Kotlin?

I believe they're almost certainly referring to Clojure macros

String substitution and raw string literals aren’t really that closely connected. I’d also suggest that if string substitution becomes nice it will get used far more, so it needs to be implemented in a way that constant folds things nicely. So if I had to guess it might get looked at after the work being done on constants.

If Java were to support to scale/fp-like match statements with case objects and unapply methods, it would probably be good enough to obviate Scala (It would still be distinctly worse though).

Pattern matching is such a powerful feature. I wish more languages (like python) would realize it’s value and add it.

Is it?

I've used Scala before.

There are some cases, like compilers where I could see matching and extracting complex and nested patterns to be useful.

But in most cases monadic operations/functors plus if/else seems pretty much as good.

Unfortunately monads aren’t in base scala (as an explicit class at least), so often you need to do some ad-hoc pattern matching on tuples or data structures and pattern matching really helps in de-structuring and binding relevant variables.

I probably use monadic operations (map, fold, etc) at least 5-10 times a day in my job (I’m a scala dev), and match expressions at least 3-5 times a day.

So pretty useful I’d say. Though then again I’m dealing with some pretty baroque case classes that contain dozens of values of financial information.

> Unfortunately monads aren’t in base scala (as an explicit class at least)

Scalaz and cats are perhaps the most common dependencies in the Scala ecosystem.

I suspect trying to combine pattern matching with java’s container classes would be an exercise in frustration. What struck me about scala was how much nicer its standard library is, even when using only things java has equivalents for.

This is on the roadmap.

Hopefully project Amber will arrive soon.

https://cr.openjdk.java.net/~briangoetz/amber/pattern-match....

I'm bummed that the raw string literals are delayed, but the new switch statement does look nice.

Reply on news.ycombinator.com