Manifold: Java Type-Safe Metaprogramming, Structural Typing, Extension Methods | Hacker News Reader

Manifold: Java Type-Safe Metaprogramming, Structural Typing, Extension Methods

github.com

126 pointsjeanlucbernard43 comments

Manifold is an amazing piece of engineering. Scott McKinney, the author, used to work on Gosu but mostly focuses on this now.

He added GraphQL support a few weeks ago:

https://github.com/manifold-systems/manifold-sample-graphql-...

You can just drop graphql files in your resources directory and start coding against them as type-safe interfaces immediately, no code-gen required.

The @Jailbreak annotation is awesome too.

Looks pretty cool. Reminds me a lot of the ways in which Groovy improves Java.

Apache Groovy really improved on Beanshell (by adding closures, in 2003), which is what improved on Java (by adding dynamic typing, in late 1990's).

I thought I knew all about Java, but this thing makes me feel like I haven't got a clue! How can it achieve this stuff by just being in the classpath?? No Java agent can be involved, no codegen during the build?! This shouldn't be possible, or??

Ok just checked this deeper. The [docs](http://manifold.systems/docs.html) actually show this is a javac plugin (I didn't know about the `-Xplugin` flag, but you ALSO need to pass that to javac for Manifold to work)... you also need the `tools.jar` file (which includes the JavaCompiler) on the runtime classpath... so suddenly, there's no magic... it's a bit misleading to say this does not require a build step, it clearly does, but as it integrates with javac itself, it "feels" seamless...

There's a significant difference between a code gen build step and what Manifold is doing with javac. Resource files are virtual source files, this represents a leap in terms of dev productivity.

Yes, I agree... just wished they made it clearer how they achieve that!

The documentation is a bit handwavy on how it works indeed. Just drop a jar file in the code doesn't quite tell the whole story. However, I'm impressed with how amazingly awesome this stuff is. A lot of the stuff it does is already in Kotlin but being able to use json and property files without boiler plate code is nice. I'd like to be able to do that in Kotlin as well.

Note tools.jar is needed only for Java 8, and only for compile-time, unless you use Manifold's dynamic loading feature. The JREs for Java 9+ bundle what used to be called tools.jar.

How much of an update hazard is this? Does it break with every major java upgrade?

See my earlier comment re javac plugin API, Manifold uses that as a means to hook into the compiler.

Probably the most interesting aspect of Manifold is its role as a type name resolver for the compiler. Manifold provides an API to dynamically resolve types corresponding with, well anything, but mostly with resources like JSON files, GraphQL schemas, SQL, DDL, CSV, scripting languages, you name it. So from the compiler's perspective it has no idea what is going on beyond normal type name resolution, but the source files it's loading up are coming from deep, dark places ;)

But yeah, no code generation step in your build, no agents, no custom class loaders, etc.

If you like this, check out Project Lombok [0] as well. It gives you var/val for local type-inferenced variables, getter/setter code generation, and others to help eliminate boilerplate.

[0] https://projectlombok.org

Its interesting with the other thread about Java EE dying and Oracle politics. The ecosystem around Java and JVM is huge, so much cool stuff going on. Definitely a great platform.

How is the user experience with that if you don't use Intellij? It seems to require explicit support by the IDE, which is why it doesn't seem to work with eclipse.

(Hi, I'm the author of Manifold) Manifold works equally well with Java 8 - 12. While no IDE is required, it is designed to fully leverage Java's static type system, which heavily benefits from the static analysis provided by a topnotch IDE like IntelliJ IDEA. Code completion, deterministic refactoring, usage searching, feature navigation, incremental compilation, etc. -- suddenly these productive features are now available to any resource connected with Manifold e.g, GraphQL, JSON Schema, Javascript, etc.

Re Intellij v. Eclipse. I'm just one guy banging on a project; I only have time to build and maintain one IDE plugin and right now my focus is on IntelliJ. I'd like to start working on an Eclipse plugin at some point... or trick someone else into doing the work ;)

First they came for the particles (electron, gluon, atom) now they are coming for mathematical constructs and give them mundane everyday meaning .

Or car parts, who can tell...

FYI: Manifold is a pretty popular Clojure library by Zach Tellman.

https://github.com/ztellman/manifold

Wow, this looks like a god-send. The @Jailbreak annotation would give mid-2000's Java coders a heart-attack :)

The only contentious point I find is that they are basically replicating Project Lombok's @SneakyThrows annotation via "Xplugin:Manifold stringsexceptions" or ModelMapper in their Structural Typing.

Furthermore it seems it should be structured in sub-projects, eg. the String-templates and ManTL in one lib, Structural Typing in another so we can mix & match.

But overall, very impressive!

Thanks!

The 'exceptions' option is quite a bit different from SneakyThrows. The 'exceptions' option completely mutes checked exceptions in your project -- you don't have to declare a throws clause or catch a checked exception anywhere in your code, thus with the option enabled checked exceptions behave exactly as unchecked exceptions. This behavior is identical to exception treatment in other JVM languages such as Scala, Kotlin, Ceylon, etc.

Regarding sub-projects on extensions, I've been meaning to do that. The project is already divided into separate targets e.g., manifold-graphql, manifold-json, etc. But manifold-ext has probably grown too big for one target.

My god, I just read a bit more about the Extension Manifold, and that is crazy good. I am doing both Java and JS, and I started to like duck-typing and extending prototypes so much in some situations. Now, Extensions give me something similar in Java?

I have the highest respect for your work, and no idea how you are doing that more or less on your own. I am just a bit sad your upcoming SQL/DDL extensions are going to kill one of my side-projects :) https://github.com/iSnow/sqlWebservice - if you can use a SQL schema as a type provider, you are leaps and bounds ahead of what I am doing there.

So, hats off, great job.

This is what I wanted all my life! I wonder what kind of dark magic makes this go and how this will come back to bite me if I use it?

Manifold hooks into the Java compiler via the javac plugin API. This is similar to the way annotation processors work, but without the overhead of additional passes (or rounds) in the compiler. The plugin route also provides access to earlier stages of the compiler, for example this is how Manifold's string interpolation feature works. The overall details are a bit deep and dark for a comment, however ;)

I've played with the compiler's API before and found it incredibly difficult to work with... and completely undocumented... how did you manage to get through that? Great job!

Thank you! After I got past the initial shock with the javac codebase (it's an unusual design) I came to appreciate it. Given it's age and the number of revisions it has undergone, it holds up pretty well. But, yeah, building Manifold required a bit of determination.

At that point, why not just use another jvm language with these features like Kotlin?

the main feature of this allows you to transparently use structured data as types in your code like type providers in .net. I love kotlin but it can't do anything remotely like that.

Yep, I'm wondering if this actually works in Kotlin as well. I'm thinking it may probably play nice with this as is since Kotlin can interact with Java classes. However, the compiler plugin might be a problem. If so, this would be an awesome thing to add to Kotlin.

at that point why not use C#?

C# doesn’t support type providers, only f# supports them.

or Scala https://www.scala-lang.org/

Scala comes with a lot of overhead in terms of features (and breaking changes in versions) and is still missing the meta-programming aspect of this library. In fact, scala has now deprecated all it's previous attempts at meta-programming and is still trying to work out it's next attempt (which is I believe black-box only so can't do what this library does).

Ever tried to "sell" an alternative language in BigCo Enterprise? Where I work, Javascript is still a front-end only language and just tolerated as a toolchain for compiling LESS, ECMAScript etc.

Scott McKinney, the guy behind that library, is also the guy behind Gosu [1].

Selling a new language is hard, selling a library is easier, even if this one has a hook deep in the code of javac.

[1] https://en.wikipedia.org/wiki/Gosu_(programming_language)

This feels a bit like Project Lombok: adding cool features to standard Java.

I like the Extensions option. It works very much like D's UFCS mechanism.

Removing checked exceptions - what a dangerous feature!

It's optional. It seems scary, but checked exceptions are a feature exclusive to the Java compiler -- the JVM does not enforce them. This is how JVM languages like Kotlin and others only use unchecked exceptions, and also how Manifold gets away with muting them as a compile-time feature.

It should not be optional :)

Checked exceptions is a stupid feature because it makes an implementation detail, the fact that a code raises an exception, part of the type of a method.

Scala, C# or Kotlin have no checked exception and it doesn't make them less safe.

No less safe until it the code is running in production on a Tuesday at 3AM and a library pukes up an exception it has never thrown before and your code doesn't handle it.

I prefer exceptions to be declared, just like types. Having exceptions declared gives me more information about the operation of the code I am calling and that is useful. To treat exception declarations just as annoyance seems like lazy punting.

Likely you don't care which exception is thrown, only that one is. In that case, it's overkill to add the exceptions to the type signature. You'd be far likely to be better off using IO<A> from Kotlin or IO[_] from Scala to model computations with exceptions as side-effects.

Besides, any non-trivial method that calls any set of libraries/frameworks would need to declare tens or hundreds of checked exceptions in your scenario. And that's just the catchable exceptions. What are you going to do about any (subclass of) Error?

that same 10s or 100s is true for error returns as well. what do you do with those?

convert it to internal types at the point you're exposed to them, then handle them in common ways. or re-throw as a general runtime exception if you don't have a way to handle it in your application. the difference with checked exceptions is that you have a correct, complete list of error scenarios that a library expects you to handle, and they provide a relatively easy way to defer handling.

there's no need for your `public static void main` to inherit all 100+ exception classes that your application is exposed to - that's just plain bad encapsulation.

---

granted, none of this touches on some of java's implementation issues, which essentially suffers from a lack of generics. e.g. you can't build an executor-framework without `raises Exception` or wrapping all errors in an internal wrapper, which loses compile-time safety. typed-exceptions-as-returns (i.e. typed result enums) keeps them within the normal type system, which does have some benefits for sure, but that ship has sailed. few languages are able to force exhaustive error handling at all, java's in a middle-ground with pros and cons.

The fact that a function can fail is not an implementation detail, it should be part of the signature of the method.

And if the method can suddenly fail in a new way, that signature should change so that callers of that method can adjust to the new failure case.

Checked exceptions force you to consider error cases, there's nothing stupid about them.

See the section on Lambdas:

http://manifold.systems/docs.html#checked-exception-suppress...

Reply on news.ycombinator.com