F# Data Type Providers in .Net Core (2018) | Hacker News Reader

F# Data Type Providers in .Net Core (2018)

lukemerrett.com

139 pointskroltan95 comments

Ah, F#, when I was desperately trying to get skills in .NET (lots of .NET in my town, and I don't have a degree) I encountered F# and lost my enthusiasm for C#. Which is doubly ironic now that I work for a JVM shop.

I still have a lot of love for it though. Especially things like measure types.

JVM has a lot of functional languages. As well as the usual Clojure, Scala, etc. I think it has some Haskell-like ones.

Is there an ML on the JVM? The best part of F# is it felt like a simpler version of Scala, even without things like higher kinded types that are present in OCaml.

If you squint very (very) hard, Kotlin can fill that role. You've got algebraic data types via sealed classes, a weaker form of pattern matching via `when` expressions, and decent facilities for higher-order functions.

I was going to comment the same thing. I moved from F# to kotlin and have about 80% of what I want / used from F#. With more tools eco system that makes management happy. Arrow adds to that, with it's library and compiler additions.

I have also been building type providers in Kotlin. Where a gradle plugin scans a file / schema then generates types and DSLs off of it. It's a pragmatic functional language.

The biggest catch to me. Is there are very few places using Kotlin in a fully functional style. On the back end more often than not you'll find Spring. Which undoes a lot of these features / patterns.

Incidentally, Java is getting pattern matching that is more capable than Kotlin's (and probably on par with Scala's).

It’s not exactly ML, but there is a pretty-ok port of Haskell to the JVM called Eta [0].

[0] https://eta-lang.org/

Nah there isn't, I looked. There's Scala, but I don't have 5 years lying around to get up to speed on it.

Frege is a Haskell inspired language on the JVM, about as close as I’ve found.

> I think it has some Haskell-like ones.

Eta might be what you're referring to: https://eta-lang.org/

(there's also Frege, but it's less haskell-y)

Could F# be ported to JVM?

Not until JVM supports value types, low level pointer manipulation and reiffed generics.

In addition to that, the .NET runtime supports tail calls but the JVM does not, IIRC.

Yes, JVM still has it under "some day it would be nice".

Actually this is one of the bytecodes that .NET Native doesn't handle, if I am not mistaken.

2 out of 3 of those are coming in Valhalla. You don't need pointers.

If you want to support F# without performance degradation emulating features, I am not so certain.

Yes, but it wouldn't be F#. F# is so deeply tied to .NET semantics that you'd have a very different language, even if the syntax looked the same.

JVM has already got clojure, scala and kotlin.

F# looks like a great language... A friend of mine who is a non-programmer DBA just reached out to me and asked about recommendations for learning .NET ...

I've not seen anything as elegant looking as F# in the .net world before ... What do people think ... -- is F# a good language for a beginner to learn which is also sufficiently well-supported that they could find themselves building useful script-like database (MSSQL) oriented utilities as early project ...?

F# has tended to lag far behind C# in Microsoft's ecosystem. I haven't kept up with in a few years, so I don't know how far behind it still is. It is a fun language for sure.

If your friend is looking into getting into .net for professional purposes, I would steer them towards C# first. That's likely to be much more useful in the long term. Especially since it can be a stepping stone to other C/C++ like languages.

With F# I would bet that they hit some limitation sooner or later where something that could be easily done in C# will take more work in F#. Which could be fine if they have the time/inclination to rough it. I just would not advise F# as a resume builder.

While I fully agree with F# being the step child of C#, an easy way to see what’s better is how many features get copied from which language to the other (hint: C# is like China copying tech from F#).

Start with F#. Learn the basics well.

Parent - Checkout

https://fsharpforfunandprofit.com/

https://docs.microsoft.com/en-us/dotnet/fsharp/introduction-...

>With F# I would bet that they hit some limitation sooner or later where something that could be easily done in C# will take more work in F#.

From my experience of working with both C# and F# for a few years, it's more likely to be the other way around.

F# lacks the Visual Studio tooling support that VB.NET, C# and even C++ enjoy.

It is also not officially supported on UWP, and needs some hacks to the generated MSIL (as .NET Native doesn't cover 100% of all MSIL).

The support for F# in VS leaves a lot to desire, indeed. However, in my experience, I've found that the developer productivity gains from F# outweigh the inconvenience.

Only if what you are doing is directly supported by F#.

If you attempt to do GUI development, then you are forced to look for community projects like Fabulous, use the XAML type provider with a dummy C#,VB.NET,C++/CLI project for the XAML generation, or actually use one of those languages for the UI, with F# used only for the logic.

So even though I am a big fan of ML language family, at the end of the day, the other .NET languages get the green light from most customers and team.

I don't recall having too much difficulty getting the VS WPF XAML Designer to basically function with F#. Some features just don't light up like event connecting/code-behind, but that's fine and what I wanted anyway (taking an MVVM-ish approach).

There's a hope that with F#'s upcoming move to the "shared project system" that a lot of the pains will go away in hopefully the near future.

(That said, Fabulous has piqued my interest and I'm debating rewriting my big F# GUI in it, at least in an exploratory branch.)

Most of the F# community seems to be using Ionide on VS Code. This works very well, but if Visual Studio is your preferred editor this can be annoying.

IDE support is much more than just editing text.

For example, try to generate EF 6 Database First code for F#, or use Blend alongside Visual Studio with F#.

Ionide gives you IDE features like type annotations and auto-complete.

Completely ignored my use cases.

I was responding to:

> IDE support is much more than just editing text.

In case anyone reading this thread comes away thinking that VS Code + Ionide is only a text editor.

Meh - recently worked on a huge corporate solution where some enthusiastic person included F# projects for tests- soon the project readme included : To get VS intellisense working disable the F# projects as it would randomly cause intellisense to break in the entire solution.

There was an issue with the C# tooling about 3 years ago that led to this, but it was fixed in the subsequent VS update.

Had this problem one year ago using VS 2017 and up-to-date software (along with the rest of the team, VS 2019 wasn't stable yet don't know if it's fixed there).

First I'd recommend upgrading to VS 2019, because you're currently missing ~1.5 years' worth of improvements in C# and F# tooling (core development for VS 2017 stopped well before the VS 15.9 release to stabilize it). If it still reproduces, then please use the Report a Problem tool to file an issue. The team who builds C# tooling would be interested in fixing that kind of problem.

Oh for sure - I'm not using .NET/VS anymore different clients and all, and I think the team stripped out the F# tests out of the solution - just saying it couldn't have been the thing fixed 3 years ago.

No, just point them at C# so they will actually be able to find answers to problems they run into with Google/StackOverflow.

C# is actually a pretty good language. It has functional features, even if not as extensive as F#, and has an OO system that is much more well thought out than Java.

If functional is what is needed, and C# is your only option, then this will help [1]

Disclaimer: I'm the author

[1] https://github.com/louthy/language-ext/

Given that C# and Java have copied features from each other throughout the years, and as someone that uses both stacks since their introduction, I wonder what well thought OO features does C# have that Java lacks.

It’s just cleaner and less verbose; eg interfaces can be implemented privately outside of the class namespace, generics are reified, etc... But then there are things like Extension methods, for example, which are incredibly useful.

Generics being reified are a CLR feature available to any .NET language (which Java in the form of J# could take advantage of [0]), don't have anything to do with C# OOP model per se, and were an impediment to properly implement Scala.NET.

This will be fixed on Java side, with value specializations as per Valhala roadmap.

Extension methods are also a way to create lots of spaghetti code, trying to track down where the method is actually implemented.

Swift and Kotlin codebases are good examples on how one can go a bit too far with extension methods.

Java's solution is to offer default interface methods as an approach to traits, which was copied by C# 8.

Regarding interfaces, if you mean explicit interface implementations, besides forcing casts everywhere I don't see what is so well thought out about them.

[0] - https://docs.microsoft.com/en-us/archive/blogs/gauravseth/cu...

Generics being reified in the CLR are entirely biased to C# (and vice versa), which is exactly why they impeded scala.net. They were simply designed for each other.

Extensions methods are great, tracking down the methods that you are calling is super easy in an IDE. The ability to abstract over generic type parameters as well as class subtype is very useful.

Default interface implementations have no connection to extension methods, they solve different problems.

Explicit method implementations are great at avoiding name collisions as well as accidental ones. The coercions needed are not hard casts, though C# could do be improved with a soft cast operator.

CLR is called Common Language Runtime and not C# Language Runtime.

In fact, C++/CLI is the only Microsoft language that has full access to all CLR features, not C#.

A language that has a much more expressive OO model than either Java or C#, and requires reiffed generics.

IDEs don't work in code review workflows.

For all intensive purposes, the CLR evolves with C# in lockstep. Multi language might have been a goal at one time, but isn’t really anymore. No one really uses C++ CLI.

> IDEs don't work in code review workflows.

Use or make better tools.

You should come out from your Ivory tower.

Thousands of developers use F#, VB.NET, Powershell, C++/CLI every day, and to lesser extent Cobol and Fortran compilers for .NET.

C++/CLI support was a must have requirement for .NET Core adoption, not only because of externals, because all .NET UI Frameworks make use of C++/CLI.

So much for no one really uses.

We use the tools our customers are willing to pay for.

Care to provide a link to a code review tool with IDE like capabilities for code navigation?

Valhalla won't solve all issues with erased generics in Java. In C# "static T foo;" is a valid declaration that creates a separate "foo" for each specialized class.

Only if T is a value type, reference types share implementations.

The final semantics for specialized value types are not yet set in stone, so what Java might end up supporting is still a guessing game.

> eg interfaces can be implemented privately outside of the class namespace

Not sure what you're referring to here (you can implement interfaces of other packages in Java), but everything else you mentioned is not related to C#/Java's OO system.

A beginner could learn F# and lean on the .net libraries for any business needs.

If he wants to learn .NET, he should start with C# and then looked into the Framework and if he is particularly ambitious - the IL, etc. But I'm guessing he doesn't want to learn ".NET" but rather programming against MSSQL if he is a DBA. C# is easily the most mature language in this domain.

Also, since he is a DBA, he could be introduced to ".NET" via powershell as microsoft has been trying to make powershell the default IT/DBA shell for the windows world. It's like bash for windows but entirely object driven.

If your using F# want to build (cross platform) UIs take a look at https://github.com/AvaloniaCommunity/Avalonia.FuncUI

Am I missing something? This appears to be just outsourcing the record schema definition to a sample object in another file.

You still have to keep that sample updated to handle new runtime data.

Yes, but it is comparatively easier getting say, the result of an API's GET and pasting into a file as a sample, than manually tweaking types. At least when you're starting out or if it's a small F# program. You could even use the actual network path, but that's a bit... wilder in terms of maintenance/reproducibility.

Plus, since you can have arbitrary providers, such as a provider that connects to a DB and generates a type-safe "ORM" over your relational database[0] with no configuration other than pointing to the data source you're going to access at runtime. Perfect if you want to write little data transformation scripts.

[0]: http://fsprojects.github.io/SQLProvider/

It might be easier, but it's trivial to come up with cases where you cannot generate an example that completely satisfies given JSON schema. It feels like this only works for some specific rigid JSON protocols. You also cannot restrict the types to anything more specific than JSON primitive types.

A "better", certainly more common solution is to generate the type based on whatever source code is generating the JSON, if you have access to it (or have the API provided generate the signature for you).

I'm still learning, but from what I gather, there's

a) less of a need to deserialize things into specific concrete classes in functional languages

b) it seems to be idiomatic to convert data yourself if you need custom conversions, rather than relying on a library to automagically deduce it. this is helped by how simple doing the manual conversion ends up being in practice.

You can write a type provider that takes a schema instead of example data.

I don’t trust an ORM to automatically reverse a database (or generate a database from POCO types) without at least some configuration - otherwise it can not determine (for example) if two tables, where one’s schema is a subset of another, are meant to be subclasses or not.

But anyway - this system feels like when we use T4 to generate types as part of the build process, but more tightly integrated. I wonder if the same could be done for C# using a Roslyn add-in.

> This appears to be just outsourcing the record schema definition to a sample object in another file.

Not quite. There's an interface called IProvidedTypes that a component - a Type Provider - is responsible for conforming to. That component reads data of some kind and produces information in that interface. The F# compiler then takes that data and includes it in the set of symbols that it calculates for everything else in your codebase. The end result is that you get typed access to data with zero configuration aside from adding a package, and because this data is exposed to tooling, you get IntelliSense for it in your editor as the blog post shows. The paper about the subject is here: http://tomasp.net/academic/papers/fsharp-data/fsharp-data.pd...

> You still have to keep that sample updated to handle new runtime data.

This is only if you're working with a sample. In most cases, developers just point it at the real data source and program against that - the target URL for a JSON payload, the connection string to the SQL database, etc. There's still issues that can arise, such as a poorly-written Type Provider, but the most popular ones are well-written and won't do silly things like slurp in your whole database into memory.

the part you may be missing is this is dynamic while you are coding, so you get autocompletion of dynamic data sources ( or during an interactive session, which gives a dynamic language kind of experience, but with types)

I guess this just seems pretty run of the mill (to me) for most statically typed languages. You import package "X" it does some meta-magic... et, voila! You get "dynamic" types because it does some fast compilation on the meta data it has and generates the method signatures, classes, et. al for you.

Honestly, I feel like this should be super cool and I'm just super missing something... but I don't know what right now.

Could you explain the interactive session part more?

Everything happens at compile time, no runtime reflection, on whatever schema you throw at it (json, sql, etc.), with instantaneous IDE code completion -- it's totally amazing.

Not aware of any mainstream language that provides similar functionality.

That other file can be an online API spec, a database instance, etc. Anything that can be accessed programmatically.

It keeps a structure and defines the types for you to some extent. But obviously if there are breaking changes just like with anything else....things break.

With compile time execution, one can do similar things in other languages too.

This video shows a very advanced and practical way of using "static introspection":

Lambda World 2019 - What FP Can Learn From Static Introspection - Aditya Siram (Deech)

https://www.youtube.com/watch?v=ElHi2h9Ho6M

What if compile time and type level programming in functional programming languages were easy, something you reach for without even thinking about it? What if you could debug type errors with a simple compile time print statement? Write highly flexible systems by being able to introspect into types at compile time? Pre-calculate large portions of your programs for great efficiency?

Typed functional programming is a great and fun way to write resilient software, and as type systems have become more and more expressive in recent years, we are able to program sophisticated and useful properties at the type level for even better compile time safety. Just one problem: It is very difficult, requires advanced knowledge of the type system, the syntax is convoluted, the error messages are impenetrable, and it is nearly impossible to debug.

This talk will dive into why we should steal static introspection from languages like Nim, and D, state-of-the-art imperative programming languages which can solve all these issues, make type systems much more approachable without losing any expressive power, and offer new design possibilities for functional programs.

D is an imperative programming language? I thought it was multi-paradigm.

Is there is a moratorium on adding IL-compatible features to F# ? If so, is it keeping pace with the additions to C# ? If not, what work is being done?

Yes and no. Some things, like _protected_, are explicitly not included because they encourage a style of programming - inheritance - that is not in line with the goals of F#. But that's really the only one. All recent "IL innovations" have either already been built into F# or are set for F# 5 (such as Default Interface Members).

Thanks

Also C# IMHO is becoming a bloated language.

I would actually remove features from F# not add them if I could.

It’s the perfect language currently available. I use it for low level stuff (working with the new Memory and Span types) and high level stuff (building UIs with FuncUI)

https://github.com/AvaloniaCommunity/Avalonia.FuncUI

What’s C# got recently that doesn’t already exist in F#?

C# are pushing for type classes https://github.com/dotnet/csharplang/issues/164

which author of F#, previously, didn't like to add to F#. (Not hating on how he wants the language to evolve)

For clarity, the position is that large features like TC/HKT won't be added in a way that could conflict with a later-added .net/IL version of the feature.

There are real problems with IL compatibility. There’s a reason F# doesn’t have ML functors.

Support for .NET Native would be an example.

Default interface methods, not sure about the current support state.

The accessors protected private.

I am also not sure if all low level programming features added to C#, taken from Midori and until C# 7 only usable from C++/CLI, are also exposed to F#.

.NET Native is being replaced by .NET Core and Mono with .NET 5, which has been the plan for some time. Doing work specifically for that toolchain might have been more meaningful 4 years ago, but the amount of investment required relative to the benefit (number of developers targeting it) would have been pretty low too. Incidentally, it also works and publishing to the store is unblocked; just not supported via the VS tooling.

DIMs are set for .NET 5. Cut from Core 3.0 since quality would have been an issue.

_private_ has been supported in F# for ages; _protected_ is explicitly not supported. I'd personally like to see it supported for interop but this is the language designer's call.

All the low-level programming features added to C# have F# "equivalents", and I use quotes because semantics differ between the languages and so your experience using one or the other is going to be quite different depending on what you're doing. But the performance output will be the same.

.Net Native works

For some definition of "works".

Visual Studio tooling support is nowhere to be seen, and the github issue does least some caveats.

well C# 8.0 introduced async streams, for example

Yes, and the F# library for asynchronous sequence processing had support added for them shortly thereafter (https://fsprojects.github.io/FSharp.Control.AsyncSeq/library...)

It's worth noting that this isn't an "IL feature", but a new type that includes some knowledge of the type shape in the C# compiler. The existing F# package for async streams understands the new type, and it "just works" in F#.

F# allows you to implement such features as libraries using computation expressions.

So does C# with LINQ [1]. I guess C# language development tends to lean more on concrete syntax than libraries, which is a shame, but it is possible. It's also a shame LINQ can't be extended like computation expressions.

[1] https://github.com/louthy/language-ext/blob/master/LanguageE...

The async model in F# is entirely different from C#. Not sure if an equivalent of C# async streams would make sense in it.

F# is great, but it seems to be somehow slowed down by C#.

If there's a feature that takes time to implement (eg: Type Classes from Haskell or Polymorphic Module from Ocaml), people would rather wait for CLR or IL to support the infrastructures first.

As a result, there are many essential features waiting to be implemented forever.

I'd try F# if it were hosted on JVM or better yet LLVM. Unfortunately .NET is borderline nightmare inducing so I wouldn't want to spend my free time using it. And it doesn't seem to have any better adoption than OCaml which is its supposedly outdated predecessor.

I think was true before .NET Core, but now the experience is quite good.

It's also possible to use F# as a compile-to-js language using Fable. This code would run on Node or the browser.

Then there is https://github.com/dotnet/corert which allows ahead-of-time compilation to native code, but it is a bit more experimental.

Is it possible for the JSONProvider to take a JSON schema as input rather than just a piece of JSON?

It looks like there is/was one, but it was abandoned by its developers:

https://github.com/jet/JsonSchemaProvider

https://medium.com/jettech/json-schema-type-provider-why-we-...

I wonder if they could get the TypeScript compiler to do something similar in future.

Yeah, that would be very helpful since there's no dependent type in a foreseeable future.

For example, As a front-end heavy language, talking with other systems is quite ordinary -- a very useful case is reflecting GraphQL or SQL type without code generation.

I wonder if Rust macros could be used to similar effect.

Many languages with powerful macro systems can do the same. Still, it's cool/promising to see a .NET language capable of compile-time execution like this when they do not support macros in the same way that other languages do (but they do have plenty of reflection features and some codegen features as we see here).

Note: article is from 2018 - since then, the FSharp.Data package has long moved out of preview and many other Type Providers work well on .NET Core.

You're right, sorry for breaking the format. Hopefully a mod can fix it since I'm not able to edit the post anymore.

I picked that article just because it is a very concise explanation, a bit more proper for sharing, than the official documentation, which currently requires a bit of hyperlink-following and generally nonzero familiarity with the language.

Reply on news.ycombinator.com