Rezoom.SQL: Statically typed SQL for F#
github.com
github.com
Relevant for folks:
FSharp.Data: http://fsharp.github.io/FSharp.Data/
SqlProvider: http://fsprojects.github.io/SQLProvider/
FSharp.Data.SqlClient: http://fsprojects.github.io/FSharp.Data.SqlClient/
Azure Storage Type Provider: https://github.com/fsprojects/AzureStorageTypeProvider
I’ve seen this statement twice on HN over the past few days, so I’m getting curious about F#. I would like to understand, however, exactly how type providers change the game.
“An F# type provider is a component that provides types, properties, and methods for use in your program.”[1]
Is there an essential difference between an F# type provider and a Haskell module (which also exports types, properties and methods)?
[1] https://docs.microsoft.com/en-us/dotnet/fsharp/tutorials/typ...
F# Type Providers work in the background _while developing_ to provide types to your solution. In one sense they're just a lifecycle hook to generate code: they create .Net Types and then expose those types to your development environment (ie provide/export).
In practice this means you can give some JSON to a Type Provider, it will create types that match that JSON, and then those types are made available while developing so you get live intellisense and code completion based on the content and its structure through strongly typed .Net objects.
Alternatively, if you're creating some kind of DSL: a custom type provider will allow you to translate your data files into code objects that can then be used for strongly typed interactive scripting. Say, opening CAD files and publishing them to different repos depending on number of objects, auther, etc.
The readme for the project linked in this thread shows them in action :)
Type Providers give huge comfort to working with strange data by seeing all the fields immediately and interactively exploring the data schema. It's also veerrrryyyy cool to write scripts and apps that will simply refuse to compile if their assumptions about their data sources aren't met if you're dealing with third party data, or chaotic production environments, or sloppy cowboy co-workers like myself ;)
For rapid prototyping, I think ORMs are still the best solution. However, their benefits start to diminish when scaling the team and product start becoming priorities. Working /with/ the database becomes a lot more fruitful than trying to wrangle it to your format. After all, web applications are just database skins.
SQL Server Data Tools does this exceedingly well. It's a Microsoft-built addon to Visual Studio.
Catching errors during compilation of SQL files is certainly very helpful! They also provide a very nice diff tool where you can diff between your project SQL files and a given server/database or between 2 different projects or 2 different databases... It also has a nice data-diff tool. (All for SQL Server only of course.)
This tool helped me very much to replace heavy ORMs (such as EntityFramework and so on) with just Dapper and a bunch of query procedures that are kept in the database itself (known as Stored Procedures in SQL Server). More detail on my setup in this comment - https://news.ycombinator.com/item?id=14852425
Personally, I prefer my ORM/query builders to be more structures and adhere to the parent language more, with sane escape hatches for small bits of raw SQL as needed. This does look like it would be cool for a project that took the inline SQL or SQL phrasebook approach that wants to add a bit of structure, as it might be easier to put this in place over that?
The choice to typecheck a SQL dialect directly instead of using an embedded DSL like LINQ is uncommon enough that it gets the majority of the focus in my own documentation. However, it wasn't what motivated me to create the project.
I wanted to have an ORM that could take full advantage of my resumption monad library, Rezoom. This library is similar in concept to Facebook's Haxl or Dataloader, in that it can combine independent requests into batches and cache ones that have already been resolved.
The main difference (besides being F#) is that it is intended to be used for mutations, not just queries. This is a bit tricky because when you mutate the stuff in the database, that may invalidate the cached results of previous queries. The problem isn't too bad since, as with Haxl and Dataloader, the cache is per-request so it's very short lived, but it could still cause problems. Contrived example:
// domain layer line item logic, includes business rules
// returns the updated invoice
let addLineItem invoiceId lineItem =
plan {
// query for the invoice
let! invoice = DB.getInvoiceById invoiceId
if invoice.IsSent then
failwith "Can't edit this invoice because it has been sent"
// run INSERT/UPDATE statement(s)
do! DB.addInvoiceLineItem invoiceId lineItem
// re-query for the invoice to get its current state
return! DB.getInvoiceById invoiceId
}
`DB.addInvoiceLineItem` needs to invalidate the cached result of `DB.getInvoiceById`. Otherwise this code won't return the updated invoice, but the invoice before updating. Again, contrived example, but when the "reload the invoice" line is in a whole other function somewhere else it could be a real problem.The way Rezoom solves this is to have "errands" (its word for a batchable data dependency) specify 2 128-bit masks. One represents its dependencies, the other represents its invalidations. When you run a command with a non-zero invalidation mask, all the cached results whose dependency bits overlap with the invalidation bits get evicted.
The caches are local to a "category" which is pretty much just a tag for which library created the errand, so that the errands wrapping, say, a web API don't stomp on the caches for the errands talking to a SQL database (actually the connection string is involved too so you can deal with multiple databases from one app). For the errands generated by Rezoom.SQL, each bit in the mask represents one table, so running an UPDATE/INSERT/DELETE invalidates the cache for all commands that read from the affected tables. If you have more than 128 tables in your database the IDs wrap around so you'll get some false invalidations possibly hurting performance, but never a false cached result hurting correctness.
This all could make it very easy to write a GraphQL server backed by Rezoom + Rezoom.SQL.
If your goal is just to run queries, either synchronously or as `Task<T>`s, that'll do just fine. If you want the batching/caching stuff I described above, you'll want to build `Plan`s, which currently requires an F# computation expression.
However, in C# 7 I think it's possible to write plans with `async/await` syntax. I just need to write the AsyncMethodBuilder to support it. I've done it for F# asyncs here: https://github.com/rspeele/FSharpAsyncMethodBuilder
This would let you get the full batching/caching/transactional benefits of Rezoom even when writing a project that's 90% C#.
I think type providers are a really fantastic feature. I just wish that I had more cause to write f#.
I am currently with SqlProvider but raw SQL would be interesting option =)