lkitching

A monad is a triple:

1. A type constructor M of a single argument 2. A function of type a -> M a 3. A function of type M (M a) -> M a

subject to the monad laws. So the 'array monad' is the triple (Array, \x -> [x], Array.concat) and the 'option monad' is the triple (Option, \x -> Some(x), \x -> match x | Some(o) => o; None => None). Saying a type 'is' a monad is imprecise but just means it has an associated monad instance. I've never seen a type with multiple monad instances but I don't know if there could be one.

The type of join for arrays is Array (Array a) -> Array a which should concatenate all the inner arrays together.

> You are so hung up on the types that you are just failing to take in the essence of what's going on.

The post is about type-driven design, which is about representing invariants in the type system where possible. The post is very clear about this. The example chosen to illustrate it is a very simple one where a list is augmented with an extra property (non-emptiness). The exact representation is not important, so yes they could have created their own custom type instead of using (NonEmpty a) but this is beside the point. You have now made two attempts to 'improve' this representation, first by just using a plain list (which the post explicitly rejects) and now by just using a single 'cache dir' instead. You can't 'simplify' the solution by just throwing away half the requirements - it's a collection of items which must also be non-empty.

> The author's example code is NOT illustrating parsing. Period

Once again, the author is very clear about what they mean by 'validation' and 'parsing':

    The difference lies entirely in the return type: validateNonEmpty always returns (), the type that contains no information, but parseNonEmpty returns NonEmpty a, a refinement of the input type that preserves the knowledge gained in the type system.

The entire point of 'parsing' in this approach is to obtain a refinement of the input type in the representation. Your 'better' example is not a refinement of a list.

You appear to be insisting that validation is just anything that throws exceptions, but this is wrong - validation is when the properties being checked are not reflected in the input type. Parsers have to be able to signal errors, and exceptions is one of the ways of doing that. This is why your previous example of 'parsing' is just validating:

    const dirs = getEnv("dirs").split(",");
     if (dirs.length < 1) throw "ERROR";
     return dirs;

The type of this expression is just `[String]` which does not guarantee the non-emptiness being checked. If you have another definition of validation vs parsing you need to state it clearly, because your counterexamples do not contradict the definition in the post.

> Their examples are defining a function with a guard that validates the input. The author then get confused and decides to go on this side-quest of how to trick the compiler because the first function wasn't actually accomplishing their goal (they have to validate twice!). But you know what would have avoided all of it? Parsing

This is literally what the post is showing by switching

    getConfigurationDirectories :: IO [FilePath]

with

    getConfigurationDirectories :: IO (NonEmpty FilePath)

you keep insisting this is 'not parsing' but the post explains why they think it qualifies and you haven't given your own definition which contradicts it.

> It's just a matter of fact that the author is (mis)using `Maybe` to the detriment of their example

There's no mis-use of Maybe in the post, and as I've already explained, the point of the post is to eliminate Maybes. It's only used in two places - to represent the partiality of the head and nonEmpty functions.

> The requirements from the post are arbitrary and could (should) be anything that best-illustrates the thesis of the post

The post is showing how using more expressive types can simplify your code through a very simple example. You started out saying the NonEmpty type is unnecessary and now you're complaining it's not complicated enough, so it's not clear what your actual objection is here.

> The function in my example does statically define that the returned list is non-empty

The type given in your example was

    getConfDirs :: Maybe [FilePath]

this is inhabited by (Just []) so no it doesn't statically guarantee that.

> A "parser" would maybe return the non-empty list if parsing was successful

That's exactly what the nonEmpty function in the post does:

    nonEmpty :: [a] -> Maybe (NonEmpty a)

The requirement from the post is for a non-empty list of file paths - your Cache representation only contains a single item so is obviously not suitable. In case it's not obvious: head is not the only operation that might be required and the author isn't using (NonEmpty a) as a wrapper around a single value. The requirements for the configuration are stronger than those provided from the input and the configuration type used by the program encodes that property in the type. That property is then enforced globally throughout the entire program and only needs to be checked once at the top level.

> The above is not best-understood as a "parser". The above is validating the input

Yes the example you gave is an example of validating in the author's formulation because it does not enforce the property it's checking in the return type. You check the list is non-empty but this information is not available anywhere else in the program. A parser would return a (NonEmpty String) as its result since that does enforce the constraint.

> Trying to redefine "parsing" to mean "the result has a different return type"

That's not a redefinition of parsing, that's what a parser is.

The entire point of Maybe is to imbue some type 'a' with an extra value - Nothing - along with a tag about which case you have. So (Maybe a) is always inhabited by more values than 'a', and that is the sense in which a variable of type a is 'more precise' than one of (Maybe a). I'm not saying Maybe is bad in any way - as you point out sometimes you do have to deal with the possibility of not having a value e.g. looking up a key in a map, looking up a user from a database etc. In Haskell there's no 'null' value which inhabits each type so you have to use Maybe, but even in languages like C# or Java where reference types all contain null I would still prefer to use Maybe/Optional to be explicit about the possibility. I don't think we disagree here. But at any point in a program you would always prefer to receive an 'a' over a (Maybe a) if you had the choice since there are fewer cases to deal with. This is the same reason languages like C# are adding support for non-nullable reference types.

Type-driven design is based around encoding invariants as much as is practical in the type system (what constitutes 'practical' is constrained by the type system you're using). The (NonEmpty a) type is just used to demonstrate a very simple example of this principle. In the same way that type 'a' is smaller than the type (Maybe a), so (NonEmpty a) is smaller than a [a] which means the operations on it are similarly more precise, which shows up in the two version of head:

    head :: NonEmpty a -> a
    head :: [a] -> Maybe a
 

But this is just one example - you could replace it with different representations of a user in a web service

    type User {name :: String}
    type User = JsonValue
 

and the consequent difference in the types of the accessor for the name:

    getName :: User -> String}
    getName :: JsonValue -> Maybe String
 

Far from being a 'sideshow' this is the main point of the approach - using a more precise representation makes all the operations on it similarly more precise globally throughout the program.

In your post the argument to restOfProgram has type [FilePath] but in the post it is (NonEmpty FilePath) so you need to handle the potential non-emptiness of the list everywhere you try to access it, either by propagating missing values to a higher level or using 'unsafe' functions like fromJust. It's defensible to prefer using a simpler representation type and dealing with the imprecision, but it's not doing the same thing - the types for a lot of the internals of your program will be quite different. This is probably the main philosophical difference with Clojure which prefers to use a small number of simple types along with dynamically checking desired properties at the point of use, something which tools like spec and schema make quite convenient. But people use static languages because of the global property checking, so it seems odd to me to endorse explicit modelling of missing values with Maybe while rejecting doing the same thing for non-emptiness since they are both lightweight approaches.

The insight of the original post is that if you choose to try make your types precise in this way (and most Haskell programers would I believe) then the process of checking the properties you want to enforce from a less-precise representation is inseperable from the process of converting into the narrower representation. This narrowing process could fail and must therefore encode the representation for the failure case. Your insistence that Maybe should be used as the one true failure representation is wrong I think, throwing exceptions in Haskell is rare but but they could have also chosen (Either String) for example. Maybe isn't even a particularly good representation since it doesn't contain any way of describing the reason for the failure, just that it happened. I agree it would have been nice to see an example of parser composition using <=< etc. would have been useful there but it's not the main point of the article.

> You have the signature for `useCache` wrong. I defined it above (`Cache -> a`)

Yes, sorry it's actually the line

    maybeCache <- (getConfDirs >>= head >> initializeCache)

which shows the issue.

> but then the author eschews it in favor of... validation (with what appears to be some tricks to fool the compiler)!

I think the author is pretty clear about how they're using the terms 'validation' and 'parsing' in the post - validation functions do not return a useful value while parsers refine the input type and carry a notion of failure. The first two examples of parsers they give are:

    nonEmpty :: [a] -> Maybe (NonEmpty a)
    parseNonEmpty :: [a] -> IO (NonEmpty a)

you seem to be arguing that parseNonEmpty is validating because it throws an exception instead of returning Maybe (NonEmpty a) but this isn't true here since Maybe signals failure by returning Nothing errors within IO can be signaled with exceptions. The author hints at how these two parser types are related later on with:

    checkNoDuplicateKeys :: (MonadError AppError m, Eq k) => [(k, v)] -> m ()

There are MonadError instances for both IO and Maybe so the general parser type is something like

    MonadError e m => a -> m b

Admittedly this could have been made clearer if it was the intention and returning Maybe is preferable to throwing exceptions in languages like Haksell.

If you were translating this approach to other languages like Java or C# though you proabably would throw exceptions to indicate failure e.g.

    interface Parser<A, B> { B parse(A input) throws ParseException; }

so I don't think your objection holds in general.

> I'd like you to define "strictly worse" here

I'm saying you would always prefer to be handed an instance of an `a` instead of a (Maybe a) since it's more precise. You can trivially construct a (Maybe a) from an a but you can't easily go in the other direction. You either need to produce a dfeault value or use a partial function like fromJust to obtain an 'a' from a 'Maybe a'. The motivation for the post is to show how using a more precise data type allow you to remove these from the rest of the code.

> But why are variables in this scope (`main`) so important

The issue doesn't happen in main, it happens throughout the rest of the program. The high-level structure is something like:

    main :: IO ()
    main = do
      maybeDirs <- getConfigDirs
      maybeDirs >>= restOfProgram

main only has to handle the parse failure and report any errors which will look similar regardless of whether getConfigDirs has type Maybe (NonEmpty FilePath) or IO (NonEmpty FilePath) (and throws an exception). But the representation of the directory list could be used anywhere in restOfProgram. Given a chain of applications fun1 -> fun2 -> ... -> funN, if funN accesses the file list with head and receives a (Maybe FilePath) there are three options:

1. Use fromJust since the list should be non-empty 2. Produce a default value 3. Propagate the Maybe in the return type of funN

Option 1 is messy, 2 is also unlikely for a low-level function and 3 forces fun1 to fun (N - 1) to either handle or propagate the partiality. Yes using >>= and <=< etc. can hide this plumbing but can be made unnecessary in the first place.

The difference between parsing and validation in the author's formulation is not between returning Maybe and throwing an exception, it's between returning a more precise type and not. Here's the types of the two version of `getConfigurationDirectories`:

    getConfigurationDirectories :: IO [FilePath]
    getConfigurationDirectories :: IO (NonEmpty FilePath)

The second version is preferred because the (NonEmpty FilePath) encodes the property that was checked in the type which means it doesn't have to be handled repeatedly throughout the entire rest of the program.

Yes the second version could have been changed to one of:

    getConfigurationDirectories :: IO (Maybe (NonEmpty FilePath))
    getConfigurationDirectories :: MaybeT IO (NonEmpty FilePath)

but this would only have moved the error reporting up one level to the main function. I would guess the existing version was chosen to simplify the types for a non-Haskell audience.

your attempted 'improvement' of using

     getConfigurationDirectories :: IO (Maybe [FilePath])

is NOT an example of parsing because [FilePath] does not remove the possibility (in the types!) of the list being empty. When you later attempted to use

    maybeCache >>= useCache

this requires the type of useCache to have type

    [FilePath] -> IO a

for some output type a. This function must deal with the possibility of the input list being empty because the type allows it. Every call to `head` returns (Maybe FilePath) and must handle the Nothing case. Neither I nor the author is unaware that there are many combinators that make this more convenient than explicit matching against Just/Nothing but doing so is strictly worse than returning a FilePath directly. Presumably none of the lower-level functions will be able to provide a default FilePath to use so every single one will be forced to return a Maybe somewhere in their return type (or use fromJust which is very ugly). This affects every single one of their callees which will again be forced to propagate Maybe up to their callees etc. To reiterate: the issue is not the possible non-existence of Cache, which can be handled in main. It's that the representation of Cache forces every single operation on it (of which head is just one simple example) to potentially have to represent conditions that should not actually be possible. This is a failure to 'make invalid states unprepresentable', which most proponents of static types aspire to.

    If there are types that allow you to not care about the type, then why have a type?

The type represents a set of properties about the inhabiting values so if you have an opaque value you don't want others to inspect, wrapping it in such a type makes that explicit throughout the rest of the program. That information is not available if every value in your type has the same type. Presumably though you will actually attempt to do something with the contained value at some point so you must have some expectation about what it is.

Your example does not achieve idential behaviour at all since it 'parses' an [a] to another [a] and therefore throws away the very property you've just checked. The (NonEmpty a) property encodes the non-emptiness of the list in the type which is then known at every point the list is accessed throughout the entire rest of the program. The point is not just to check the non-emptiness in main as you appear to be implying. Any use of head on an [a] must continually deal with a (Maybe a) even though this possibility has been ruled out. In contrast NonEmpty.head returns an element directly so removes entires chains of Maybes that would be propagated, conveniently with map/bind or otherwise. Parsing allows to replace N + 1 instances of Maybe with just 1 so you can't claim your approaches are the same just because it hasn't been eliminated entirely.

The author explains what they mean by parsing in the post:

> Really, a parser is just a function that consumes less-structured input and produces more-structured output. By its very nature, a parser is a partial function—some values in the domain do not correspond to any value in the range—so all parsers must have some notion of failure. Often, the input to a parser is text, but this is by no means a requirement, and parseNonEmpty is a perfectly cromulent parser: it parses lists into non-empty lists, signaling failure by terminating the program with an error message.

So the properties checked by the parser are reflected in the output type. Reifying these properties in the type is what allows the validation to be done once at the top level and avoided throughout the rest of the program. Your complaint about throwing exceptions is focusing on an irrelevant detail in a small example - yes this could have been moved into the main function but doesn't affect the overall behaviour.

However your argument that propagating Maybe values is more idiomatic than parsing into a more precise type is one I - and I assume most - static typing advocates would disagree with. Given the choice you would always prefer an 'a' over a 'Maybe a' since a Maybe represents a point of uncertainty which you would rather not have. As a result, having to chain this imprecision using various combinators is inherently more complex than not having to do so. Yes, using bind etc. is preferable to manually destructing Maybe values but avoiding Maybe is more preferable still.

This line:

    maybeCache <- (getCofDirs >>= head >> initializeCache)

is doing exactly what the post is arguing against. getConfDirs is validating the list is non-empty but the [FilePath] list it contains does not encode that information. Now you immediately have to handle the possibility of a missing value from head that you already know cannot happen. This isn't too apparent here since you've combined it into a single expression but if you need to pass the confDirs list to any other part of the program they will also have to continually handle the possibility of the list being empty even though you already checked for that possibility. Now every function that interects with the confDirs list will have to include (Maybe a) in its return type unnecessarily. The post is not suggesting you can remove Maybe entirely but it has moved it to a single point in the program (the point where the config dirs list is checked for emptiness) and removed it everywhere else. Your approach must continually guard against an impossible condition everywhere the dirs list is accessed because you discard the property you checked for in getConfDirs.

The monadic operators make it convenient to propagate missing values through a chain of operations but they are not the primary benefit of an explicit Maybe type. Much like IO, the benefit of having an explicit Maybe type is when you _don't_ have it since its absence represents more information at that point in the program. Likewise a (NonEmpty a) contains more informatation than [a] which consequently makes the implementation of head more informative.

The parsers in this approach have types like

    a -> Maybe b

where type b contains the extra information extracted by the parser. Your getConfDirs function only contains a function with type

    [a] -> Maybe [a]

so isn't parsing in the same way.

1) Looking at the TIOBE index, all the static languages I recognised on there are: C,C++,C#,Visual Basic,Go,Fortran,Swift,Delphi,Cobol,Rust,Scala,Typescript,Kotlin,Haskell and D. Of these C and Go are the only two that don't appear to support generics so I don't think this approach excludes a lot of static languages.

2) If you want to distinguish GET and POST requests statically then you just need a type for them e.g.

    GetRequest<TBody> implements Request<TBody> { }

if you don't need to do this then you can just add a method field and use a single type for both. Either way you don't need to use sum types so a language like Java can express it.

3) Yes you'll have to make functions that don't care about the body type generic so this approach could become unwieldy if you have a few such properties you want to track.

On the contrary the The NonEmpty type is fundamental to the approach in that example since it contains in the type the property being checked dynamically (that the list is non-empty). The nonEmpty function is a simple example of the 'parse don't validate' approach since it goes from a broader to a more restricted type, along with the possibility of failure if the constraint was not satisfied. The restriction on the NonEmpty type is what allows NonEmpty.head to return an a instead of a (Maybe a) and thus avoid the redundant check in the second example. The nonEmpty in your alternative implementation is only validating not parsing since after checking the input list is non-empty, it immediately discards the information in the return type. This forces the user to deal with a Nothing result from head that can never happen. Attempting to clean the code up by propagating Nothing values using bind is just hiding the problem that the validating approach avoids entirely.

In my experience the need to mock out individual methods like this is an indication that the code is badly structured in the first place. The time source is effectively a global variable so in this example you'd want to pass the time as a parameter to `sayCurrentTime` and avoid the need to mock anything in the first place. A lot of C#/java codebases do seem to make excessive use of mocks and DI in this way though.

If you want to use DI, in java 8 you could inject a java.time.Clock instance in the constructor and provide a fixed instance at the required time in your test e.g.

    Instant testNow = ...
    User u = new User(Clock.fixed(testNow, ZoneOffset.UTC));
    u.sayCurrentTime();

although it would be better design to have sayCurrentTime take a date parameter instead of depending on an external dependency.

The purpose of Maybe is to explicitly represent the possible non-existence of a value which in Haskell is the only option since there's no null value which inhabits every type. The existence of the monad instance is convenient but it's not fundamental. The type of getConfigurationDirectories could be changed to MaybeT IO (NonEmpty FilePath) to avoid the match but I don't think it would make such a small example clearer.

Any statically-typed language with generics can express that by parameterising the request type with the body type. A bodiless request is then just Request[Nothing] (or Request[Unit] if your type system doesn't have a bottom type). Accessing the headers just requires an interface which all static languages should be able to express.

Having a preferential contract with a single supplier is not remotely the same as imposing export controls on every supplier within a territory. The EU is and always has been allowed to source vaccines from the UK and the fact it voluntarily chose not to does not mean it was 'effectively' banned from doing so. The British government spent months and tens of millions of pounds setting up the AZ supply chain while the EU were haggling over its contract. It's not then unreasonable for the British government to require that it has priority on the supply it was instrumental in creating.

If you only have a day and month like 25th December you should represent it with a type that contains just that information. Java for example has the MonthDay class (https://docs.oracle.com/javase/8/docs/api/java/time/MonthDay...) which can be converted into a local date when the additional information is available. If your users want to refer to that as a date then that should be handled in the UI but should not lead to ambiguity in the internal representation.

The fact that one is checked statically and the other is only enforced at runtime is a significant different between them, since the static approach will reject some programs that are dynamically safe. The term duck typing when employed by dynamic languages like Ruby is misleading since it's a consequence of not having a type system at all.

I'm not sure what you mean by 'not exact' since the article you posted explictly contrasts duck typing with structural type systems of which Go is given as an example.

The author is advocating this method when the functions being derived are sufficiently polymorphic which your add function is not.

This isn't about encoding the validation rules in the type system, only the requirement that validation with respect to some properties has been done. So in your example you would just define a function

    createPurchaseRequest :: UserRequest -> Purchase -> Maybe PurchaseRequest

now everywhere in the code taking a PurchaseRequest can assume it is valid.

> CS majors spend their first few years unlearning all of the theoretical constructs they were taught in college

Do you have any examples of these theoretical constructs? I find most of the things I learned in my CS degree useful for commerical software development and can't think of anything I had to 'unlearn'.

I don't think this is true, do notation (see https://www.haskell.org/onlinereport/haskell2010/haskellch3....) specifically depends on the Monad typeclass and exists as syntactic sugar for monadic operators:

> A do expression provides a more conventional syntax for monadic programming

> If a system misbehaves you will catch in higher level tests in the test pyramid eventually.

You might catch it eventually, or you might not. That's the problem with hoping you write enough tests.

> he can and should add a dedicated unittest to reproduce the issue when he fixes the bug

His argument is that dynamic language don't 'really' require writing more exhaustive tests than in an expressive static language. In this scenario, not only would he need to write an extra test but he also needs to spend time after the fact tracking down an error thrown in a higher-level function.

> In a software solution you can't prove 100% correctness

This isn't about proving 100% correctness, it's about what happens when you pass a non-list into a list processing function. More expressive type systems allow you to reduce the number of nonsensical questions you have to answer.

But the reason Javascript isn't a functional language is because its built-in data structures are all mutable which encourages an imperative style, not because its functions are tupled or it lacks a function composition operator.

I agree syntax is important, but your definition rules out languages like Clojure which most people would consider functional. Haskell programmers might disagree, but I would assume on the grounds of purity rather than the syntactic overhead of partial application.

> you really can’t do FP without automatic currying and the function composition and application operators (. And $) in Haskell

But lambdas are enough to define these operators e.g.

    (defn comp [f g] (fn [x] (f (g x)))
    (defn ap [f x] (f x))

Operators and currying by default can use useful but it's a syntactic issue rather than a fundamental one.

I've been working with clojure professionally for over 4 years. The default navigation in emacs is non-existent as far as I can tell and only works at all once you've loaded the source into the REPL. Even that only works for resolving top-level vars and doesn't work for local bindings. Admittedly I tend to use cursive so this may have changed recently. But that still doesn't help you understand the argument/return types of functions etc.