Yes, it is vague. This was 20 years ago.
> Do you have any more details?
If you read the text carefully, and strip it of the "we were so awesome and those idiots didn't understand just how awesome we were" you can actually see quite a bit of what I was saying in the text itself.
For example: "... neither [of the authors] attended the kickoff meeting at NSWC". So it's not entirely surprising if (as I suggest), they weren't fully aware of the actual purpose [especially if, as the other paper suggests, there were relevant discussions that preceded the kickoff meeting].
Or the following: 'One observer described the solution as “cute but not extensible” (para-phrasing); this comment slipped its way into an initial draft of the final report, which described the Haskell prototype as being “too cute for its own good”'
Again, the authors frame this as the judges not understanding how wonderful their solution was, i.e. "everybody else is an idiot". Well, maybe everyone else isn't an idiot, and maybe the evaluation had something to do with not really contributing to the actual software-architectural purpose, which in turn had something to do with the authors not being there for the kickoff meeting?
Of course, I wasn't there either, but this sure is what it looks like to me.
Here is another paper, which talks more about the original problem and purpose of the competition, which was to be a sample problem (with sample solutions) for software architecture:
http://www.cs.cmu.edu/afs/cs/project/able/ftp/aegis-iwssd8/a...
I remember seeing a more comprehensive report, which also had larger code samples and talked about the disqualifications, but I can't seem to find it right now.
> It happened that some modifications and extensions to Reference 1 were agreed to at the experiment kickoff meeting at NSWC. Dr. Jones and Dr. Hudak were not informed of those extensions until after the completion and review of the initial draft of Prototype 1. [...]
> Prototype 2: We believe that our Prototype 2 will be most comparable to the prototypes prepared by the other teams. This prototype satisfies all mandatory requirements of the problem as defined in References 1 and 2, and also incorporates generalizations to support anticipated future requirements. To accomplish this task, the original author of Prototype 1 made a number of modifications and enhancements requiring about five hours [...]
P1 and P3 solutions are attached in appendices, along with some others.
[1] http://www.cs.yale.edu/publications/techreports/tr1031.pdf
I skimmed through the paper and I don't see where they state the P1 prototype (or any prototype, for that matter) was "disqualified", as user mpweiher claimed. The authors continue to state they found Haskell and FP to be great for rapid prototyping... and in my opinion, anyone who's ever written a prototype will agree that making changes at short notice is expected :) And Haskell turned out to be very suitable for this. I think they were right to call it a success.
More importantly, that they were able to successfully (and with relatively little effort) generalize and extend the prototype shows that it was indeed extensible. That they were able to provide three prototypes is also telling, because even if some were delivered after the experiment, they were still made in a very short time. Imagine managing this feat in C++ or Ada!
It'd be interesting to see how many of the other teams which completed the assignment also had to implement some changes after the review.
The author singles out the use of higher-order functions as an elegant technique of which the other participants weren't aware, even though some of their languages actually supported it! This technique is precisely what another participant considered "too cute for its own good" -- even though today we know higher-order functions are hugely useful and relatively mainstream, enough that few devs would find them surprising. Another objection was that they couldn't believe the Haskell prototype was a finished executable program and not a top-level design document mixed with some pseudocode... which speaks a lot in Haskell's favor!
To me, Paul Hudak actually sounds humble in his paper. He acknowledges the limitations of the experiment, he is careful not to say this "proves" anything, but is enthusiastic about what it suggests, i.e. that Haskell can be used for rapid prototyping with great success! I'd say this is an uncontroversial conclusion.
It's unfortunate that you cannot back the claim that the Haskell solutions were disqualified. It certainly contradicts the paper, which shows the final scorecard by the review panel...
Well that's precisely the vague stuff you were talking about in your original post, so that's very unfortunate.