ixmatus

That sounds like a feature not a product.

An Architecture, to me, is an established and idiomatic process of applying solutions to produce a novel output.

In the case of a computer program the "architecture" should be extremely well-thought out, idiomatic, and using established practices to produce a program that is easy to maintain, understand, and test.

I don't believe you need diagrams but in the case of Haskell, the types (if you design type-first) serve as one piece of the blueprint that is to become your program, your cabal file, directory structure, documentation (both user docs and API docs), and tests are all part of that blueprint.

Not quite, I would call that "solution".

Not sure if "Architecture" is the right word here. More like, "How I built a Real World Haskell Application".

When I read about how to "architect" something I'm think about how to structure my Cabal file (a common pattern in my apps is a "library" of types, parsers, and utilities that are jointly used by multiple executables of the same program). I also think about structuring my tests, the naming and structure of module directory hierarchies, maybe also how to structure the program around the Main entry point.

I think there may also be a more interesting couple of "sub-articles" in this article: "How I built a server daemon in Haskell" and "How I built a GUI client in Haskell".

It's about accountability.

There are two types of it: a leader either recognizes and understands the gain to be had from using something like Haskell and has the resources to hire someone really good at it (who is therefore accountable, but he/she knows how to replace that person if they leave), or the leader themselves is an implementor in the esoteric technology and is comfortable being an accountable party themselves.

You rarely see the first type, more often it's the second type. Being accountable means that even if all of our programmers left us, I could at least keep the lights on without panicking (funny enough, that's actually easier to do in Haskell than a large Python / Ruby codebase).

BTW, it isn't hard to find them but it is expensive. These are people that have a level of motivation above the average and subsequently have a level of knowledge and skill also above the average. Basic economics can be used to answer why they are more expensive. Particularly when you start looking at specialized fields with a specialized technology: applied mathematicians that are also skilled Haskell programmers, or kernel hackers that also understand Haskell well.

Not quite black-hat; I consider tiered link building to be more blackhat.

Scraping the search results was about getting data on where items were positioned in the index, not pumping spam into the search results. Although to be clear, I did help do stuff like that for a while too but it never felt good and we vastly preferred the rank tracking product to link building services.

Also, Google's getting extremely good at combating search spam.

Meh. I didn't have any qualms with that, per-se. As others have pointed out I think there are ethical concerns with what Google is doing itself.

Also it's indicative of something if people are building businesses to scrape something they could offer as a pay-for API that many people would gladly / happily pay lots of money for. The arguments for NOT doing that are ridiculous because people will get the data regardless.

Right, but distributed systems are solved problem at this point, anonymity is much harder, and GNUnet's primary model is distributed / mesh darknets as a foundation for some of their anonymity features.

Read the docs? Their API is C based so it's not necessarily as easy as a Node.js or Ruby programmer might be used to.

They do have a P2P chat program built on-top of it so you can use that as an example and I think someone did their thesis on building a "facebook for GNUnet" on an older version. Not sure if that code is public though (it most likely is, I just don't know where).

I think you could, check out GNUnet, seriously. Those guys have been laboring away for a long time to produce a platform for anonymous and encrypted communication.

GNUnet can be used to build a darknet of any sort: P2P chat, a social networking application, email, you name it. Brilliant people are plugging away at it to build a platform for people with less knowledge of cryptography (like me) to build this sort of stuff on top of.

Writing distributed and highly-concurrent software in Erlang to scrape Google's search results, with localization, tens of millions of times per-day.

I don't work on that stuff anymore, but it definitely challenged my problem solving abilities to:

A) Learn Erlang. B) Learn how to write solid distributed software in Erlang. C) Figure out how to work around Google's temp-banning policies using IP balancing, captcha solving to produce cookies to balance connections on, and also how to localize the searches for geographic accuracy.

I don't like working on projects that are actively pitting me against someone though, so I'm happy to not be working on that. I now write scalable software for my energy-focused startup, we receive energy data from homes in near-real time, which has its own challenges.

Never seize but that's not the true problem, is it? Anonymity is the actual problem.

Why aren't there any of these guys trying to build this sort of thing on-top of https://gnunet.org/ ?

That's what I meant by "background backup slave". The slave can handle point-in-time backups which is better than doing dumps or hot backups on the primary DB server.

Good point re: multiple providers.

While this comment isn't particularly constructive to the discussion (nor is the one I'm writing right now), I did have a laugh :)

This is the reason for idempotent server configuration via Ansible, Chef, Salt, Puppet, or even better NixOS. Backing up a deployed application is kind of silly when the tagged releases should be sitting in your source control and that the press of a button have an entire server built and configured from scratch that is identical to the one you need. Then storing that configuration in source control.

Backing up configuration files and what-not is also pretty silly and indicative of bad devop practices.

The database should be versioned and maintained with a migration tool. So the only feature of the database that needs backing up should be the data it holds - live replication servers that can act as background backup slaves are perfect for this.

The only stuff you should be backing up on your servers are the latest log file rotations, user-generated data in the database, and any critical files that were also generated / uploaded by your users (but technically, you should be storing those in S3 with duplicates in another availability zone).

One other thought: people usually don't think to encrypt their backups. Duplicity + GPG + S3 is what I use right now and is super easy to setup with some fiddling. Once Tarsnap becomes a bit more professional I'll be moving all of my server backups to Tarsnap.

I'm moving over to BtSync ASAP; I really wish there was a Tarsnap like service but synced my data across multiple devices :( I can only really use it for backups...

If that's your reason for trusting an implementation in Go over C++ then I would say they should level up and use Haskell. Or even better, Agda. There's also ATS but I think the community around Haskell and Agda is more populous.

A garbage collected language is an improvement over possible memory leaks due to human responsible memory allocation, but I think a larger problem is most programmer's (very human) inability to separate pure from impure code. Languages with very strong type guarantees do it for you and give you the tools to protect yourself from a lot of problems that are common in any language from Assembly all the way through to Go (I think Rust is actually well suited as well since it has a better type system than Go, but I haven't used the language so I can't speak to that).

[EDIT] minor edits.

It's pretty easy to understand and it's not a slippery slope; what is slippery is bringing in tools and libraries that have nothing to do with Javascript.

Also, this whole argument has gone on far more than I intended - simply pointing out that the title was misleading and the reason why I thought it was misleading has suddenly turned into a holy jihad (proof given by the downvoters) to defend MongoDB's inclusion in the list (which is sad because MongoDB isn't a great DB either but that's not part of this argument).

I hold on to the fact, though, that it was indeed misleading and my desire for clarity is important to me and meaningful writing; when people criticize my writing for the smallest or the largest of reasons I want to hear it. Misleading title? Okay, why and what would you suggest?

A point worth considering, the spirit of what I said though should be considered - not specifically the details.

The spirit being, that someone WILL fill this gap in the market. Someone. I would rather it be Tarsnap and Colin; though.

To provide some scale reference: my previous company an SEO metrics company had customers paying $1,500.00 per month or more just to know where their customer's URL's were in Google search results.

If there are people willing to pay that much for that, what and who do you think exists out there that would pay 5x that amount to make sure a $25Million class action lawsuit is launched against them? Or to protect hundreds of millions in assets?

If the assets I'm backing up are worth millions to me (tangibly), spending $20,000.00 per month for highly secure and reliable backups is easy.

I would love to but I'm building a different company focused on helping people optimize their home energy - I plan to do it (get rich) on that one.

You also mis-characterized what I said. Completely. But whatever, pretty typical Hacker News. It's actually really an interesting dichotomy: there are so many brilliant hackers on here that believe asking for a lot of money for a valuable product and service is evil and yet they're all participating on a forum built and hosted by an organization that specializes in smacking hackers with a fish until they realize the deep and fundamental mistake in that way of thinking.

My argument was that many different kinds of good can be had from the availability of resources and casting resource acquisition in a stigmatic light is cutting innovation off at the knees. All because some less enlightened people figured out how to acquire a lot of resources and use them in non-society friendly ways.

Do you really believe Elon Musk could have turned Tesla into such an awesome company by being Geek2Geek? Fuck no. Elon understands the value of charging for value and how much more value he can usher into the world - so do other amazing entrepreneurs.

I would rather see cperciva acquiring a lot of resources than many other people in the world. I would rather see a lot of people on Hacker News be wealthy people instead the alternatives out there - but they never will be until money and the having of it is no longer stigmatized by them.

For many businesses, $1,500.00 per month to feel secure is pennies. I would rather see Tarsnap, an actually secure service, pulling in that revenue than other "secure" offerings that actually aren't.

Or increase your salary and use the surplus to donate. But really most of that profit should probably be directed to improving the business, or R&D that goes back into the business (which can be opensource!).

Yes, seriously. So by your logic we should be listing the browsers, the operating systems, the compilers, the linkers.

No. The title is misleading, I clicked on it expecting a good breakdown of the current can't-do-without javascript libraries for web application programming.

MongoDB? PostgreSQL? Why don't we mention Python too since not every Javascript programmer is using NodeJS, oh wait - maybe we should start mentioning Haskell to JS compilers too, yes, because those are Javascript tools and libraries.

All of those things are interesting but the title mislead me and I made note of it.

Get off my back.

No. You cannot.

Do I donate money to certain projects of mine? Sure. When did I donate the most money? When I billed a private equity firm $300.00 per hour for development work and had a surplus of $15,000.00 dollars. I gave quite a bit to the FSF and others.

Do you really think that by charging the PE firm $50.00 per hour I would be enabling them to donate to the FSF? FreeBSD Foundation? Haskell? GNU? Do you even think they would or care? I certainly wouldn't have enough money to do so at that rate, but I did at $300.00 per hour.

Highly naive my friend. I'm sorry to be offensive but I'm sick of my intelligent peers (sometimes even more intelligent) squandering what is, a massive opportunity sitting in front of them that is being ignored for a totally ridiculous noble cause when their "noble causes" could be enhanced so much more without an iota of evil behavior.

Tarsnap isn't going to become an big evil corporation by charging more. It might if he decides to sell it, but I doubt that. It would be enabling him to do even more amazing things for software, geeks, the world, whatever. Those amazing things could be donations to enable OTHER brilliant people or it could be a new business, or new features that make backups a better experience, or it could be starting an R&D lab for crypto that pumps out purely Open Source research on cryptography that makes its way back into Tarsnap or different business.

I am not happy with it at all.

You know what he could do with the millions Tarsnap could make him? DONATE IT TO FREEBSD. How about that being Geek2Geek? How much more better off would the world be if Colin donated $500,000.00 of his surplus income from the business to the FreeBSD foundation? Or the Python foundation? Or THE FSF?

It's infuriating how blind many intelligent people are to how much better the world would be if they allowed more money. I don't care if that ends up being them buying a new house or donating it or building another amazing business or having the money to live while they write more amazing open source software.

What if by making more money Colin is able to start another business that does unknown and amazing things for cryptography! The possibilities are endless and makes me SO ANGRY that people with such amazing knowledge and skill that is so much more than mine are SQUANDERING IT BY THINKING THEY ARE DOING THE WORLD JUSTICE BY NOT CHARGING MORE MONEY!

It is very possible and I'm remarkably tempted to do it myself in my free-time.

I also believe that was the intent of Patrick posting this publicly - to stimulate Colin into realizing what people really fucking want and that someone may actually go build a better tarsnap using tarsnap if he doesn't.

I'm sick of Tarsnap's complete disregard for its users; charge more money, listen to your users, improve the product!

WITH ALL OF YOUR SURPLUS MONEY, INSTEAD OF BUYING A $2MILLION DOLLAR MANSION, DONATE IT TO THE FREEBSD FOUNDATION.

Holy shit! What could HE DO FOR FREEBSD with the personal surplus in income from properly implementing Tarsnap as a business!?!?! IT'S FUCKING MIND BOGGLING.

</rant>

Title is a bit misleading, this is a list of development tools and libraries with a focus on Javascript. There are quite a few non-js tools in there.

I think there's a fundamental misunderstanding from people that do not know how to program "in the large". Real programming is actually heavily reliant on human creativity and synthesis. Just like good visual design.

Programmers are always automating and we are succeeding at it. Many tasks and roles have been eliminated in the last ten years by better tools and services.

Designers don't really do that, they get better tools to better represent their vision from...programmers.

I think design and art is crucial and the people that do it well are critical. But not as critical as the people that support the entire ecosystem.

If I were to say who would rule anything, it would be mathematicians, in every domain of knowledge - even design - mathematics is the ultimate intellectual tool to express abstraction, cardinal to the act of automating and reducing.

Haskell is definitely not as "easy" as Python but I actually think it's more straight forward than Python is. There's more cognitive work up-front but the pay off is very big in terms of stability, maintainability, etc... in the long run. I actually think it costs less in amount of work to do something in Haskell than in Python (and Erlang, even).

When I have to drop into Python or Javascript I feel like I'm pulling out my MiniTonka toys vs. the actual dump truck out back by the barn. When I went from PHP to Python it felt like that too.

As it stands though, I write systems critical services in HS now and keep the web app stuff in Python. I have written two production web apps in Snap and I loved every minute of it.

Thanks for the input, I also corrected attribution of the script, thanks for pointing that out :)