hassy

Redis is still very much open source and free; the difference is that there will exist some amount of code under Commons Clause which will support its further development. Moreover, nobody is forcing anyone else to use any of that Commons Clause code.

AWS Lambda is basically CGI, but on a huge scale.

It's an amazing tool for some tasks, but there's a lot of hype around it too.

Only if you take everything literally.

To those down-voting me: Firstly, I would invite you to consult a good dictionary. Secondly, I'd invite you to undertake a moment's reflection, which I believe should make readily apparent the utter uselessness of a word that describes not a single person that ever lived.

Great message, but "self-made" never meant someone who became successful with no outside help. One is self-made if their fortune wasn't inherited, that's it.

Because Lambda/Cloud functions are meant for scripting tasks in your cloud environment, not for building full-fledged apps.

(Lambdas couldn't even be triggered with HTTP calls when AWS Lambda launched.)

Because you now either have to run those open-source alternatives yourself or be at the mercy of another vendor to provide those for your as a service, and personally my trust in pretty much any other vendor would be much lower than my trust in AWS by default.

Out of curiosity, what does your C program do? I don't imagine yours is a typical use case if you don't use any other services (not even S3?) and when you consider that you're limited to 300 seconds of running time and not more than 100 concurrent executions.

The power of Lambda comes from being able to use other AWS services. Lambdas on their own are not very interesting.

A true Lambda alternative, open-source or otherwise, would also have to offer alternatives to DynamoDB, S3, SNS/SQS etc. Without those you're basically reinventing CGI.

Depends on the application, but as one possible general solution: have microservices talk to each other directly when it makes sense to rather than communicating over RabbitMQ / central message bus out of laziness/convenience.

I'd argue the offending party is the library pulling in Request for making a few simple HTTP calls.

Request itself in my opinion is a great piece of work, it does pretty much anything one would want to do with an HTTP client.

I agree with pretty much everything you said - and thanks for sharing your perspective. Node.js is still very young and the enterprise ecosystem has a long way to go, but it's getting there (e.g. IBM's acquisition of StrongLoop, the EnterpriseJS initiative - https://enterprisejs.io) I've done a bit of work with Erlang, which is all about the 9s (though it doesn't have the enterprise ecosystem on par with Java or .NET around it) but for a lot of software, Erlang is an overkill, as would be formal verification for example. A lot of software is just fine being built with Node or other young tech trading stability for productivity and time-to-market.

I'd like to plug an open-source project that I've been working on here - https://artillery.io - as it relates to point (6) in the article. It's a load-generator, and while it's not microservice-specific per se, it's a lightweight tool that's quite easy to use and it lends itself well to being used frequently, which is great if you want to start doing load-testing early on during development (which is something I've started doing and had good results with). It has HTTP and WebSocket support out of the box, with AMQP coming in the next release (which is often employed as microservice glue).

> But we've reached a point where modern web applications are pulling in dozens of dependencies in both production and development toolchains, and increasingly those dependencies are themselves built on top of other dependencies.

Um, take ANY non-trivial application, written in any language, and then follow the chain of its dependencies all the way down to the kernel, and shriek in horror. JS is not unique here, the only unique part is perhaps how obvious npm makes it just how much stuff your code depends on.

You cannot mention the ban on Highland dress without mentioning Bonnie Prince Charlie and his attempt to overthrow the legitimate government specifically (which ended at Culloden, with several Scottish regiments fighting on the side of the government), and the vast divide between the Lowlands and the Highlands in culture, language, religion, economy and industry, and attitudes to Union with England in general. Scottish history is very complex.

Elixir is a great project, I'm saying this as someone who wrote Erlang full-time in 2007-2009. It's attracting a lot of attention to an amazing platform (I don't get to write much Erlang anymore, but it's still my favorite language by far).

Articles like this one though are doing it a disservice in my opinion. It's heavy on hype and subjective opinion, light on facts and outright misleading in a few places, for example:

- 1-1.5kb of memory per process is for a process that does nothing. It's less overhead than any other platform I know of, but I am not sure how that translates to: "Want to keep track of a large portfolio of stocks? Easy. Want to keep track of the whole market? No sweat."

- 9 9s uptime is a function of architecture, not language. OTP makes it easier to build resilient systems, but it's not a silver bullet and there's a tonne to learn in order to just use OTP effectively. Elixir is not going to give you anything for free.

"Massive parallelism also results in massive redundancy." - how exactly? My massively parallel application could be running on one machine. The author must have meant distribution which Erlang/OTP is indeed very good at.

"Want to coordinate across markets or geographies?"

You would certainly not want to distribute an Erlang cluster across two different data centers, much less geographies.

"Add more hardware, and Elixir's performance will increase almost linearly along with it."

No it won't - https://en.wikipedia.org/wiki/Amdahl%27s_law

"Moreover, whereas Erlang was built to run on telecom networks, Elixir is built to run websites, phone apps and web services. New and exciting stuff."

What? Elixir runs on top of BEAM, the exact same VM that Erlang runs on. Moreover, using Elixir in your typical 12Factor web-app makes very little sense - you wouldn't be benefitting from any of OTP and taking a hit on development time due to lack of libraries.

Nice read, even though I haven't written any Python in a long time. Memory management in modern OSes is very complicated and there isn't a simple answer to "how much memory is my process using exactly". [1] Memory growth / usage isn't usually something to worry about unless memory growth is constant under load and/or OOM killer kicks in.

Also consider that `heapy` will probably only report on objects created by your Python code, and not any memory taken up by native code in the interpreter itself or any shared libraries.

1. For example see:

- http://stackoverflow.com/questions/860878/tracking-actively-...

- https://mail.gnome.org/archives/gnome-list/1999-September/ms...

- http://bmaurer.blogspot.co.uk/2006/03/memory-usage-with-smap...

Well, the gist of it is that it's not that hard technically, but could be more difficult to implement organisationally. Production load-testing is something that needs to be agreed with various teams/people across the organisation (easier if you're a small startup), think ops, marketing, analytics etc.

The basic thing is to make sure you can separate real and synthetic requests (with a special header for example). This will allow you to mock/no-op certain operations like attempting to charge the user's card or reducing the quantity of stock you have. It'll also allow you to remove synthetic requests from your analytics data, so that marketing does not get excited by the sudden influx of new users. If you have user accounts on your system, make all fake users register with @somenonexistentdomain.com so you can filter for that too etc.

Obviously start slow and ramp up over time as you iron out issues.

JustEat.co.uk run daily load-tests in production at +20-25% of their peak traffic. As in: extra 20% simulated load during their peak hours, which happen to be between 6-9pm every day. They process a very respectable number of real-money transactions every second, a number that a lot of ecommerce sites would be very happy with. (Source: a presentation at ScaleSummit in London this year)

Feel free to @message me if you want to talk more about this.

Daily load tests against both staging and production (yes, really) can help catch a lot of issues of the kind described in the article. You do have to have a solid monitoring & alerting set up still though.

This comment doesn't even make sense.

Kong is a cool product, but Kong spam from a couple of accounts [1] on HN is getting tiresome. (Maybe it's just that I look through the comments of most stories to do with APIs.)

https://hn.algolia.com/?query=mashape%2Fkong&sort=byPopulari...

Patronage has existed for a long time.

https://en.wikipedia.org/wiki/Patronage#Arts

Thanks for taking the time to explain!

I understand that each { follower visits page } is dependent on the { Ashton Kutcher tweets } event, but I still don't understand why events are not independent of each other, and why the N visits to the webpage in time period T from the tweet would not likely be described by a Poisson process. Not expecting an answer btw, just thinking out loud here. :)

Correct me if I'm wrong, but surely in the scenario that you are Ashton Kutcher and tweet a link to your followers, the average inter-arrival time would decrease but overall it would still be a Poisson process?

In what sense does Erlang not handle clustering well? It's true that you wouldn't run a "native" (ie EPMD-based) cluster of hundreds of nodes, but what would stop you from implementing a clustering architecture you'd build with, say, J2EE in Erlang?

I don't seem to see a mention of the file descriptor limit. As recently as last year the default on Ubuntu was 2048 iirc, which is laughable given that it was 2014.

I guess it's not strictly performance related, but should definitely be one of the first parameters to tune.

    # in in /etc/sysctl.conf:
    fs.file-max = 100000

    # then:
    sudo sysctl -p

    # in /etc/security/limits.conf

    * soft nofile 100000
    * hard nofile 100000

    # then:
    ulimit -n 100000

(tweak the exact number as required)

Personal anecdote - <2% of visitors that play a preview ever click on a buy link on http://wonderwheel.fm

It's something I built for myself and I didn't expect to make anything, but it's still been entertaining to have the expectation confirmed.

The site has seen some okay traffic but hasn't made enough to buy me a half of Tennent's.

Increasing the number of file descriptors on machines running HTTP servers is one of the first recommendations I make to my consulting clients.

It's much easier to overlook than you'd probably imagine. I have seen apps serving hundreds of thousands of API requests per day that had the default settings. It's one of those quick changes that can have a big impact.

If you liked this you should read The Score Will Take Care Of Itself by Bill Walsh who lead San Francisco 49ers to win three Super Bowls.

Jack Dorsey recommended this book at the recent Startup School

http://www.amazon.com/The-Score-Takes-Care-Itself/dp/1591843...

http://www.youtube.com/watch?v=wEQawgkCMOU

£800 for a two bed on the Royal Mile?

I have some awful elisp code that glues together org mode, iimage.el and Deft to do something along these lines. All notes are stored as plain text with images in a sensible directory structure. I don't use git, everything just sits in a Dropbox folder. Emacs works well for me as the UI, but there's no reason a web-based UI couldn't be written. I could look into tidying it up over the Christmas break and publishing it if there's interest.

> I think what needs to be done after this is checking how much data can be sent per second. Both in terms of raw bandwidth and messages/sec.

These are on the list :)