Learnings from One Year of Building an Open Source Elixir Application
achariam.com
achariam.com
Thats much appreciated but I was kind of hoping you (the author) would go into more details about request time. Most people can (and should) do the above. However I would be more interested in what Elixer + CRUD is like particularly for TTFB. Like does the author do streaming (I don't necessarily mean websockets or comet)?
After all if the TTFB is really slow the CSS optimizations and what not matter little.
In traditional request per thread (or whatever is analagous) web framework paradigms the request is a single thread and often waits for the database to finish before moving on to display the page. I would imagine Elixir has a better answer at least for read only pages.
Not really telling you much, I know, but these are pretty standard numbers for Elixir + Phoenix apps.
If you want something like Devise from the Rails world, there is also Coherence. https://github.com/smpallen99/coherence
Auth is genuinely hard, and turnkey solutions often aren't enough.
Given the massive attack surface for a web application, it's absurd to think someone could (or should*) develop an entire auth framework from scratch for all their projects. Turnkey solutions, like Ruby's Devise, are a godsend. Even in situations were a custom flow was needed, it's saved both me and my clients hundreds of hours.
In addition, I benefit from the community around a turnkey solution. Think of all the years the software has been tested by in umpteen production environments. Think of all the people that have an eye on the code and report security flaws while you sleep. Your custom session management implementation will never have that benefit.
Also, why the hell are you building auth when you could be building app? Is that really such a critical experiential part of your app that you can't possibly rely on something turnkey and then move on to the features that actually matter?
Elixir's Phoenix framework has very good plugins -- like Guardian -- that give you 95% of the tools you might need for your own security solution and you can assemble it together in 15-60 minutes (this of course assumes your security model isn't wildly different than what the various plugins support). You have separate Plugs -- think of them as pipes of sort -- who handle cookie session management, another that handles JWT tokens, HTTP basic auth etc.
The only thing you must do is to assemble such pieces in a code module that basically serves as a pipeline with several Plugs (pipes) connected in the order you deem appropriate for the security model of your app. Any of these Plugs can modify or remove headers or tokens, can consult a 3rd party system (say, for single sign-on), can provide throttling, temporarily lock out an account that made too many invalid login attempts, can refuse a session due to non-whitelisted IP, etc. etc.
So no, you're not inventing cookie session management or anything there. You simply have control on which security mechanisms -- and when, and at which conditions -- happen on request.
And it possibly saved you from embarrassment as well.
The "cruft" from the Phoenix framework is absolutely minimal -- mandatory 4-6 files per project at the most. And they're quite small, too.
While implementing the ranking algorithm, which is very similar to the one mentioned in the article, I decided to run a periodic job every 60 seconds that updates the rank for each submission and stores it in the database so querying the ranked data is more efficient than recalculating the rank on every page request. Are you doing something similar or did you take a different approach?
Ranking all stories works well if the total number of submissions is a small number, but I imagine the approach is a little different for large sites like HN. Ranking all submissions periodically seems like a waste since people rarely view submissions beyond 10 pages. One approach is just to rank submissions from the past n days, where n depends on the average daily submission volume.
For the part that displays time since a submission was made, I implemented the HN model, where it displays only minutes, hours and days. Python code here : https://dpaste.de/5d1w
> Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something.
Once you start to scale, the bottleneck is rarely the application layer. For the typical crud web app it's likely to be the database.
Then you only have to recalculate when there is a submission or vote.
You seem to be downplaying the importance of an efficient app layer and I respectfully disagree.
1. Using well-optimized tech for the app sends you right at the DB scaling problems phase, so you never go through the "our app layer is too damn slow" phase -- which can kill a startup pretty damn quick (and history knows examples). So, it's a huge win in my eyes.
2. Most apps never even take off to the point where the DB is the bottleneck so a strong app layer is a godsend; I'd estimate 80% of the apps struggle with optimizing backends and caches and not DBs -- at least judging by my 15 years of career which, I admit, is anecdotal evidence and doesn't mean much.
3. Those apps that do scale that far are very grateful to their backend engineers who spared them all the "tune our Java / Ruby / PHP / Python VM's memory usage and GC behaviour parameters" dance (and that dance can last for weeks and weeks almost without sleep!).
For the lazy, here is the github link from this article:
Which means it's a poor user experience. Sorry for the trouble, I wonder if any folks have better ideas here?
Not sure that would work or is as helpful in this case. The community seems more friendly than most and the general goals of many are to learn something new, powerful and fun. If you build a community around those themes I think it will attract the right crowd.