CloudFlare's New Dashboard
blog.cloudflare.com
blog.cloudflare.com
The dashboard, while useful, always felt like it was put together rapidly to integrate a bunch of disparate parts of such a huge service like Cloudflare. Compared to the incredible service offerings, the dashboard was not one of the best parts of Cloudflare. Some data points were a bit off, some charts didn't work quite right, and you had to hop in and out of various parts of Cloudflare to get things done. Some stuff in settings, over there, some stuff in the charts over there, some stats here, but not there.
Now, the dashboard is just as amazing as the service. Kudos to the overall implementation of the information architecture and user desires for the service. What a tremendous upgrade to an already valuable offering. All of the important settings and information are now within arm's reach and easy to find. The new charts, stats, and graphs are absolutely amazing and insightful without being frivolous. There's much more detail and new data to work with. The responsiveness of the dashboard itself has also increased a huge amount, which I'm sure was no small technical feat.
Very well done to all involved. I can feel even more comfortable continuing to recommend Cloudflare. Excited to read the followups on how they managed to create such a different and successful overhaul to the main window into their service.
The second time I tried Cloudflare was a few months ago for a personal project. I run a moderately popular vBulletin forum in my spare time and I wanted to see if CF could speed up my site. The results were similar--Cloudflare actually slowed things down. I was at the free tier and perhaps I would have seen improvement at the $200/mo plan with Railgun but I wasn't going to waste any more time to find out.
The nail in the coffin was when I discovered that they provided CDN services to ISIS propaganda sites and openly oppose any efforts to shut down the sites. I'm all for freedom of expression on the internet and it's certainly their right to support the distribution of jihadi videos featuring the beheading innocent aid workers but I'm not going to support it with my money.
I too originally tried CloudFlare and left, tried again and left again, before eventually trying it about 18 months ago and staying.
I too run highly dynamic sites, the same as you a few were vBulletin forums and I also had a Django web app that has a login form on every page. In both cases the pages are not able to be cached. The vBulletin page has personalisation or CSRF tokens (login form) depending on your login state, and the Django web app CSRF tokens again act to prevent caching there.
It is obvious to me now, as it is to you, that any asset (page in this case) that isn't cacheable is only resulting in an additional hop and the additional delay that introduces.
The reason I eventually remained a customer is Railgun.
Railgun is the missing link that allows dynamic content to be mostly cached with only the diff between requests passing over the wire between origin and CloudFlare. Dynamic content still isn't cacheable from the perspective of the origin or the end user, but Railgun doesn't send 99% of the page over the wire, just the diff (this is a simplistic explanation but works well enough for this).
Railgun is the only thing that I personally felt took my highly dynamic sites and made them feel dramatically faster than going direct to the origin. It also works really well for APIs.
Additionally, for vBulletin I'd recommend setting up a page rule so that your attachment.php file is aggressively cached. Especially if your vBulletin is configured to store attachments in the database as this will spare your database server.
Of course, if you're running WordPress, Drupal, or a CMS designed to produce cacheable sites then Railgun isn't needed. CloudFlare does act like a silver bullet in those cases and by default the caching is very good and the speed improvement the end user experiences is dramatic. But for dynamic sites, Railgun and Page Rules help to get the best out of the service. Hmm, it may be worth us producing Knowledge Base articles for "gold configurations" for common web applications, or better still to provide some way for us to let you just tell us how you're using something like vBulletin so that we can configure some things for you and recommend the other things (Railgun) that will make a difference.
On ISIS sites, it's likely true that amongst the millions of sites there are some that are probably related to terrorism, or to carders, or whatever else is unwelcome. We have a trust and safety team and of course they work with law enforcement and courts in all jurisdictions in which we operate. Our service is to protect customers from DDoS and other attacks, so to some extent yes we are a shield for our customers as we do protect origin servers. However any valid legal request from a court or law enforcement agency is respected. That's the way to have something removed, not a subjective policy applied in a non-transparent way by a private company.
The CF servers need to manage a separate chunk of memory for the customer cache and this is outside of the shared caches that can be used for assets. It also requires you to install something on your server (the vast majority of CF features don't require this), and that introduces additional support costs.
Wherever there is a real cost to supply a service, you usually see those things in the more expensive plans.
Where it's possible to offer something at negligible cost, you usually see those things in the cheaper plans.
And yes, if something has virtually no cost to offer then it's a benefit to the security and performance of the web in general if we can push those things into the free plans and give them to everyone.
There are exceptions to this of course, Universal SSL is an example of a feature that has a cost but the benefit to customers and the web in general is so significant that we give that on the free plans.
Thanks for the response
Most CDNs are used to cache static files (javascript, images, etc) so they download faster but the webpage still loads from the origin server. CloudFlare is different and proxies everything through their servers (that's the only way they can provide the protection and caching benefits). If your origin is close to your location, then it WILL increase latency because a request now has to go through CloudFlare's network, to the origin, then back through CloudFlare and then back to you. It's an extra jump that will add some latency to all page requests that have to go to the origin.
It does help for far away users though because CloudFlare's private network and proxy servers can contact your origin server faster than the end-user making it all the way on their own connection. In either case - near or far - the latency should be minimal and definitely not something that adds seconds to load time.
If your pages don't change much, you can also just cache the HTML itself on the proxy servers which will definitely decrease load time to pretty much instantaneous. POST requests will still go back to the origin through CF's network but the GETs should be fast and the higher-end plans give pretty precise control over how long the caching lasts. (You can also do this with modern CDNs with more control but they don't provide the security benefits and the other apps that are part of CF).
EDIT: By the way, Hacker News uses CloudFlare.
Although analytics is not the reason we use Cloudflare, it's interesting to note that in their new analytics, the primary metric for measuring traffic has changed from page views to requests. Since Cloudflare is the source of serving the pages, it seems like it would have the greatest opportunity for capturing accurate pageview counts. I wonder what the story is behind the change.
Client-side stuff is unreliable and is open to everything from javascript blockers to network connections in the middle. CloudFlare even had a post about how their tracking was better (cant find it now).
EDIT: Yes I know newer ajax sites are completely different and there's probably no perfectly accurate way to track them unless the devs setup analytics properly, and even then its up to the same issues all client-side tracking faces.
If anyone has a question, feel free to ask me here :)
We're going to be publishing more engineering-related blog posts about the new dashboard in the coming days/weeks.
Right now the page rules system seems like an added feature but I think it should be the heart of the settings system. When a new site is created, it would be logical to just create a default page rule that covers everything like "example.com/*"
Each rule can have a summary of settings as they do now and all the other separate pages can go away. Speed, Caching and ScrapeShield can just be settings for each rule instead of full pages. It'd be a lot faster to manage settings this way and more natural to view a site's settings as a cascading set of rules with their own configuration.
Likewise, Traffic can probably just be made another tab under Firewall.
Hope that makes sense.
(i.e. this is the 1.0 release...we have a LOT of stuff planned for the next months in various areas)
Since there's not a lot of images of the full dashboard page in the article, I made two quick additional screenshots to compare the two. [1]
I think part of it is just that I'm not used to the new one yet - it takes time to find options which I could previously find more easily - but at the same time, if I want to put a bunch of sites into development mode to name one example, I now need to individually go into each specific domain's page, click the "caching" nav icon, and change it there. I feel like it essentially takes a single page two-click mechanism and changes it to a three-click mechanism across multiple pages that forces me to think about where I'm going much more than previously.
There's also a full API, so you can actually automate a lot of bulk actions (probably not so useful to end users, but if you have a lot of sites it might be worth it, and there could easily be tools.)
Also, the responsiveness of the development team to feedback has been very positive; every issue I've raised was responded too quickly and clearly (even pedantic niggles[0]).
[0] But seriously, what algorithm is the domain list sorted with? It's a total mystery, please fix it.
As a long time user of the "new" dashboard I would love to see a way to version control my cloud flare configuration for a domain. The team and I are always scared of pushing the wrong button due to the lack of confirmation buttons.
One route is to do what CloudFlare has done: create the "pro" UI where you have every setting imaginable across multiple namespaces.
Another is to simply make an API for everything and let your customers implement their own UI (usually integrating your product with the 15 other vendors they probably use).
I'm increasingly in favor of the second approach. A pro-user UI is great, but most of your customers who are complex enough to need a truly pro interface would be better served by an API. That was honestly the revolution that Amazon brought to the infrastructure space: they had an API you could use to provision... Well, anything.
So IMO the API is the new expert mode. There are just too many platforms out there (AWS, CloudFlare, Akamai, etc) that are often used in tandem that a vendor-provided interface will never be enough for the most advanced users.
Going API-only for the advanced features might be okay for something like AWS, but it will alienate a lot of users for the vast majority of online services out there.
CloudFlare is not just for highly experienced programmers who can whip up a Ruby script to do whatever they want. A large percentage of their customers are probably just average "webmasters" who are using CloudFlare to improve security and page load times for their WordPress sites and whatnot. They're comfortable with clicking around the website to get the result they want, but if you just gave them an API and some npm packages to access it, it would take them several hours to get the same thing done. They might do better the next time, but if that initial friction hurts CloudFlare's conversion rate, why bother?
We've been seeing the same problem for several years already with *nix desktop environments and applications. Some rockstar programmers and designers collectively decided that they'll optimize the graphical UI for newbies and hide all advanced features in a bunch of manually editable config files, command-line flags, or (heaven forbid) the source code itself. Result: newbies are happy because the UI is simple and streamlined. Advanced users are happy because they can still do what they want to do. But a lot of semi-advanced, I'm-still-learning users in the middle of the bell curve are pissed off because they see no reason why they shouldn't be able to use a mouse to get work done. And since we're talking about a bell curve, that's a lot of pissed-off users.
Many people here on HN have warm fuzzy feelings toward the concept of mashing keyboards all day long. The rest of the business world, on the other hand, is still very attached to the mouse. They're like, "Duh, why would you write <foo bar="yes" /> when you can just click on 'Yes'?"
But seriously though, nice work on the reorg, and the inline help and responsive on mobile rocks. Well done!