DivineTraube

The updated version gives me this (after successful setup with the example alien thing):

UnfilteredStackTrace Traceback (most recent call last) <ipython-input-2-0e20e3adf861> in <module>() 2 ----> 3 image = generate_image_from_text("alien life", seed=7) 4 display(image)

67 frames UnfilteredStackTrace: TypeError: lax.dynamic_update_slice requires arguments to have the same dtypes, got float16, float32.

The stack trace below excludes JAX-internal frames. The preceding is the original exception that occurred, unmodified.

--------------------

The above exception was the direct cause of the following exception:

TypeError Traceback (most recent call last) /content/min-dalle/min_dalle/models/dalle_bart_decoder_flax.py in __call__(self, decoder_state, keys_state, values_state, attention_mask, state_index) 38 keys_state, 39 self.k_proj(decoder_state).reshape(shape_split), ---> 40 state_index 41 ) 42 values_state = lax.dynamic_update_slice(

TypeError: lax.dynamic_update_slice requires arguments to have the same dtypes, got float16, float32.

With CDNs, there are definitely privacy (and GDPR) concerns, since often the complete traffic passes through the CDN. And in order to cache, rewrite and optimize responses, SSL connections are typically terminated in the CDN. And this implies that the CDN provider processes and potentially sees sensitive user data in plain text. Due to the distributed nature of CDNs with many edge locations, one can never be really sure what the exact legal circumstances are when user data is passed through the CDN nodes in different countries. And even though CDN providers won't leak user data on purpose, cloud bleed [1] has shown that this can happen by accident and at massive scale.

This is one of the reasons, why at Baqend we opted for a different approach: by using Service Workers, we can make sure that as a service provider we will only see public data. Personally identifiable information like cookies and sensitive session information does not leave the user's browser. And if you think about it that makes a lot of sense, since the public data really is what makes a site fast or slow. User-specific data (e.g., a profile, payment data, etc.) is hardly ever the cause of performance problems. Nonetheless, we do use CDNs for public data as they are an indispensable tool to achieve low latency.

[1] https://en.wikipedia.org/wiki/Cloudbleed

"What if you could have a fully managed database service that's consistent, scales horizontally across data centers and speaks SQL?"

Looks like Google forgot to mention one central requirement: latency.

This is a hosted version of Spanner and F1. Since both systems are published, we know a lot about their trade-offs:

Spanner (see OSDI'12 and TODS'13 papers) evolved from the observation that Megastore guarantees - though useful - come at performance penalty that is prohibitive for some applications. Spanner is a multi-version database system that unlike Megastore (the system behind the Google Cloud Datastore) provides general-purpose transactions. The authors argue: We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. Spanner automatically groups data into partitions (tablets) that are synchronously replicated across sites via Paxos and stored in Colossus, the successor of the Google File System (GFS). Transactions in Spanner are based on two-phase locking (2PL) and two-phase commits (2PC) executed over the leaders for each partition involved in the transaction. In order for transactions to be serialized according to their global commit times, Spanner introduces TrueTime, an API for high precision timestamps with uncertainty bounds based on atomic clocks and GPS. Each transaction is assigned a commit timestamp from TrueTime and using the uncertainty bounds, the leader can wait until the transaction is guaranteed to be visible at all sites before releasing locks. This also enables efficient read-only transactions that can read a consistent snapshot for a certain timestamp across all data centers without any locking.

F1 (see VLDB'13 paper) builds on Spanner to support SQL-based access for Google's advertising business. To this end, F1 introduces a hierarchical schema based on Protobuf, a rich data encoding format similar to Avro and Thrift. To support both OLTP and OLAP queries, it uses Spanner's abstractions to provide consistent indexing. A lazy protocol for schema changes allows non-blocking schema evolution. Besides pessimistic Spanner transactions, F1 supports optimistic transactions. Each row bears a version timestamp that used at commit time to perform a short-lived pessimistic transaction to validate a transaction's read set. Optimistic transactions in F1 suffer from the abort rate problem of optimistic concurrency control, as the read phase is latency-bound and the commit requires slow, distributed Spanner transactions, increasing the vulnerability window for potential conflicts.

While Spanner and F1 are highly influential system designs, they do come at a cost Google does not tell in its marketing: high latency. Consistent geo-replication is expensive even for single operations. Both optimistic and pessimistic transactions even increase these latencies.

It will be very interesting to see first benchmarks. My guess is that operation latencies will be in the order of 80-120ms and therefore much slower than what can be achieved on database clusters distributed only over local replicas.

> So here are some facts and trivia that are not so well-known or published that I collected by talking to Parse engineers that now work at Facebook. As I am unsure about whether they were allowed to share this information, I will not mention them by name.

It's both lessons learned by Parse engineers and us, so I think the intended ambiguity is okay.

I do believe you, but can your share some more hard facts?

I've changed the passage to "one of the greatest". As in the disclaimer above the article, I'm mainly presenting assertions by Parse engineers, who were very sure about being the largest MongoDB user in terms of cluster size.

Sure, the benchmark and the application under test are public: http://benchmark.baqend.com/

We also do have some benchmarks comparing real-time performance, which are under submission at a scientific conference, so we cannot share them yet.

Unfortunately, testing against Firebase using established benchmarks such as YCSB is not very useful, as it's unclear which limits are enforced on purpose and which limits are inherent to the database design. Would you be open to providing a Firebase setup for heavy load testing w.r.t. to throughput, latency and consistency?

The Google Datastore is built on Megastore. Megastore's data model is based on entity groups, that represent fine-grained, application-defined partitions (e.g. a user's message inbox). Transactions are supported per co-located entity group, each of which is mapped to a single row in BigTable that offers row-level atomicity. Transactions spanning multiple entity groups are not encouraged, as they require expensive two-phase commits. Megastore uses synchronous wide area replication. The replication protocol is based on Paxos consensus over positions in a shared write-ahead log.

The reason for the Datastore only allowing very limited queries is that they seek to target each query to an entity group in order to be efficient. Queries using the entity group are fast, auto-indexed and consistent. Global indexes, on the other hand, are explicitly defined and only eventually consistent (similar to DynamoDB). Any query on unindexed properties simply returns empty results and each query can only have one inequality condition [1].

[1] https://cloud.google.com/datastore/docs/concepts/queries#ine...

Could you share what your lessons learned were? That would be a very interesting perspective. Do you disagree with some of the points?

Do you have a source for that? It is what the Parse engineers told me, but that kind of information is notoriously hard to corroborate. Any time I asked this question to MongoDB engineers they were evading concrete numbers for deployment sizes.

Author of the post here. If you have additional key insights about the Parse infrastructure, please post them here and I will directly add them to the article.

> Some problems are caused by DBMS to backend and backend-to-frontend network utilization, which is generally not something a specialized DBMS can solve.

I absolutely agree. RDBMSs are the kings of one-size-fits-most. In the NoSQL ecosystem, MongoDB starts growing into this role. Of course, many performance problems are not caused by the backend but rather the network (an average website makes 108 HTTP requests) or the frontend (Angular2 without AOT, I'm looking at you). We took an end-to-end approach to tackle this problem:

1. As objects, files and queries are fetched via a REST API, we can make them cachable at the HTTP level to transparently use browser caches, forward proxies, ISP interception proxies, CDNs and reverse proxies.

2. In order to guarantee cache coherence, we invalidate queries and objects in our CDNs and reverse proxy caches when the change using an Apache Storm-based real-time query matching pipeline.

3. To keep browser up-to-date we ship a Bloom filter to the client indicating any data that has a TTL that is non-expired but was recently written. This happens inside the SDK. Optionally client can subscribe to queries via websockets (as in Firebase or Meteor).

The above scheme is the core of Baqend and quite effective in optimizing end-to-end web performance. Coupled with HTTP/2 this gives us a performance edge of >10x compared to Firebase and others. The central lesson learned for us is that it's not enough to focus on optimizing one thing (e.g. database performance) when the actual user-perceived problems arise at a different level of the stack.

The core problem was that the sustained number of requests did not really cause any bottlenecks. Actual problems were:

- At the Parse side: expensive database queries on the shared cluster that could not be optimized since developers had no control over indexing and sharding.

- At the customer side: any load peaks (e.g. a front page story on hacker news) caused the Parse API to run into rate limits and drop your traffic.

You're right of course, I just changed that.

There are definitely some similarities. Like in Gomix, the idea of Baqend is to let developers easily build websites, bots and APIs. However, Baqend has a stronger focus on how data is stored and queried, while Gomix is more focused on a rapid prototyping experience inside the browser.

In my opinion, the whole movement about BaaS is all about making things as smooth as possible for developers and shorten the time-to-market to its minimum. What some providers like Parse lost on the way, are the capabilities for small prototypes to grow to large systems. That requires being scalable at both the level of API servers, user-submitted backend code and the database system. And at some point, it also requires letting the user tune things like indices, shard keys and execution timeouts. This is the route we took at Baqend. We do not want to be the simplest solution out there, but we aim to provide the fastest (we use new web caching tricks) and most scalable one (we expose how the database is distributed).

The database was not the bottleneck for Parse, how they used it was. Actually that is the case for most databases systems, you have to choose a set of trade-offs your application can live with. We wrote about this in more detail here: https://medium.baqend.com/nosql-databases-a-survey-and-decis...

Of course, it's a little unfair to declare it failed. The exit definitely was impressive. However, the product still died. It's hard to tell without the real business metrics whether this was due to lack of sustainable revenue and growth or just missing alignment with FB's overall strategy.

From a standpoint of traction it was no failure. A lot of developers liked it for a good reason. However, the pricing model and the lack of performance guarantees lead Parse to being used for prototype stuff in the free-tier mainly. I think, if they had a little more trust in the capabilities of developers to know what they are doing, the service might still be alive.

That is definitely true for some of the older versions of Mongo. Still, a quorum write concern will yield far fewer consistency violations. An interesting trade-off to take: losing a little data is better than losing a little latency.

A little Parse trivia a gathered from talking to (very capable) Parse engineers that now work at FB:

- Still the world's largest MongoDB user

- Had 1M apps, largest one with 40M users

- Server was Rails at first (24 threads max. concurrency), later rewritten in Go

- >40 MongoDB Replica Sets with 3 nodes each. Storage Engine: RockDB (MongoRocks). No sharding (DB-to-replica-set-mapping). Only instance storage SSDs, no EBS.

- Write Concern 1 (!) - some people complained about lost data and stale reads (slave reads were allowed for performance reasons)

- Partial updates were problematic as small updates to large docs get "write amplification" when being written to oplog

- Experienced frequent (daily) master reelections on AWS EC2. Rollback files were discarded -> data loss

- Special flashback tool that recorded workloads that could be rerun for load and functional testing

- JS ran in forked V8 engine to enforce 15s execution limit

- No sharding automation: manual, error-prone process for largest customers

- Indexing not exposed: automatic rule-based generation from slow query logs. Did not work well for larger apps.

- Slow queries killed by cron job that polled Mongos currentOp and maintained a limit per API-key + query combination

- It was planned to migrate Parse to FB's infrastructure but the project was abandoned

- Clash of clans used Parse for push notifications and made up roughly half of all pushes

I find this extremely interesting, as we are building a BaaS, too, but have a very different approach (Baqend). Coming from a database background, our idea is that developers should know about details such as schemas and indexes (the Parse engineers strongly agreed in hindsight). Also we think that BaaS is not limited to mobile but very useful for the web.

Also I think that providers should be open about their infrastructure and trade-offs, which Parse only was after it had already failed.

Thanks a lot, it's fixed now.

We fixed that problem and deployed it. It turned out to be the iframe with the tutorial app. That sets a new hash with a random todo list id and that captures the back button. It was not intended to keep people on our site.

Thanks for pointing the issue out!

BTW, I agree, RDBMSs are a great choice if the hot data set fits in main memory and queries are well-optimized through appropriate indexes. Issues usually only start to occur if 1) the data set is too large 2) queries cannot be optimized by the query optimizer and involve full-table scans 3) interleaved transactions contend on few rows.

> It is true that with Shopify or similar platforms you aren't able to make backend choices or implement caching like the article has described, and yes it's true that all the themes in the theme store aren't perfectly optimized for page load time -- but the question really is do you want to do any of this stuff yourself?

You're right, that is surely a thing you do not want to do yourself. This is why Baqend does all the browser and CDN caching, TTL estimation, query invalidations and so on automatically.

In terms of the frontend if you do not want to rely on a stock theme - and many larger ecommerce site won't - then you will have to do these optimizations yourself. Fortunately with good tooling available as outlined in the article.

I think Shopify is in fact a great improvement over systems like Magento. But if Shopify is well-enginneered (which I'm convinced of), how do you handle sudden load spikes in the database? Do you reshard and migrate data in real-time? How does Shopify scale the application tier? How can tenants run scalable custom backend code? What happens if a database or server node crashes?

The advantage of Baqend is that its inner workings are published in research papers and blog posts, so people can judge the performance and scalability not only through benchmarks and real-world examples like Thinks but also by looking at the design.

In contrast to Shopify, Baqend is backend-as-a-service which means customers have full freedom on how they design their site or shop. They do not have to live with a server-generated HTML that they cannot change - they can for exmaple easily build a React/Angular single page applications, run server-side JavaScript code, do custom queries, build a native app around the same business logic and APIs, subscribe to websocket-based change streams etc.

While I respect that it took you 10 years to build Shopify, this is usually not an indicator for scalability and a low-latency design. Often such projects end up with solutions like MySQL sharded at application-level, intermingled technology stacks and legacy systems.

Disclaimer: I'm the Baqend CEO and its caching approach is my PhD thesis.

How about Googles Doubleclick (2016): 53% of mobile site visits are abandoned if pages take longer than 3 seconds to load [1].

Mobify (2016): For every 100ms decrease in homepage load speed, Mobify's customer base saw a 1.11% lift in session based conversion, amounting to an average annual revenue increase of $376,789. Similarly, for every 100ms decrease in checkout page load speed, Mobify's customers saw a 1.55% life in session based conversion, amounting to an average annual revenue increase of $526,147 [2].

Staples (2014): conversion increase per second [3].

Or the Obama campaign (2012): making the page 60% faster increased donations by 14% [4].

More new studies from recent years can be found at WPO stats [5].

[1] https://www.doubleclickbygoogle.com/articles/mobile-speed-ma...

[2] http://resources.mobify.com/2016-Q2-mobile-insights-benchmar...

[3] http://de.slideshare.net/cliffcrocker/velocity-ny-how-to-mea...

[4] http://kylerush.net/blog/meet-the-obama-campaigns-250-millio...

[5] https://wpostats.com/

We built Baqend around the caching algorithms that we developed at the university. Our goal was to make the performance optimizations applicable to as many contexts a possible. We went for the "serverless" pardigm and backend-as-a-service to accelerate any application (website or app) that uses our APIs and hosting.

Currently most of our customers are e-commerce, web and app agencies, startups and software companies that look for web performance in combination with scalability.

We're currently getting lots of request from e-commerce, so we are building out our support for shop features (e.g. advanced search, payment, content management). We are developing a Magento plugin to make the transition to our platform and its performance benefits as easy as possible.

That should be done a lot more than it's usual today. Part of "mobile first" should be testing mobile speed.

One problem is of course that mobile networks vary greatly in both latency and bandwidth. A great resource on performance considerations for mobile networks is Ilya Grigoriks open book [1].

[1] https://hpbn.co/mobile-networks/

Sure. There is an abundance of studies. My favourites are [1]:

-100ms of additional page load time cost 1% revenue (Amazon)

-400ms means -9% vistors (Yahoo)

-Showing 30 search results instead of 10 and therefore accepting 500ms of additional page load time means traffic goes back by 20% (Google)

-Every second of page load time reduces conversions by 7%

There are many more. There is a nice infographic summarizing a few more studies in a nice way [2].

In fact, I think the trend is quite the opposite: as devices and connections get better, people start to expect that websites are very responsive and load fast.

[1] http://www.baqend.com/product.html

[2] http://infographicjournal.com/wp-content/uploads/2016/02/How...

You're absolutely right, latency in the checkout process is a conversion killer. Somehow Paypal currently seems broken, maybe it's still related to the Dyn DDOS attacks which took Paypal down completely.

During our testing, Paypal had latencies of ~100-500ms but these new latency outiers of 10 seconds are a no-go. We will definitely change the code to make that API call asnychronous. Generally Paypal Plus is still quite buggy and not well documented.

All major browsers do respect the TTL if one is set. So it does save many requests. If it's not possible to the the TTL because the content might change at any time it is still useful to a caching header with a TTL of 0. Otherwise the browser will perform heuristic caching and compute a (often imprecise) estimation based on the last-modified header.

Unfortunately on the server that method executes a synchronous API call to Paypal Plus, which has very flaky response times.

You're right that browser caching has a few tricky parts, which is why we do a some things differently.

What you describe is an HTTP revalidation (If-None-Match, If-Modified-Since) which the browser triggers in two cases:

-if the TTL defined in the Cache Control header has expired

-or if the user hits F5

In the normal case where a user revisits a site, the browser takes all the cached resources without doing a revalidation and that is the case one should optimize for.

The cases you seemed to have in mind is explicit refreshes (F5), expired cache entries, or servers that respond with a max-age=0 header (which is useful in many situations).

The most difficult part when caching dynamic data is purging stale content from browser caches. Because if the TTL is too high and the content changes ealier, users will see stale cached data. At Baqend we solve this problem by piggybacking a Bloom filter of potentially stale URLs to the client [1]. The client checks the Bloom filter to determine whether to do a revalidation or use the cache. In order for the Bloom filter to remain small, we have a learning TTL Estimator that calculates TTLs by approximating the expected time to the next write.

[1] https://medium.baqend.com/announcing-availability-of-baqends...

You're pointing out an important point: third-party services have to be fast, too. The call that starts the checkout does a synchronous request to Paypal Plus, which has very ugly request latencies from time to time.

The shop is indeed quite minimal at this time. The optimizations scale very well to regular ecommerce sites, though. Detail pages and categories can be cached very well, as they only change by actions of human editors. A notable exception are recommendations. If they are item-based (e.g. a-priori algorithm) they can also be cached very well. If they are user-based (e.g. collaborative filtering) that part has to be fetched from the server-side recommendation model. The same is true for search results. The heavy-hitters, i.e. the most-searched-for keywords are cached, while the others are fetched from the database. That is the part where server-side scalability and latency optimization is of utmost importance. At Baqend we support both MongoDB full text indexes and automatic separate indexing in an ElasticSearch cluster with automatic CDN and browser caching for the most frequent search results.