Nican

Oh! Actors is one of my favorite paradigms of lately. I have been reading plenty about Akka as well.

When I start looking into the subject further, I start falling into a rabbit hole of how Actors are sometimes treated as a database, that is hard to query, and provides no transaction-ability across actors. There is some argument that Actors could be reimplemented using Postgres' "SELECT ... FOR UPDATE", since you can lock the row for the duration of the transaction.

Anyone else have experience managing large amounts of data inside of Actors have a say about this?

My personal guess: A queue calls into an external 3rd party service, but the external service is not transactional, so it can receive and store the request, but before it finishes executing, it throws an exception and returns an error code.

The queue keeps retrying into the external service. While the external service keeps printing cards.

That was my initial guess as well, but I can not imagine where in the pipeline that would occur.

A customer might request more than 1 card, but it does not explain the wildly different expiration dates, and different delivery dates.

Albeit, now it makes me wonder how much of the functionality of dolt is possible to be replicated with CockroachDB. The internal data structures of both databases are mostly similar.

You can have infinite point-time-querying by setting the TTL of data to forever.

You have the ability to do distributed BACKUP/IMPORT of data (Which is mostly a file copy, and also includes historical data)

A transaction would be the equivalent of a commit, but I do not think there is a way to list out all transactions on CRDB, so that would have to be done separately.

And gain other benefits, such as distributed querying, and high availability.

I just find interesting that both databases (CockroachDB and Dolt) share the same principal of immutable B-Trees.

I love the idea of this project so much. Being able to more easily share comprehensible data is an interest of mine.

It is not the first time I have seen immutable B-trees being used as a method for being able to query a dataset on a different point in time.

Spanner (and its derivatives) uses a similar technique to ensure backup consistency. Solutions such as CockroachDB also allows you to query data in the past [1], and then uses a garbage collector to delete older unused data. The Time-to-live of history data is configurable.

[1] https://www.cockroachlabs.com/docs/stable/as-of-system-time....

Thank you for taking your time to write that up. To illustrate my point a little more:

Suppose a customer has several resources, and each resource has several metrics. From my understanding, Materialized could be used to have an aggregated view of metrics per customer.

The problem is that resources can also be migrated between customers. When a resource migrates between customers, the whole history of the customer changes. This could cause huge updates depended on how many resources are moved, or how many metrics per resource are being collected.

I have a conundrum between doing the "customer-resource join" late, and causing huge CPU cost when running queries. Or making aggregates early, and then having huge Disk cost when migrating resources. At the moment, we just have daily jobs that aggregates the TBs of customer data daily, because there is no way to do the joins in real-time.

Is Materialize designed to be able to handle something like this?

Materialize is a really interesting solution, and I love for what it stands.

But the documentation is missing more details about the architecture overview. A single update could cause many gigabytes of data to shift on the materialize view, and I do not understand how Materialize would handle that scale.

Why was there not an upper limit on the price that customers are willing to pay?

This seems to fall in the same category of AWS customers accidentally having bills enough to bankruptcy.

It is not the first time that I have looked at IPFS and I still have a hard time understanding how the ecosystem is going to work.

From my understanding, the file hash is basically the file URL, such that any change to the file content is a change to the file url as well. For hosting something like Wikipedia, how would one create pages that link to one another? And if indexes need to be created on top of the content, how are the different indexes kept in sync?

Also not a lawyer, but I find the language in the license pretty interesting and ambiguous choice of words.

> [...] where obtaining access to the Elastic Software or the features and functions of the Elastic Software is a primary reason or substantial motivation for users of the SaaS Offering to access and/or use the SaaS Offering

It seems like you can still offer Kibana to end customers, but I wonder where the cutoff line for "substantial motivation" ends.

That is a name I have not seen in a long time. I remember when I use to game with Ventrilo/Teamspeak/Mumble about a decade ago.

For anyone who still uses Teamspeak today, what do you use it for?

Out of curiosity, as I have no visibility about the infra actually required- but at that cost, would it not be easier to just have a machine under a desk somewhere?

I currently use btrfs with RAID1 at home, and it works great. But btrfs also does not have the track record for being the most stable filesystem as compared to ZFS.

[1] https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@h... [2] https://lore.kernel.org/linux-btrfs/20200627030614.GW10769@h... [3] https://lore.kernel.org/linux-btrfs/20200520013255.GD10769@h...

dRAID looks really fascinating, but presentation is pretty abstract. Would it allow to add/remove drives from a pool, and allow ZFS to rebalance itself?

Would be great for home use, where I have a lot of drives that I collected over the years that are not the same size.

EDIT: The more I read into this, it still seems assume that all drives must be of the same size.

I really wish that the US voting system did not rely on strategic voting, in which voters are basically forced to choose the lesser of two evils.

I am a huge proponent of ranked choice voting, and I donate to https://www.fairvote.org/. Politics have a wide spectrum of opinions and choices, and our government should represent that.

I have been out of the loop of IPFS. I know IPFS mostly deals with data distribution, but last I checked, it did not have any mechanisms for data durability. Has anything changed on that front?

That is pretty interesting. I used to love developing with the Source Engine, and I found their code base to be stable and easy to read. I hope TC2 gets the long term support and care that it needs to grow a healthy community.

I worked on extending the Source Engine with C#, some 7 years ago, and I feel like I got pretty far, but nothing really usable: https://github.com/SharpMod/SharpMod

But I have not kept up with the changes of what exactly Source 2 is, how backward compatible it is, and how it compares with Unreal/Unity.

I might be wrong here, but can someone comment on the process required to add additional nodes on Vitess?

Every time I start reading about Vitess, I fall into a hole about manual resharding, hosting multiple vitess instances per host (I think 250GB of storage each?), and that cross-shard reads may be inconsistent.

I am taking a quick look at the documentation of TiDB, and they have automatic data re-balancing, 2PC Raft-based-transactions, global snapshot isolation, and actually some nice dashboards to analyze cluster health.

Vitess just does not seem nice from an operations perspective.

I will have to take a look Slonik.

I feel like I have been to hell and back with ORMs, between Sequelize, Entity Framework, Hibernate, SQLAlchemy, etc.. and frankly, I think they just cause more headaches than solve problems.

I would love to have strongly typed SQL queries, but I have found that Dapper [1] fills a special place in my heart.

[1] https://github.com/StackExchange/Dapper

grpc-web is attractive to our use case because it would take away from manually implementing REST interfaces and every once in a while having type mismatches. The less room for mistakes, the better.

.NET Core just recently last month that gRPC-web is stable [1]. It would remove a huge amount of boilerplate in setting up services on both client and server side.

Some people also recommend to use Envoy [2], but usually the less cogs the better.

[1] https://devblogs.microsoft.com/aspnet/grpc-web-for-net-now-a... [2] https://grpc.io/docs/languages/web/basics/

Yeah. I am being very cautious about their desires. I learned that the best way to discourage someone from taking a large project is to show how much work it would be. Just shutting down developers is not nice.

I come from a C# background, but to list out a few hurdles that I have already deal with with TypeScript/NodeJs.

  1. C#'s AsyncLocal has made a few things simpler to trace SQL queries to a request. 
  2. We chose to use hapi.js some 2 years ago, because we found the interface superior to Express, but I did not expect the sole developer of hapi.js to decide stop working on the project [1], and left my head scratching on what I am going to do about that. 
  3. TypeScript's interface sometimes feels too much like just "suggestions", and every once in a while run across scenarios that are not fully supported. One that I really dislike is that Sequelize's "where" options has no type support for the model. [2]
  4. Sometimes libraries does not use async correctly, and breaks stack traces.
  5. Sometimes TypeScript's generic errors can be as bad as C++.

My favorite feature of TypeScript is being able to just define types/objects in-line, but I actually like to stay on the side of caution and stability on a large project with many people.

[1] https://github.com/hapijs/hapi/issues/4111 [2] (GitHub is down, or I would grab the link)

There is an advocate on our team that wants to migrate our web service from Nodejs to Rust.

While I am not a huge fan of Typescript, at least libraries are readily available and generally easy to use. On the other hand, being able to show that we are able to do monitoring, user auditing, ORM, opentracing, gRPC-web with Rust is non trivial.

Now that I think of it, being able to do a "hello world" on any language is pretty simple. But having all the tools around it to build a production level service is a different story.

While the implications are huge, it is hard to think about what is actually being done. XML and JSON are just languages, and the type of information could be stored in any number of ways.

In my point of view, the problem here is that you could say something like "'water' and 'heat' produces 'steam'", but knowledge is never that simple, and understanding of that information is even more complicated.

I would think that Abstract Wikipedia is not the first people trying to solve such a problem, and I am very curious to see what they come up with.

EDIT: Looks like they have an example: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Examples

Not going to lie, playing Crysis was a lot of fun, and I never knew this was the underneath function running it.

This is on the front page again. Please take some time to read this wonderful function: https://github.com/CRYTEK/CRYENGINE/blob/release/Code/CryEng...

EDIT: That whole function is a minefield. Just taking a quick look:

* 814 lines of code

* goto inside 3 nested for-loops

* macros

* commented out code

* new/delete, with no RAII

* thread specific variables and locks (?)

Doing a quick google search, there are claims of "6,000 tweets per second in 2020", or about 6 tweets per millisecond. The blog posts mentions there is an edge case for getting more than 16 tweets per millisecond.

Rather close margins assuming an exponentiation usage growth of Twitter. I wonder how long that variant is going to last.

If I remember correctly, every time a tweet is made, a fan-out operation is made, adding the tweet to each of the follower's timeline. I would think the app just needs to remember the position on their own timeline?

Stackoverflow's DBA had just posted about this: https://twitter.com/tarynpivots/status/1260680179195629568

He has 564,295 tables in one SQL Server. Apparently this is for "Stack Overflow For Teams"

> While your relational database queries slow down as your data grows, DynamoDB keeps on going. It is designed to handle large, complex workloads without melting down.

I mean- hand a person a gun, and they might shoot themselves in the foot. While you can make bad queries/workloads for a relational database, you can just as easily make bad workloads for DynamoDB.