roncohen

As someone who myself worked on a hobby-level Rust based Kafka alternative that used Raft for metadata coordination for ~8 months: nice work!

Wasn't immediately clear to me if the data-plane level replication also happens through Raft or something home-rolled? Getting consistency and reliability right with something home-rolled is challenging.

Notes:

- Would love to see it in an S3-backed mode, either entirely diskless like WarpStream or as tiered storage.

- Love the simplified API. If possible, adding a Kafka compatible API interface is probably worth it to connect to the broader ecosystem.

Best of luck!

Trunk-based development works.

In fact I suspect most people working in SaaS are doing some kind of Trunk-based development - some unknowingly.

There are tons of misconceptions about _what it is_. I wrote a blog post on common misconceptions about trunk-based development here: https://bucket.co/blog/trunk-based-development-crock-of-shit

Agreed! It’s apparently a divisive topic.

It’s in the post :)

> In practice, many teams will use a pull request-based workflow because the tests could be too complicated to run locally or need to facilitate code reviews. For trunk-based development to fulfill its promise, it’s imperative that pull requests are kept small and that the team is committed to reviewing them quickly.

June is cool. Best of luck!

Would you consider adopting the STARS framework[1] ? It was designed to solve the questions you mentioned like “Are our key customers adopting the new feature we launched?” but at the same time it also incorporates qualitative feedback.

[1] http://starsframework.org/

Full disclosure: I helped design it.

Lots of great additions. I will just highlight two:

Column selection:

When you have tons of columns these become useful. Clickhouse takes it to the next level and supports APPLY and COLUMN in addition to EXCEPT, REPLACE which DuckDB supports:

    - APPLY: apply a function to a set of columns
    - COLUMN: select columns by matching a regular expression (!)

Details here: https://clickhouse.com/docs/en/sql-reference/statements/sele...

Allow trailing commas:

I can't count how many times I've run into a problem with a trailing comma. There's a whole convention developed to overcome this: the prefix comma convention where you'd write:

    SELECT
      first_column
      ,second_column
      ,third_column

which lets you easily comment out a line without worrying about trailing comma errors. That's no longer necessary in DuckDB. Allowing for trailing commas should get included in the SQL spec.

Couple of gripes:

    - Still no date type
    - Why make it more loose? Like "Strings may be single quoted." doesn't bring any value.

I was delighted to see Opbeat on there (under Application Performance Monitoring).

Opbeat was acquired by Elastic, Inc and turned into their APM product. You can try the Elastic APM product for free as well: https://www.elastic.co/apm

(I was an Opbeat co-founder and part of the team that built Elastic APM)

The other problem here is that politicians are either oblivious or simple don't care about the cost associated with complexity in the laws. On top of that, the other unfortunate dynamic that exists is that politicians all work to get their little special case into new bills so they can point to that when their constituents ask how they influenced the new law.

Taken together, the complexity of the law just accumulates. Tax law in Denmark being a particularly gross example.

How do you feel about the EU being connected to the Turkish electricity grid? [1] and Germany being increasingly dependent on Russian gas? [2]

[1] https://www.avrupa.info.tr/en/plugging-turkey-eu-electricity...

[2] https://www.dw.com/en/nord-stream-2-pipeline-row-highlights-...

Off topic, but here goes:

If you're using Apache Kafka or similar I'd love to understand how you're using it and which problems and annoyances you're running into. Please shoot me a mail at cohen1@gmail.com if you're up for a brief Zoom call or email exchange.

I'm doing customer/product research on Apache Kafka and similar systems to inform a software project I'm working on.

taking this a bit further, the original authors could probably employ this as a strategy: Since it's an anonymous auction, the original author could win the auction and then refuse to pay repeatedly until all the real potential buyers have given up.

another contender is zstd: https://github.com/facebook/zstd. It typically offers better compression ratios than LZ4 at a slight (depending on your data) cost in speed. Additionally it offers a training mode to tune the algorithm to increase compression ratio on specific types of data, particularly useful for compression of small pieces of data.

> Not only is what the subject of the article did probably not illegal

Why do you say that?

The method they used consists of simply filing forms to get withheld taxes paid out that were never withheld. The amounts paid out corresponds to owning all stocks in the actual companies many times over.

Sanjay himself went drinking with the employee of the Danish tax authority in Copenhagen who was in charge of approving the applications while the (I'll go ahead and use the word) fraud was going on, presumably to ensure things would continue to go on[0]. The employee has since been jailed for approving fraudulent applications in another case[1]

American pension funds who similarly exploited this have settled with the danish authorities and returned all the money they received[2]

Sanjay says he suggested a settlement, offering to return 1.9B DKK (~311m USD) in 2018[3]

Yes, the danish authorities were sleeping at the wheel, but that does not make it legal.

[0] https://www.dr.dk/nyheder/webfeature/milliardkuppet

[1] https://www.dr.dk/nyheder/penge/seks-aars-faengsel-til-sven-...

[2] https://www.skm.dk/aktuelt/presse-nyheder/pressemeddelelser/...

[3] https://www.dr.dk/nyheder/penge/sanjay-shah-jeg-tilboed-danm...

> This post is obviously from an engineer

Not really. The post is by Matt Asay who's title is Head of Open Source Strategy and Marketing. The post reads like an attempt at improving upon the negative image AWS has garnered in open source by making money on open source projects without paying, or sometimes without even crediting, the authors. Most recent examples include Headless Recorder[1] and before that Kafka, Elasticsearch etc.

[1] https://news.ycombinator.com/item?id=24799660

Maybe Elastic APM fits your need? It works well on-prem as well as on the Elastic hosted offering and you can get started for free.

Disclosure: I work for Elastic

This caught my eye: Home made leader election protocol that relies on UDP.

As an aside, one of them burned last year: http://www.powerengineeringint.com/articles/2017/08/new-vest...

By the way, you can walk right up to them. They are very impressive up close.

Thanks! If you shoot me an e-mail i can get you access: ron@opbeat.com

And if you're curious about how this sort of thing could look on the client/browser side of things, check out https://opbeat.com/react

(I work on it)

Opbeat | Copenhagen, Denmark | ONSITE | Full-time

Help build next-gen performance monitoring solutions.

We're looking for:

* Frontend developer

* Backend developer

Interview process is low stress, includes real world coding problems.

Small, skilled team. High momentum, low friction environment. Very few meetings. Flexible hours. Dogfooding.

https://opbeat.com/jobs

Great trick. Incredible that it's not on by default.

if you've applied that trick and still find yourself wondering what's taking up time, we just released a performance monitoring tool. We're looking for feedback, so please let me know how you find it: https://opbeat.com/angularjs

Copenhagen, Denmark

Opbeat - The first ops platform for developers[0].

Work on solving problems so your fellow developers don't have to. Python, Go, Ruby, Node + Cassandra & Postgres + AWS. Small, skilled, international team. Wonderful problems.

We're looking for developers in infrastructure, web backend frontend and Android. Get in contact: ron@opbeat.com

/Ron

CTO&PM

[0] https://opbeat.com

Most merchants are OK with the Dankort. Handling cash (in a near cashless society, mind you) is not free either. Bottom line is: if i can't pay with my Dankort, I'll go somewhere else.

It's a card with a "chip" and a pin. No NFC/paypass and no plans to change that as far as i know :(

I believe the trend in the nordics is largely driven by the Dankort and similar (http://en.wikipedia.org/wiki/Dankort) which is a debit card that is free to get and use for consumers. Merchants pay an annual fee depending on their transaction volume and they are not allowed to pass the cost on to the consumer. It doubles as a VISA which is neat when we go abroad.

An interesting data point is that every single taxis here is happy to take your credit card. I was surprised to learn that I was unable to pay with my VISA in many taxis in the UK and USA.

Super interesting. I'd love to learn Chinese and be able to do this one day: http://www.usatoday.com/story/tech/2014/10/22/mark-zuckerber...

Though, I'm not sure reading novels like the website suggests is the right approach when starting from nothing.

We also migrated a while back (opbeat.com). While we run a smaller setup, I imagine that might be the case for most readers. We run a pretty standard setup with ELB, Postgres (master/replica), webservers and job processing servers.

This recipe details what we did (as i recall it):

  1) Prerequisites: Running at least two of everything in
     separate AZs and expertise (or courage) to fail over
     to a replica DB.
  2) Boot up instances of everything in the VPC
  3) Set up a new ELB inside the VPC, add the web servers inside to the ELB
  4) Make sure your instances inside the VPC can talk to
     which ever service they need outside (replica database
     and web servers needs to reach master outside +
     memcached). Use `telnet` to make absolutely sure :)
  5) Make sure web and job servers can reach the
     replica db and memcached inside
  6) Test out the new VPC ELB from outside
  7) Switch DNS over to the new VPC ELB, wait for it to
     propagate.
  8) Do a failover from your master to the replica inside.
  9) Same for memcached
  10) Shutdown everything in EC2 classic.
  11) Drinks

EDIT: formatting

Well done Jan Erik and team!

I'd love to see Netflix let the chaos monkey loose on their PostgreSQL servers.