kainosnoema

Clearly not consulted earlier, ChatGPT weighs in on these two reasons: https://chat.openai.com/share/7cd52d82-b36b-42c6-9d13-eb7172.... Edit: even following the basic steps it outlines would've resulted in a better outcome.

This is just not true for us. I'll share again what I've said before: we’ve been testing the upgraded GPT-4 models in the API (where you can control when the upgrade happens), and the newer ones perform significantly worse than the original ones on the same tasks, so we’re having to stay on the older models for now in production. Hope OpenAI figures this out before they force the upgrade because quality has been their biggest moat up until now.

I don't know why I keep seeing comments like these. We've had to delay the latest API model upgrade because it's dramatically worse at all of our production tasks than the original GPT-4. We've quantified it and it's just not performing anymore. OpenAI needs to address these quality issues or competitor models will start to look much more attractive.

We’ve been testing the upgraded models in the API (where you can control when the upgrade happens), and the newer ones perform significantly worse than the older ones on the same tasks. Tweaking the prompts helps some but not enough. We’re staying on the older models for now in production.

Hope OpenAI figures this out because quality has been their biggest moat up until now.

Really enjoyed all the minute details in this piece, like watching a documentary.

Yes, have been planning on it for awhile but yesterday's outage was last straw for us. Probably moving to Fly.io (https://fly.io).

The announcement implies that they no longer attempt fast charging over USB-C, so lower power requirement will improve reliability with 3rd party hardware.

My guess is this change was brought about by all the poor quality USB-C cables, hubs, and chargers out there. I bricked my Macbook Pro M1 by trying to power it from a supposedly compatible hub but wasn't negotiating the power output properly. Had to send it in for repair and have heard many other similar reports... guessing they decided to bring back the Magsafe for fast charging to avoid this headache of cheap/improperly manufactured USB-C hardware.

It depends on the organization. We've had leaders move back to IC roles (and are going through one of these transitions now, actually), but I think it requires the organization to be explicitly supportive of such moves. We call it finding your "zone of genius". We'd much rather keep a good person then lose them because they're best suited for another role.

Clearbit | San Francisco, CA or Remote | FULLTIME, REMOTE | https://clearbit.com

Our mission is to build the data backbone for modern businesses. Clearbit is built by a team with backgrounds at Stripe, Twitter, Google, and Heroku. We've come together to transform the world of business intelligence data and build a product companies love to use. We're VC-backed but growing with revenue.

Currently hiring across the team, but most in need of a sysops engineer with experience in AWS and K8s:

* SysOps Engineer: https://jobs.lever.co/clearbit/f29fbc8f-4a03-4a73-80fc-50389...

Yes. As of a couple years ago at least, Slack was a monolithic PHP app that could support a few thousand users in a single team. Each “workspace” was assigned to a server and scaled vertically. LinkedIn notoriously had two Slack workspaces in order to support all of their users, and I’m guessing other large companies have had to do the same.

I’m sure Slack is using their vast resources and large team to solve these problems at this point, but it probably hasn’t been easy to create a horizontally scalable system from the early codebase, especially when the data model was designed for a single team of users.

Most Swift code is divided nicely up into modules, which are compiled together and have their own namespace. https://developer.apple.com/library/content/documentation/Sw...

Unfortunately, iptables can't protect against all forms of DDOS attacks. Even just getting flooded by packets being routed to a particular IP can cause a datacenter's network to be affected. Something like CloudFlare Warp is the only way to truly prevent packets from being routed to your servers in the first place (I don't work for CloudFlare).

It's related. This was during the early versions of the security patch, which was causing performance and clock skew issues for PV instances (https://twitter.com/kainosnoema/status/941816042103324672). Since then AWS has mostly resolved these, but we've moved all our instances to HVM AMIs as well since they're apparently not affected.

Just enabled it for our account (incredibly easy, single-button activation), and by morning we had results showing some minor vulnerabilities in our public subnets that we were able to patch immediately. Highly recommend.

Actually I think their approach is great—it's very modular and there's little overlap between use-cases. An exception (among a few) may be ECS and EKS, where ECS was probably the wrong bet for them now that K8s is getting so much traction. Hence EKS.

But being able to put use either EKS or ECS for orchestration, and then schedule those tasks on either EC2 or Fargate (depending on compute needs), opens up a lot of options. You can start simple and grow as requirements become more complex—without fundamentally changing the deployment artifact. That was the promise of containers originally, so it's nice to see it play out.

Yes, EKS is just managed K8s, which is the orchestration layer. You still need to have EC2 instances for the EKS tasks to be scheduled on. Unless you run your EKS tasks on Fargate, which is coming in 2018.

We launched our production workload in us-west-2 (Oregon) 4 years ago for this exact reason and it was one of the best decisions we made. Our staging servers in us-east-1 have experienced all of the outages between then and now, but production hums right along. Of course we're ready to failover to us-east-1 if us-west-2 has trouble, but it's been far more stable (knock on wood).

Also getting 503 Slow Down and 500 internal server errors from S3 in us-east-1

Absolutely, I wasn't suggesting that you should change the naming. In fact I agree that nonce is likely not the best term for this.

But the concept as it applies to HTTP APIs is a great one that I think could have a broadly recognized implementation. Whether folks start adopting the "Idempotency-Key" header or some other approach, having awareness and maybe even a standard could be great.

In principle, what you've built is quite similar to a cryptographic nonce [1], usually used to prevent replay attacks against authentication services, etc.

We use similar tokens in our API for idempotency as well, though we called them nonces. Standardizing a bit on terminology and how to implement them properly would be really helpful.

[1] https://en.m.wikipedia.org/wiki/Cryptographic_nonce

This is one reason we decided at Cotap to purge messages after 14 days by default, and only keep them longer if requested (https://cotap.com/blog/customizable-data-retention-for-busin...). Contrary to what one might expect, most users embraced the change.

It may not always be better since you'll likely never be able to call from Go into Cocoa libs, but if you can write a shared library and use it on the server, iOS and Android, then it's a big win. Linking a Swift app against the shared Go library and calling directly from Swift into Go sounds pretty great to me.

In either case—assuming you run a VPC—you usually configure one or more NAT instances to allow EC2 instances to communicate with the internet (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NA...). I suppose a bastion could do double-duty as a NAT, but in most cases you want one per availability zone to add isolation and redundancy.

Elasticsearch actually sacrifices both consistency and availability during partitions, even when inside a single datacenter (trust me, they happen in production).

If you configure ES to prioritize consistency somewhat (minimum_master_nodes), it prevents writes during a partition—but there's at least one "split" partition scenario where even minimum_master_nodes doesn't prevent inconsistent writes. If you configure ES to prioritize availability during a partition, it isn't consistent. Remember, ES doesn't claim to be consistent and doesn't even use any sort of consensus algorithm.

Apple just announced in the Platform State of the Union that they won't guarantee source compatibility until Swift is released along with iOS 8 (changes to the language will require source conversions), so I believe they're taking a route closer to option #1.

This is amazingly great advice. Don't go pre-"optimizing" by adding a bunch of needless network calls—you'll only end up with more instability if done naively. DO start from the beginning with clear interfaces and boundaries between concerns in your codebase. Design for services from day one. If you don't have discipline not to tightly couple objects together, split into multiple codebases (gems, modules, whatever) tested in isolation that can be deployed together until it's time to pull them apart.

A well designed class interface can be turned into an HTTP (or socket) API easily enough. By the time you need to, you'll probably have the resources to monitor, scale and maintain the additional service properly.

I don't believe it's "master-slave" in the way you're thinking. Elasticsearch shards its indexes among all available nodes, storing replicas of each shard on separate nodes when possible. This ensures that the entire cluster is available as long as at least one replica of a shard is still online. In fact, if configured properly, it has better availability than consistency since by default it only flushes its oplog to the Lucene index segments every second (though writes aren't considered committed until they reach a quorum of nodes, so consistency is fairly good in practice as well).

I'm surprised so many people miss this. Out of the box, Elasticsearch is a distributed NoSQL store with better write consistency (and arguably performance) than MongoDB offers in its default configuration. The major missing feature was backup snapshots and restores, which 1.0 delivers—along with aggregations that more than rival MongoDBs. The team has intentionally avoided marketing themselves as a NoSQL store (was told this directly by an employee), but they're aware of the potential and have customers using it as such.

The OP is working on a nicer machine-readable output for exactly this purpose, though I think he's hoping for a Vim plugin.

I'm also be wanting a Sublime plugin, preferably one that can cycle the selection through tagged words and allow editing. I started hacking one out last night, but got stuck on the complexity of the plugin architecture. I'll update this if I ever get something working.