Incoming "Look at all this cool postmortem stuff about our fuckup" blog post. It's getting a bit old guys.
2,445 karma · joined July 4, 2012
It also helps with things such as change of ownership so after a certain period of time you can have the peace of mind that certs potentially issued by the previous owners are not lingering around as active (I understand things such as revoking and pinning can help with this too but It's nice to have a plain time based expiry too).
At that point you turn into a "classic" software vendor where you have to help people "operate" your software. After you have long moved on from something someone will still be on the version from 3 years ago and talking to you about "upgrade/migration path".
I firmly prefer a world where there is only "one operator" for the product and I fully manage 1 instance of the product as a total black box and the end users use it as a ... hello? ... "as a service".
My advice is unless someone cares enough to write you a life-changing check ... stay away from it.
Nice initiative, but don't go bankrupt guys!
If we are talking about the account creation form of Facebook, you bet you will need some CAPTCHA. If it's a random form with no obvious benefit of spamming then I'm not sure how many "attempts" will be done to begin with regardless of the protection mechanisms.
In those cases you may be enocuntering bots that "blast spam" and usually the slightest form of barrier stops them because they tend to be made for the common denominator, for example by targeting popular blog/forum software that have a predictable form structure that the bot can be programmed for.
I have seen some basic anti-spam features that are "home-made captcha".
For example it says something like "Pandas are black and:" and you have to enter "white".
Those can sometimes be made in a way that is more user-friendly compared to a "real captcha".
However it takes some careful consideration and knowing your audience to make sure that they understand what to do. Some users may not understand it due to language or cultural differences or due to people being used to the traditional captcha.
You may want to remove the protection mechanism to see if you get any spam at all or not (or at least log and measure success vs failure cases).
Without knowing anything about your use case, personally I'd remove the CAPTCHA and see how many spams come through. Then I'd put a very basic and gentle barrier just enough to remove those spams and gradually increase the barrier if required.
Another thing to consider is that if your users have to login you can have some kind of basic reputation metric so that "known good" users are not subject to the same restrictions.
Especially in a large codebase where lots of different modules are technically in the same package as each other.
It's easier when a library author is publishing a small package for external consumption by other parties.
Another issue is that in Python any name in a module is implicitly available to be imported from other places.
For example let's say you have "A.py" inside of "A.py" you have "from something import foo". Now "B.py" can do something like "from A import foo".
So now "B" has a dependency that nobody really wanted to create.
Later "foo" may not be needed in "A.py" anymore and it will look like a totally unused import to anyone. Except it is being used by "B.py" too behind your back.
Someone deletes the "unused import" and something else somewhere else breaks because it can no longer import "foo" from "A".
(I have seen variations of this issue happen many many times so don't think I'm making some theoretical scenario.)
In large codebases you can have a problem where people randomly import things from different locations leading to a host of issues including circular import issues.
For example someone finds a convenient helper function in "my_thing.foo.bar.helpers" and imports it but you don't want that dependency between those modules and the helper function was not intended to be used outside of that module.
In Python this problem is especially severe because there is no native module encapsulation mechanism other than a weak conventio of using the underscore prefix.
A tool like this helps you enforce "intentional" module boundaries so modules can't randomly reach into each other to import things. You will be forced to consider an intentional modular design where the helpers that are needed by many things are separated out and other things are allowed to import from it.
There are technically "infinite" different reasons it could be shown. For example a variable is unexpectedly undefined due to a bug in the code.
So the error message can't elaborate in detail exactly what's wrong or why it happened or what to do about it, or whether trying again will help or not, etc...
Behind the scenes (hopefully) the actual error is captured and logged for further inspection.
In other words the end user will see a generic and friendly error message but behind the scenes the developers will see the actual code error along with other information such as a stacktrace to show how the code reached the path where the error happened.
Asking someone to be a programmer in a loud chaotic open office environment is not dissimilar to asking them to program while juggling two balls and sitting on a unicycle. Its just excess difficulty that doesn't need to be added on top of the jobs.
The advantage of using your DB as a queue is that a traditional DB is easier to interact with (for example using SQL queries to view or edit the state of your queue).
In most business applications the message payload is a "job_id" pointing to a DB table so you always need and have to go back to the database to do something useful anyway. With this setup it's one less thing to worry about and you can take full advantage of SQL and traditional database features.
The only downside and bottleneck of having your DB act as a queue is if the workers processes are hitting the DB too frequently to reserve their next job.
Most applications will not reach the level of scale for that to be a problem.
However if it does become a problem there is an elegant solution where you can continously populate a real queue by querying the DB and putting items in it ("feeder process"). Now you can let the workers reserve their jobs from the real queue so the DB is not being hit as frequently.
The workers will still interact with the DB as part of doing their work of course. However they will not ask their "give me my next job ID" question to the DB. They get it from the real queue which is more efficient for that kind of QPOP operation.
This solution has the best of both worlds you get the best features of something like Postgres to be the storage backend for your jobs without the downside of hammering the DB to get the next available job (but in general the DB alone can scale quite well for 95% of the businesses out there).
If a technical founder is sitting down and writing code until 2am on the weekends the non-technical founder better be making phone calls and sending emails, etc...
The inexperienced "idea guy" thinks by sharing "the idea" they have done their part and are just now going to sit and watch as the "the programmer" implements it.
Sure the work has evolved too but a lot of the fundamentals are still the same and you still massively benefit from having that background context and history.
There are a lot of fresh "bootcamp cloud engineers" who know the buzzwords and know which buttons to click on AWS but they don't have the depth of knowledge to safely navigate on their own.
They can give you a 30 minute spiel about why you must move to serverless ASAP but ask them what 777 permission is and they have no idea.
Not that being a beginner is any issue at all if you are self-aware but some of them are lost in YAML and buzzwords and have no idea about their gaps in knowledge and experience.
They arrange things in such a way that all other options other than "shitty engineering" are thoroughly eliminated.
Many companies are not interested in good engineering. At least if they admitted it I'd be way less bitter about it. The insult to the injury is when they insist on pretending like they are into engineering.
If you have 18 years of professional experience in anything, you are senior. It's not that complicated and at that point it doesn't matter what the company is.
Not that it automatically makes you a brilliant developer. But at that level of experience you can't consider someone as junior or mid. It's senior or above.
Once this exceeds a certain amount it will accelerate because more and more people reluctantly start gaming the system to avoid the penalty.
The more automation and AI and bulk/mass stuff is thrown into it the worse it will get (since you get to put a multiplier behind any bad behaviour). When it was slower and more expensive to participate in the job market it was better.
You can find parallels to this in other areas such as the dating market.