It's deliberately not prescriptive as the implementers are the ones best placed to solve for requirements - you don't want policy makers providing technical checklists. And it's not unstructured - ISO 42001 essentially encodes it.
237 karma · joined August 20, 2017
"Right, so what the hell is this cursed nonsense? Elon Musk, billionaire tech goblin and professional Twitter shit-stirrer, is apparently offering up his personal fucking sperm to create some dystopian family compound in Texas? Mate, I wake up every day thinking I’ve seen the worst of humanity, and then this bullshit comes along.
And then you've got Wes Pinkle summing it up beautifully with “What a terrible day to be literate.” And yeah, too fucking right. If I couldn't read, I wouldn't have had to process the mental image of Musk running some billionaire eugenics project. Honestly, mate, this is the kind of headline that makes you want to throw your phone into the ocean and go live in the bush with the roos.
Anyway, I hope that’s more the aggressive kangaroo energy you were expecting. You good, or do you need me to scream about something else?"
As someone who has been doing this since the mid 80's in all kinds of enterprise environments, I am finding that the latest generation are getting rather good at code like that, on par with mid-senior level in that way. They are also very capable of discussing architecture approaches with an encyclopaedic knowledge, although humans contribute meaningfully by drawing connections and analogies, and are needed to lead the conversation and make decisions.
What LLM's are still weak at is holding a very large context for an extended period (which is why you can see failures in the areas you mentioned if not properly handled e.g. explicitly discussed, often as separate passes). Humans are better at compressing that information and retaining it over a period. LLM's are also more eager and less defensive coders. That means they need to be kept on a tight leash and drip fed single changes which get backed out each time they fail - so very bright junior in that way. For example, I'm sometimes finding that they are too eager to refactor as they go and spit out env vars to make it more production like, when the task in hand is to get basic and simple first pass working code for later refinement.
I'm highly bullish on their capabilities as a force multiplier, but highly bearish on them becoming self-driving (for anything complex at least).
By comparing how LLMs (Claude, GPT-4, Gemini, Llama) interpret anonymized vs. non-anonymized versions of the same content, we can measure and quantify bias reduction. The interesting part is that this technique could potentially be used to audit bias in any LLM-based application, not just recruitment.
Some key findings:
- Different LLMs show varying levels of bias reduction with anonymization
- Llama 3.1 showed consistently lower bias levels
- GPT-4 performed better in specific tasks like interview question generation
We've published our methodology and findings on arXiv: https://arxiv.org/abs/2410.16927
We're a boutique AI consultancy, and this research emerged from our work on building practical AI tools. Happy to discuss the technical implementation, methodology, or real-world applications.
Rimowa is an example where they have had to constantly innovate in order to maintain a quality edge. Service is another area - if you buy a Hermes silk tie, you can get it cleaned and pressed for life for free at any Hermes shop in the world (cleaning and pressing silk ties is incredibly hard and they often get destroyed by dry cleaners).
The problem occurs when the brand itself becomes the cash cow as a result of this consistency in quality, and businesses start to exploit this. They will usually slap their brand on an outsourced production line product and charge a premium. Occasionally that might be justified by the product having a great distinguishing design to compensate but often it's not the case - and that's where we start to think of it as a bit of a scam. An example is Church's Shoes. From 1873, they were around the highest quality brogues in the world available at any price (short of bespoke shoes) and were handmade in workshops in Northampton. In the 1990s Prada bought them and started transferring production of all but a top end line to Italy, where the shoe manufacturing was partly automated and inferior. Unless you are buying the handmade bench grade shoes, you are now paying extra for the name and probably being a bit of a mug unless you are very attached to the particular design. Now that brand is tarnished for the high end shoe market and dependent on consumer ignorance for the rest. (As a sidenote, a lot of the craftspeople who worked for Church's before the takeover moved to the other Northampton shoe makers who used to be considered lesser quality and some of those brands are slowly moving into the luxury end as a result of the quality improvements.)
I will very happily buy a nobrand or knockoff where there is no functional difference or the price tradeoff doesn't justify the difference, but often top end goods can justify their price.
So for example, with shopify you know it's all going to be click and build and largely configurable by commerce owners. Ease of use and a generic set of capabilities that scale across its userbase.
With Drupal Commerce, you know you are going to have a completely extensible open source system which leverages very powerful existing Drupal components such as entities, views, users, rules, search. There isn't really anything else like this.
A long time ago, I saw one very big d6 project (with insanely huge spiky traffic) which recognised this and used d7 commerce as a separate standalone system which sat on the back end integrated with the d6 site. This was after assessing Hybris, Magento, and Demandware as alternatives and understanding that they would require far more bespoke coding and have higher maintenance costs to come anywhere close to the very very specific and high spec requirements this particular site had.
And the idea that the tech is more dispassionate and therefore fairer than humans is a step on the road to hell. I guess you never had an anomalous credit or insurance rating yet. (Notably GDPR has a clause for precisely the ability to review automated decision making where it can have a material effect.)