- "Never download anything unless it's from the Apple App Store"
- "Never buy anything unless you're on amazon.com"
- "Dont use the internet outside of ChatGPT"
I'm actually somewhat less critical of Apple/Google/Facebook/etc. than probably most readers would be, on the grounds that it simply isn't possible to build a "walled garden" at the scale of the entire internet. It is not possible for Big Tech to exclude scammers. The scammers collectively are firing more brain power at the problem than even Big Tech can afford to, and the game theory analysis is not entirely unlike my efforts to keep my cat off my kitchen counter... it doesn't matter how diligent I am, the 5% of the time the cat gets up there and finds a tasty morsel of shredded cheese or licks some dribble of something tasty barely large enough for me to notice but constitutes a nice snack with a taste explosion for the much-smaller cat means I'm never going to win this fight. The cat has all day. I'm doing dozens of other things.
There's no way to build a safe space that retains the current size and structure of the current internet. The scammers will always be able to overpower what the walled garden can bring to bear because they're so many of them and they have at least an order of magnitude more resources... and I'm being very conservative, I think I could safely say 2 and I wouldn't be really all that surprised if the omniscient narrator could tell us it's already over 3.
[1]: https://9to5mac.com/2025/09/25/new-study-shows-massive-spike...
[2]: To forstall any AI debate, let me underline the word "lazy" in the footnote here. Most recently we received a shirt with a very large cobra on it, and the cobra has at least three pupils in each eye (depending on how you count) and some very eye-watering geometry for the sclera between it. Quite unpleasant to look at. What we're getting down the pipeline now is from some now very out-of-date models.
You don't even need a human to review these ads but inserting one wouldn't be expensive.
It's 100% possible. It might not be profitable
An app store doesn't have the "The optimum amount of fraud is not zero" problem. Preventing fraudulent apps is not a probability problem, you can actually continuously improve your capability without also blocking "good" apps accidentally.
Meanwhile, apple regularly stymies developers trying to release updates to already working and used by many apps for random things.
And despite that, they let through clear and obvious scams like a "Lastpass" app not made by Lastpass. That's just unacceptable. Anything with a trademark should never be possible to get a scam through. There's no excuse.
Unfortunately it is. You've even provided examples of a false positive and a false negative. Every discrimination process is going to have those at some rate. It might become very expensive for developers to go through higher levels of verification.
The answer is that it just takes a lot of people. What if no content could appear on Facebook until it passed a human moderation process?
As the above poster said, this is not profitable which is why they don't do it. Instead they complain about how hard it is to do programmatically and keep promising they will get it working soon.
A well functioning society would censure them. We should say that they're not allowed to operate in this broken way until they solve the problem. Fix first.
Big tech knows this which is why they are suddenly so politically active. They reap billions in profit by dumping the negative externalities onto society. They're extracting that value at a cost to all of us. The only hope they have to keep operating this way is to forestall regulation.
Move fast and break things indeed.
The more of those people you hire, the higher the chance that a bad actor will slip through and push malicious things through for a fee. If the scammer has a good enough system, they'll do this one time with one person and then move on to the next one, so now you need to verify that all your verifiers are in fact perfect in their adherence to the rules. Now you need a verification system for your verification system, which will eventually need a verification system^3 for the verification system^2, ad infinitum.
At the end of the day, I can't make an ad and put it on a billboard pretending to be JP Morgan and Chase. I just can't.
Worldwide and over history, this behaviour has been observed in elections (gerrymandering), police forces (investigating complaints against themselves), regulatory bodies (Boeing staff helping the FAA decide how airworthy Boeing planes are), academia (who decides what gets into prestigious journals), newspapers (who owns them, who funds them with advertisements, who regulates them), and broadcasts (ditto).
> At the end of the day, I can't make an ad and put it on a billboard pretending to be JP Morgan and Chase. I just can't.
JP Morgan and Chase would sue you after the fact if they didn't like it.
Unless the owners of the billboard already had a direct relationship with JP Morgan and Chase, they wouldn't have much of a way to tell in advance. If they do already have a relationship with JP Morgan and Chase, they may deny the use of the billboard for legal adverts that are critical of JP Morgan and Chase and their business interests.
The same applies to web ads, the primary difference being each ad is bid on in the first blink of an eye of the page opening in your browser, and this makes it hard to gather evidence.
Again, the newspaper model already solves this. Moderation should be highly localized, from the communities for which they are moderating the content. That maximizes the chance that the moderator's values will align with the community. Small groups are harder to hide bad actors, especially when you can be named and shamed by people that you see every day. Managers and their coworkers and the community itself are the "verifiers."
Again, this model has worked since the beginning of time and it's 1000x better than what FB has now.
While I'd be just fine with Meta, X etc. (even YouTube, LinkedIn, and GitHub!) shutting down because the cost of following the law turned out to be too expensive, what you suggest here also has both false positives and false negatives.
False negatives: Polari (and other cants) existed to sneak past humans.
False positives: humans frequently misunderstand innocent uses of jargon as signs of malfeasance, e.g. vague memories of a screenshot from ages ago where someone accidentally opened the web browser's dev console while on Facebook, saw messages about "child elements" being "killed", freaked out.
A lot of people = a lot of cost. That would probably settle out lower than the old classified ads, but paying even a dollar per Facebook post would be a radically different use than the present situation.
And of course you'd end up with a ban of some sort on all smaller forums and BBS that couldn't maintain compliance requirements.
These are the effectively the same thing. Asking a business to harm its profits is like asking a person to self-harm.
It may be unreasonable to demand that a small business tackle a global problem at the expense of its survival. But we are not talking about small or unprofitable business. We are talking about Meta, Alphabet, Apple, Amazon. Companies with more money than they know what to do with. These global companies need to funnel some % of their massive profits into tackling the global problems that their products have to some degree created.
Okay, but if it matches the illustration on the storefront, can it really be called a scam?
We have also received a number of shirts where AI has been used to create unlicensed NFL shirts and other such actual frauds. And whatever your feeling about IP laws, it was definitely low quality stuff... looked good if you just glanced at it but when you went to look at any particular detail of the shirt it was AI garbage. (I say "AI garbage" precisely because not all stuff from AI is necessarily garbage... but this was.)
Sigh. I learned from my pre-boomer parents that if the product were any good it wouldn't need to be advertised.
> looked good if you just glanced at it but when you went to look at any particular detail of the shirt it was AI garbage.
To be fair, that was also all over the place before "AI" as currently understood. (And I don't think that previous iterations of machine learning techniques were involved.)
It has some of the disadvantages, some of the advantages. It's not (and never was) perfect.
That said, at this point I am a disillusioned app developer: I no longer believe that the creation of an app is a good business idea in most cases.
My two most recent examples: a couple of rolls of 3D printer filament that looked nothing like as advertised (bad sales images there I think, rather than a comingled-with-a-cheap-scammy-alternative issue) which was taken back unquestioned for same-day full refund despite one of them being opened, and a couple of years ago a replacement drive for my media RAID array that, while the right drive and not, as far as I could tell, counterfeit, certainly wasn't new/unused which is what I ordered, which again was taken back with no quibble or cost (other than my time of course).
There are problems dealing with Amazon sellers, but those can mostly be avoided with care and a healthy dose of cynicism (to avoid ordering crap in the first place). I'd never buy some things from there though: safety equipment, for instance.
Though as mentioned, I find it very easy to believe this will vary by location and account for various reasons.
Technically, on Bol.com, a EU-platform, EU consumer protection is in place. So if a product breaks within guarantee terms, is dangerous, never gets delivered etc. the person re-selling is responsible. They are importing "illegal" goods and could even go to jail for it.
So, technically, that premium price brings me me the assurance that I am protected by EU consumer laws. That a TV I buy can be returned, is CE certified, won't explode and isn't a 12" TV pictured in a tiny living-room on the images on unpacking.
Except these products often don't meet EU criteria, aren't adhering to (food, safety, chidren protection) EU laws and money-back is often hard because the re-seller just dissapears. In the last case, Bol.com will step up and refund, because they have to. But for the rest, they plead innocence: It wasn't us that sold illegal goods, it was that reseller from which we skim a lot of fees.
The incentives are just wrong. And the solution simple: Make platforms by proxy legally responsible for their "users". Resellers in my case. Or advertisers in the case of TLA.
If some-guy sells a TV that explodes, and can't be found or held responsible, then make Bol.com responsible. Let their CEO go to jail in the very worst case. Let's see how fast they solve this.
That is bog-standard drop-shipping. Every open online market had a pile of that. It isn't that they've taken the images from AliExpress it is that both sets of sellers are drop-shipping product from the same source or collection of sources (or buying and reselling though that is much less common as it means managing stock) and the images come & other sales material come from there.
> So, technically, that premium price brings me me the assurance that I am protected by EU consumer laws.
When comparing Amazon (UK) or eBay to the sellers on, for example, Facebook, often there isn't a premium, Amazon (or AliExpress, or similar) are often cheaper than sellers on social media and/or advertising via adverts on YouTube and their ilk. Those sellers will often try to make the product out to be some unique high quality item with a price to match (which of course is heavily discounted if you buy in the next hour or two), and if you check your preferred general marketplace you'll find several people with the same thing, often with the same images, making no such pretence of it being unique or high-value, at a price noticeably cheaper than the seller from SM/etc. I assume this is the same with Amazon in other jurisdictions and other marketplaces like Boi.
Shouldn't the manufacturer have some liability?
But how are you going to enforce that liability? Making and selling knock-off "Lego ™" is already "illegal"¹ yet Ali-express is filled with this. How would this change when this knock-off-lego is also made with poisonous plastics?
Point is that e.g. within the EU the liability is clear and enforcable and the manufacturer has a role there. But with imported products, it doesn't. That liability lies entirely on the importer.
¹ (in legislations that recognize the International Trademark and copyright laws)
The most common one I've run into is third party sellers taking items that come in multiple to a pack from the manufacturer and splitting them up but then also listing the single item for the same price as the multi-pack's MSRP.
As an example, pouches of cat food treats that come 10 to a pack. Scam sellers will split the pack and sell each pouch for the same price as the full 10 pack and because Amazon has historically done nothing to guard against this, their scam listing appears fully comingled with the manufacturer's listing in a way where it is very hard to recognize the scam option even if you are aware of the possibility.
Amazon has made some noise about fixing these comingling issues this year, but their plans have been vague and for me the well is already poisoned after years of letting it go.
Its actually shocking that it took until this year for Amazon to really acknowledge this as an issue. Manufacturer/brands can't have been happy about this considering that for any item that can be scammed like this you'll find lots of bad reviews on Amazon where the review isn't really complaining about the product, but the scam.
Some example reviews that I just randomly and easily found on Amazon:
Could I interest you in some very durable car fuses that don't actually trip? https://youtu.be/B90_SNNbcoU?si=5QUpXUHwSlZj4i4G
Or perhaps radioactive protection pendants are your thing? https://shungite-c60.com/quantum-pendant/
Could I interest you in some Amazon choice firecrackers? https://edition.cnn.com/2020/09/10/business/amazonbasics-ele...
Let's not even mention the health and nutrient products that make the FDA shudder.
Sure, you can ask for your money back, and flag the seller. But new sellers pop up selling the same crap all over again with a new name and company ID. This is all while real sellers of real (and safety certified products) get pressured by Amazon and dissuaded from taking their business off platform.
Avoid Amazon if at all possible. It's not good for consumers nor sellers, and it's keeping a leach on online retail.
Most countries have laws around liability of sold products. This is often set up to fall on the importer of said product. Amazon Europe (and perhaps USA) is doing something very funny with these laws; You, the consumer, is the importer. If your house burns out, then it's between you and a random chineese ghost companny that just disappeared into smoke. Amazon is "handling the import paperwork for you", and not taking liability for anything.
A lot of consumers have no idea they got a cheap imitation. Counterfeiters have gotten quite good, and in many cases the scam is "falls apart in a year instead of ten", not "it's completely non-functional".
Might as well do AliExpress, same quality control/misleading descriptions but lower prices.
Is there even a trustworthy online shopping site/platform nowadays?
I never buy food, supplements, OTC meds, etc. online from any source. That stuff I always buy in person at a local retailer.
Smart, on device agents that are aligned with a user's interests will be able to act as the "walled garden" the user needs. In fact, this future is anti-dysopian, because the agent will not care about existing walled gardens and digital fiefdoms, and to the extent that it's using them it's going to deprive them of ad revenue, and they'll have to sit and take it because being agent unfriendly will be a death sentence for a business.
Too late for that one. I have been scammed a few times from Amazon sellers.
do that sooner rather than later.
Voice mimicry is so much easier now, that you might not be able to tell from the phone. This is why a verbal password from family is important, esp. in unusual situations.
You probably only need just one, for authentication purposes. I doubt grandmother can perform this sort of "spycraft" well if it's more complicated. Better to have something simple and easy, than to forget and panic later.
Worse, your fake version will be convincingly begging on the call for God knows what while being horribly tortured. Audio versions of this are already a thing.
So my strategy here has been to start downloading anything that I think I might need from the Internet and keeping a local copy. It's free and abundant now. It could become inaccessible within a matter of minutes if the right powerful person says so. There may be a low probability of that happening, but given the potential disruptions to our life of our always-on connectivity going away, it's worth being prepared.
Having a copy of your own medical records could be critically important if suddenly your friend who is a doctor is now your doctor because there's nobody else.
Money tends to be worthless is such situations anyway - it's backed by the full faith and credit of a government that best case no longer cares about you and worst case no longer exists. So you aren't going to worry about your bank account, nobody will accept credit or debit anyway.
There is a whole lot of other data on the Internet that can be very, very useful in such situations. Even just having a few hundred hours of collected kids TV shows means you can sit them down in front of Bluey while the adults do stuff that is critical for survival. Knowing how to build a smelter and bellows out of clay that you can find locally means you can restart metal production in a matter of days rather than thousands of years. Knowing what the local plant species are and which are edible might keep you from starving. Knowing how to scavenge parts and wiring, as well as what their datasheets are and how you need to hook them up, means you can fix broken electronics and potentially create new ones, which gives you continuing access to knowledge.
- a substrate for business VPN tunnels (like the leased lines of old),
- regulated, approved, monitored e-commerce walled gardens,
- regulated, approved, monitored streaming walled gardens (like the cable TV of old),
- regulated, approved, monitored social media walled-gardens including chat focused ones like Discord and video focused ones like YouTube (replacing most for-print media, forums, and websites)
I know most of that was driven by the tragedy of Adam Walsh, but it was still great OpSec I'll never drop.
You’ll be suspicious and ask for the pass phrase. The attacker now knows the nature of the protection you setup between you and your mom.
And then the real attack on your mom, with you describing the system you’d agreed to, and claiming you can’t remember the word/phrase.
Better is the Terminator-style lie to see if it gets detected.
God help anyone not armed with AI in the future, that's why it cannot be locked up by corporations or government.
trends point to will be locked up by corporations for the near to medium term
I have done this already and convinced a friend to do it after her father fell victim to a scam where he was convinced the sheriffs department wanted him to pay off a fine in gift cards.
I am also concerned that one might steal a trove of texts from someone and plug it into AI which could mimic the writing and tone of someone.
None of that is personal obviously just the gut reaction I got reading that initially. I suspect maybe nobody has mentioned it before and it might be helpful to hear on the assumption that others feel similar.
I don’t know how this changes for people who are less sure how to have that conversation and I suspect the fact that I’m a real life security person might have something to do with it.
Edit: I just saw the website after this and I get what you’re doing and can see how maybe it makes sense for some but I’d never recommend this to my own parents, I don’t think sticking an AI in the middle of all of their personal communications is the right answer and I’d have a lot of questions about how that data gets used to be honest.
Again, nothing personal but there’s just no possible universe where I’m setting something up so that every personal message I ever send my parents again is getting silently sucked up into some random company’s cloud to be read and analyzed and then paying them money for that. One of the things I actually had to show them was how to disable that kind of shit on their Gmail accounts for example.
A passphrase is cheap. If you never need to use it, so what?
These scams are imaginary. Yes they have probably happened but it's far more likely that your mother will get scammed by a "legitimate" financial advisor than some stranger using an AI to impersonate you.
And no need for AI. I already had family members getting scammed by facebook or mail impersonation.
But at some point, the capability for AI is going to get so cheap it will be mass-produced by bots from millions of identities like SMS or email scams.
It's not an "if". It's a "when".
I'd rather have my mother at least have the reflex to try to send me a message if I can't remember the passphrase to check if it's indeed me.
And if it doesn't, work, it doesn't. I'm not working for a 3 letters agency and my mum is an old tech illiterate lady. There is a limit.