Please stop externalizing your costs directly into my face
drewdevault.com
drewdevault.com
Even then, I see bot after bot, pulling down about 1/2 GB per day.
Like, I distribute Python wheels from my site, with several release versions X several Python versions.
I can't understand why ChatGPT, PetalBot, and other bots want to pull down wheels, much less the full contents when the header shows it hasn't changed:
Last-Modified: Thu, 25 May 2023 09:07:25 GMT
ETag: "8c2f67-5fc80f2f3b3e6"
Externalizing their costs onto me.
I know 1/2 GB/day is not much. It's well under the 500 GB/month I get from my hosting provider. But again, I have a tiny site with only static hosting, and as far as I can tell, the vast majority of transfers from my site are worthless.
Just like accessing 'expensive endpoints like git blame, every page of every git log, and every commit in every repo' seems worthless.
"I have to review our mitigations several times per day to keep that number from getting any higher. When I do have time to work on something else, often I have to drop it when all of our alarms go off because our current set of mitigations stopped working. ... it’s not long before we’re complaining about the bots and asking if the other has cracked the code to getting rid of them once and for all."
means that no one has managed it.
BTW isn't the obfuscation of the bots' activity a tacit admission by their owners that they know they're doing something wrong and causing headaches for site admins? In the copyright world that becomes wilful infringement and carries triple damages. Maybe it should be the same for DoS perpetrators.
If they are really just training the same model, and there's no benefit to training it multiple times on that data, then presumably they could use a statistical data structure like https://en.wikipedia.org/wiki/HyperLogLog to be check if they've trained on the site before based on the Last-Modified header + URI? That would be far cheaper than a cache, and cheaper than rescraping.
I was also under the impression that the name of the game with training was to get high quality, curated training sets, which by their nature are quite static? Why are they all still hammering the web?
The clever LLM scrapers sound a lot like residential traffic. How does the author know it's not? Behavior? Volume? Unlikely coincidence?
> random User-Agents that overlap with end-users and come from tens of thousands of IP addresses – mostly residential, in unrelated subnets, each one making no more than one HTTP request over any time period we tried to measure – actively and maliciously adapting and blending in with end-user traffic and avoiding attempts to characterize their behavior or block their traffic.
I host my blog at Mataroa, and the stats show 2K hits for some pages on some days. I don't think my small blog and writings are that popular. Asking ChatGPT about these subjects with my name results in some hilarious hallucinations, pointing to poor training due to lack of data.
IOW, my blog is also being scraped and used to train LLMs. Not nice. I really share the sentiment with Drew. I never asked/consented for this.
I've been trying to write a body of audio code that sounds better than the stuff we got used to in the DAW era, doing things like dithering the mantissa of floating-point words, just experimental stuff ignoring the rules. Never mind if it works: I can think it does, but my objection holds whether it does or not.
Firstly, if you rip my stuff halfway it's pointless: without the coordinated intention towards specific goals not corresponding with normally practiced DSP, it's useless. LLMs are not going to 'get' the intention behind what I'm doing while also blending it with the very same code I'm a reaction against, the code that greatly outnumbers my own contributions. So even if you ask it to rip me off it tries to produce a synthesis with what I'm actively avoiding, resulting in a fantasy or parody of what I'm trying to make.
Secondly, suppose it became possible to make it hallucinate IN the relevant style, perhaps by training exclusively on my output, so it can spin off variations. That's not so far-fetched: _I_ do that. But where'd the style come from, that you'd spend effort tuning the LLM to? Does it initiate this on its own? Would you let it 'hallucinate' in that direction in the belief that maybe it was on to something? No, it doesn't look like that's a thing. When I've played with LLMs (I have a Mac Studio set up with enough RAM to do that) it's been trying to explore what the thing might do outside of expectation, and it's hard to get anything interesting that doesn't turn out to be a rip from something I didn't know about, but it was familiar with. Not great to go 'oh hey I made it innovate!' when you're mistakenly ripping off an unknown human's efforts. I've tried to explore what you might call 'native hallucination', stuff more inherent to collective humanity than to an individual, and I'm not seeing much facility with that.
Not that people are even looking for that!
And lastly, as a human trying to explore an unusual position in audio DSP code with many years of practice attempting these things and sharing them with the world around me only to have Microsoft try to reduce me to a nutrient slurry that would add a piquant flavor to 'writing code for people', I turn around and find Google, through YouTube, repeatedly offering to speak FOR me in response to my youtube commenters. I'm sure other people have seen this: probably depends on how interactive you are with your community. YouTube clearly trains a custom LLM on my comment responses to my viewers, that being text they have access to (doubtless adding my very verbose video footnotes) to the point that they're regularly offering to BE ME and save me the trouble.
Including technical explanations and helpful suggestions of how to use my stuff, that's not infrequently lies and bizarro world interpretations of what's going on, plus encouraging or self-congratulatory remarks that seem partly drawn from known best practices for being an empty hype beast competing to win the algorithm.
I'm not sure whether I prefer this, or the supposed promise of the machines.
If it can't be any better than this, I can keep working as I am, have my intentionality and a recognizable consistent sound and style, and be full of sass and contempt for the machines, and that'll remain impossible for that world to match (whether they want to is another question… but purely in marketing terms, yes they'll want to because it'll be a distinct area to conquer once the normal stuff is all a gray paste)
If it follows the path of the YouTube suggestions, there will simply be more noise out there, driven by people trying to piggyback off the mindshare of an isolated human doing a recognizable and distinct thing for most of his finite lifetime, with greater and greater volume of hollow mimicry of that person INCLUDING mimicry of his communications and interpersonal tone, the better to shunt attention and literal money to, not the LLMs doing the mimicking, but a third party working essentially in marketing, trying to split off a market segment they've identified as not only relevant, but ripe for plucking because the audience self-identifies as eager to consume the output of something that's not usual and normal.
(I should learn French: that rant is structurally identical to an endlessly digressive French expostulation)
Today I'm doing a livestream, coding with a small audience as I try for the fourth straight day to do a particular sort of DSP (decrackling) that's previously best served by some very expensive proprietary software costing over two thousand dollars for a license. Ideally I can get some of the results while also being able to honor my intentions for preserving the aspects of the audio I value (which I think can be compromised by such invasive DSP). That's because my intention will include this preservation, these iconoclastic details I think important, the trade-offs I think are right.
Meanwhile crap is trained on my work so that a guy who wants money can harness rainforests worth of wasted electrical energy to make programs that don't even work, and a pretend scientist guru persona who can't talk coherently but can and will tell you that he is "a real audio hero who's worked for many years to give you amazing free plugins that really defy all the horrible rules that are ruining music"!
Because this stuff can't pay attention, but it can draw all the wrong conclusions from your tone.
And if you question your own work and learn and grow from your missteps to have greater confidence in your learned synthesis of knowledge, it can't do that either but it can simultaneously bluster with your confidence and also add 'but who knows maybe I'm totally wrong lol!'
And both are forms of lies, as it has neither confidence nor self-doubt.
I'm going on for longer than the original article. Sorry.
You put it on the information super highway baby
...with proper licenses.
Here, FTFY, baby.
Addenda: Just because you don't feel like obeying/honoring them doesn't make the said licenses moot and toothless. I mean, if you know, you know.
This has been happening since long before LLMs. Fifteen years ago, my blog would see 3k visitors a day on server logs, but only 100 on Google Analytics. Bots were always scraping everything.
They accomplish this by providing home users with some app that promises to pay them money for use of their connection. (see: HoneyGain, peer2profit, etc.)
Interestingly, the companies selling the tunnel service to companies and the ones paying home users to run an app are sometimes different, or at least they use different brands to cater to the two sides of the market. It also wouldn't surprise me if they sold capacity to each other.
I suspect some of these LLM companies (or the ones they outsource to capture data) some of their traffic through these residential proxy services. It's funny because some of these companies already have a foothold inside homes (Google Nest and Amazon Alexa devices, etc.) but for a number of reasons (e.g. legal) they would probably rather go through a third-party.
Our Wiki periodically gets absolutely hammered by LLM scraper bots, rotating IP addresses like mad to avoid mitigations like fail2ban (which I do have in place). And even when they're not hitting it hard enough to crash the game (through the external data feeds many of the wiki pages rely on), they're still scraping pretty steadily.
There is no earthly way that my actual users are able to sustain ~400kbps outbound traffic round the clock.
I think you cannot distinguish. But the issue is so large that Google now serves captchas on legitimate traffic, sometimes after the first search term if you narrow down the time window (less than 24 hours).
I wonder when real Internet companies will feel the hurt, simply because consumers will stop using the ruined Internet.
After using Claude code for an afternoon, I have to say I don't think this bubble is going to burst any time soon.
if you don't, since you're using a throwaway handle, I'll just assume you're paid to post. it is a little odd that you'd use a throwaway just to post LLM hype.
is that you, Sam?
In practice it only means it is meant to be anonymous and possible to throwaway.
Or it was meant for throwaway but they kept using it.
Machine: 2021 Macbook Pro with M1 Max 32GB
LLMs I usually use: Qwen 2.5 Coder 7B for coding and the latest Mistral or Gemma in the 4B-7B range for most other stuff
For interactive work I still use mostly Cursor with Claude, but for scripted workflows with higher privacy requirements (and where I don't want to be hit with a huge bill due to a rogue script), I also regularly use those models.
If you are interested in running stuff locally take a look at /r/LocalLLaMA [0] which usually gives good insights into what's currently viable for what use cases for running locally. A good chunk of the power-users there are using dedicated machines for it, but a good portion are in the same boat as me and trying to run whatever can be fit on their existing local machine, where I would estimate the coding capbilities to lag ~6-9 months in comparison to the SOTA big models (which is still pretty great).
The few requests I tried were correct, I think that phi4 the 14 b parameters model produced better code though. I don't recall what it was, it was rather simple stuff though.
QwQ seems to produce okay code as well, but with only 40GB of vram I can only use about 8k context with 8bit quantization.
By far the most useful use case for me, is when I want to do something in a repl or the shell, I only vaguely remember how the library or command I am about to use works and just ask it to write the command for me instead of reading through the manual or docs.
Which is extremely dumb, but when the alternatives are “10x capacity just for stupid bots” or “hire a guy whose job it is to block LLMs”… maybe that’s cheapest? Yes, it sucks for the open web, but if it’s your livelihood I probably would consider it.
(Either that or make following robots.txt a legal requirement, but that feels also like stifling hobbyists that just want to scrape a page)
A legal requirement in what jurisdiction, and to be enforced how and by whom?
I guess the only feasible legislation here is something where the victim pursues a case with a regulating agency or just through the courts directly. But how does the victim even find the culprit when the origin of the crawling is being deliberately obscured, with traffic coming from a botnet running on exploited consumer devices?
The VPNs and legal botnets would be heavily incentivized to not allow this to happen (and presumably already are doing traffic analysis), and illegal botnets should be shutdown anyway (some grace in the law about being unaware of it happening should of course be afforded, but once you are aware it is your responsibility to prevent your machine from committing crimes).
Illegal botnets aren't new. Are they currently shutdown regularly? (I'm actually asking, I don't know)
> If we made not following robots.txt illegal in certain jurisdictions, and blocked all IP addresses not from those jurisdictions
That sounds kinda like the balkanization of the internet. It's not without some cost. I don't mean financially, but in terms of eroding the connectedness that is supposed to be one of the internet's great benefits.
> make following robots.txt a legal requirement
Even better: make them publish their IP addresses so we know what to block, just like robocallers. Every scrape made from another IP is crime.
The people stealing and ddos-ing literally do not care. Nobel prizes are handed out for this crap. Politicians support the steal.
Thanks Drew for being an honest voice.
Alternatively, a technique like Privacy Pass might be useful here. Basically, give any user a handful of tokens based on reputation / proof of work, and then make each endpoint require a token to access. This gives you a single endpoint to rate limit, and doesn’t require user accounts (although you could allow known-polite user accounts a higher rate limit on minting tokens).
Would you be rate-limiting by IP? Because the attacker is using (nearly) unique IPs for each request, so I don't see how that would help.
> someone recently built a simple proof of work CAPTCHA for their personal git server
As much as everyone hates CAPTCHAs nowadays, I think this could be helpful if it was IP-oblivious, random and the frequency was very low. E.g., once per 100 requests would be enough to hurt abusers making 10000 requests, but would cause minimal inconvenience to regular users.
The nice thing about the proof-of-work approach is that it can be backgrounded for users with normal browsers, just like link prefetching.
Obviously you can do this, but that will start blocking everyone. How is that a solution?
Also what prevents attacker-controlled browsers from backgrounding the PoW too?
I took it as read that for something to qualify as a solution to this problem, it needs to affect regular users less badly than attackers.
Two corrections: It will start rate-limiting (not blocking) anonymous users (not everyone). It's a mitigation. If specific logged-in users are causing problems, you can address them directly (rate limit them, ban them, charge them money, etc). If nonspecific anonymous users are causing problems, you can limit them as a group, and provide an escape hatch (logging in). If your goal is to "free access to everyone except people I don't like but I can't ask people if they are people who I don't like", well, I suppose it isn't a good mitigation for you, sorry.
> Also what prevents attacker-controlled browsers from backgrounding the PoW too?
The cost. A single user can complete the proof of work in a short period of time, costing them a few seconds of CPU cycles. Scaling this up to an industrial-scale scraping operation means that the "externalized costs" that the OP was talking about are duplicated as internalized costs in the form of useless work that must be done for you to accept their requests.
Ah. Yes, the part in quotes here is what I think would count as a solution -- I've been assuming that simply steering anonymous users towards logging in would be the obvious thing to do otherwise, and that doing this is unacceptable for some reason. I was hoping that, despite attackers dispersing themselves across IP addresses, there would either be (a) some signal that nevertheless identifies them with reasonable probability (perhaps Referer headers, or their absence in deeply nested URL requests), or (b) some blanket policy that can be enforced which will hurt everyone a little but hurt attackers more (think chemotherapy).
> It will start rate-limiting (not blocking) anonymous users (not everyone).
If some entities (attackers) are making requests at 1000x the rate that others (legitimate users) are, the effect in practice of rate-limiting will be to block the latter nearly all the time.
> Scaling this up to an industrial-scale scraping operation
My understanding was that the PoW would be done in-browser, in which case this doesn't hold -- the attackers would simply use the multitudes of residential browsers they already control to do the PoW prior to making the requests, thus perfectly distributing that workload to other people's computers. What kind of PoW cannot be done in this way?
I could be mistaken, but I don't think these residential VPN services are actual botnets. You can use the connection, but not the browser. In any case, you can scale the work factor as you want, making "unlikely" endpoints harder to access (e.g. git blame for an old commit might be 100x harder to prove than the main page of a repository). This doesn't make it impossible to scrape your website, it makes it more expensive to do so, which is what the OP was complaining about ("externalizing costs onto me").
All in all, it feels like there's something here to leverage proof of work as a way to maintain anonymous access while still limiting your exposure to excessive scrapers. It probably isn't a one-size-fits-all solution, but with some domain-specific knowledge it feels like it could be a useful tool to have in the new internet landscape.
Fair enough, that would likely be the case if they're using "legitimate" residential IP providers, and in that case they would indeed need to pay for the PoW themselves somehow.
The ARPANET was never meant to be commercial or private. All the protocols developed for it were meant to be subsidized by universities, the government or the military, with the names, phone numbers, and addresses of anyone sending a packet being public knowledge to anyone else in the network.
This made sense for the time since the IMPs used to send packets had less computing power than an addressable LED today.
Today the average 10 year old router has more computing power than was available in the whole world in 1970, but we've not made any push to move to protocols that incorporate price as a fundamental part of their design.
Worse is that I don't see anyway that we can implement this. So we're left with screeds by people who want information to be free, but get upset when they find out that someone has to pay for information.
Would be nice to see some effort on this front, a la "we don't scrape", or "we use this reputable dataset which scrapes while respecting robots.txt and reasonable caching", or heaven forbid "we train only on a dataset based on public domain content"
Even if it was an empty promise (or goodwill ultimately broken) it'd be _something_. If it exists, I'd certainly prioritise any products which proclaimed it
Do we need to make one?
Also - Honestly don't even understand why so many people would need to scrape for training data.
Is it naïveté (thinking it necessary), arrogance (thinking it better than others, and thus justified)?
Aren't most advances now primarily focused on either higher level (agents, layered retrieval) or lower level (eg. alternatives to transformers, etc.. which would be easier to prove useful on existing datasets)?
Genuine questions, all of these - if I'm off the mark I'm keen to learn!
My "main" projects I want to keep public are moved/cloned to GitHub, the rest I just don't bother with any longer.
Does this statement have ramifications for sourcehut and the type of projects allowed there? Or is it merely personal opinion?
> This blog post is expressing personal experiences and opinions and doesn’t reflect any official policies of SourceHut.
I'm seeing this as well. Some of the websites my company operates suddenly go through some 10-25x in requests per minute at night. Most often the AS where the ip comes from is from some Microsoft datacenter (meaning, it's most likely OpenAI).
At this point i'm starting to consider blocklisting Microsoft AS numbers and/or requiring login (or some kind of api key) when coming from a datacenter network. Reddit does this as well, already, for example.
> If you personally work on developing LLMs et al, know this: I will never work with you again, and I will remember which side you picked when the bubble bursts.
This sounds... childish (to put it politely) ?
Once the allmende, the grass runs out, things get interesting. We shall see Cyperpunk like computational parasitism plaguing companies and attempts to filter these work out. I guess that is the only way, to really prevent that sort of bot. You take arbitrary pieces of the UOW they want done- and reject them on principal. And depending on the cleverness of the batch algorithm, they will come back with that again and again, identifying themselves via repetition.
Most bot nets, like he mentions in the blag post, come from 1 IP address only 1 time. With thousands to tens of thousands of IP addresses in the bot net, there is just no way to block by IP address anymore.
The culture of "fuck you got mine" is something i despise on a very deep level, not just including ai but most big tech shitfuckery that has been going on since forever
But the “bubble” threat the post mentions is just emotional; things are accelerating so quickly that what is hype one day isn’t hype in a few months. LLM usage is just going to get heavier.
What should get better are filters that prevent bots from easily scraping content. Required auth could work. Yes, it breaks things. Yes, it may kill your business. If you can’t deal with that, find another solution, which might be crowdfunding, subscription, or might be giving up and moving on.
Working towards a solution makes more sense than getting angry and ranting.
Creating a solution requires cooperation. Making these models fit on smaller systems, optimizing them to work with less resources need R&D engineering, which no company wants to spend on right now. Because hype trains need to be built, monies need to be made, moats need to be dug (which is almost impossible, BTW).
My 10 year old phone can track 8 people in camera app in real time, with no lag. My A7-III can remember faces, prioritize them, focus on mammals' and humans' eyes, track them and keep them in focus at 30FPS with a small DSP.
Building these things is possible, but nobody cares about it right now, because AI Datacenters live in a similar state of ZIRP economy. They're almost free for what they do, even though they harm the environment in irreparable ways just to give you something which is not right.
As if people lost their mind about this thing.
With LLMs come a promise of safety in that they will solve our problems, potentially curing disease and solving world hunger. They can sound like people that care and make us feel respected and useful, making us feel like we belong and boosting our esteem. They do things we can’t do and help us actualize our dreams, in written code and other things. We don’t have to work hard to use them anymore, and they’re readily available.
So, people are going to focus on that.
We have been using models, and accurate ones at that, for drug discovery, nature simulation, weather forecasting, ecosystem monitoring, etc. well before LLMs with their nondescript chat boxes have arrived. AI was there, the hardware was not. Now we have the hardware, AI world is much richer than generative image models and stochastic parrots, but we live in a world where we assume that the most noisy is the best, and worthy of our attention. It's not.
The only thing is, LLMs look like they can converse while giving back worse results than these models we already have, or develop without conversation capabilities.
LLMs are just shiny parrots which can taunt us from a distance, and look charitable while doing that. What they provide is not correct information, but biased Markov chains.
It only proves that humans are as gullible as other animals. We are drawn to shiny things, even if they harm us.
Do you also think E-Mail is fundamentally tainted because spam is widespread? [2]
[1] https://bitcoin.org/bitcoin.pdf
[2] https://www.emailtooltester.com/en/blog/spam-statistics/
Which is ugly in so many levels.
It sounds reasonable, and i'm not joking
I'm constantly looking for some way that I can put everything back up, but nothing has presented itself yet. Plus, if OpenAI's attempts at legalizing what they do are successful, it seems unlikely there will ever be an adequate solution.
You might see a couple of small players or especially egregious executives get a slap on the wrist for bad behavior but in this political climate there's no chance that Republicans or Democrats will put a stop to it.
Assuming that established companies are automatically ethical and just is not correct. Meta used a "laptop and a client tuned to seed as minimally as possible to torrent 81TBs of copyrighted material" to train their models.
For every picture perfect and shiny facade, there's an engineer sitting on the floor in corner and hacking things to prop up this facade.
Frankly, as a person committing copyright infringement 24/7 I have no problem with that. Especially with them releasing the weights.
I'm sure you'll have no problem with that, either.
No.
> What problem would you have with me doing that with your comment?
Nothing.
Addenda:
However, while I'll be putting that comment to my favorite screenshots folder, and letting it adorn my cubicle for some time, there's also an implicit meaning. With that wording, the subscript is "That's brave". [0]
It's not meant to be an insult, but to signify a greatly different point of view compared to OP (me, in this case).
While the tone may came across as angry, slightly agitated is a more correct measurement of my feelings.
In either way, it's fair game and no hard feelings. Also, the answer from the original commenter is good sport. Thanks mate!
[0]: https://english.stackexchange.com/questions/574974/etymology...
I'm one of these people, and I'm not visiting or contributing to Reddit anymore. My logic is simple: "Ask me first". I may say yes, I may say no, but ask me first.
I'm bullish about ethics and transparency. I work on FAIR Data at work, as a side quest. Transparency, consent and licenses are a big part of it. Honoring it is important at every level. Both ethically, and both for integrity reasons.
As a result, when someone does something behind me, and if they didn't ask me first, I leave when I find out. I was this close to abandoning Go when they first declared that the telemetry will be opt-out. Now it's opt-in, and so I still use Go.
It's not that I don't send usage data, either. Some applications and parties I trust get telemetry from me, but all of them were opt-in and asked me first.
In short, a bit of decency and agency goes a long way in my eyes.
Stop defaulting to expecting honor from businesses.
Unless you're clutching pearls for dramatic effect, any surprise at all is an indication your orientation is wrong.
No point in taking effort to recreate what's already out there as authentic not-goodery :)
>Please stop legitimizing LLMs or AI image generators or GitHub Copilot or any of this garbage. I am begging you to stop using them, stop talking about them, stop making new ones, just stop.
Anyone with any inkling of the current industry understands this is /Old-Man-Yells-At-Cloud. Why offer this as a solution even, I'm unsure.
Here's potential solutions: 1) Cloudflare. 2) Own DDoS protecting reverse proxy using captcha etc. 3) Poisoning scraping results (with syntax errors in code lol), so they ditch you as a good source of data.
Here' my pragmatist comeback: Is your preferred solution to lack of democracy in middle-east to go and protest in front of the palace?
Up until this point you had a good start to a very inspiring story. ;)
I carry a laptop almost everywhere, and I even use JMP.chat for sending SMS messages, so it's functionally similar to a mobile phone, just bigger and heavier. It doesn't connect to the cell network but Wi-Fi works fine.
I think there's more to a mobile phone than SMS but I'm stuggling to think of what since I don't use it. I have GPS in my car, instead of using GPS in a mobile phone. I don't take pictures very often but when I really needed to I could use my laptop webcam (built-in or USB attached), and recently I bought a camera at a garage sale that seems higher-quality than the webcam and is much more convenient.
The ability to call 911 from anywhere is the main thing I think about that I'm missing, which is also one of the reasons I'd like to get a mobile phone working at some point, but it's difficult to gauge how important that is (since the need is rare). My friends might worry about this more than I do.
Before college, I did use a mobile phone (but mostly for gaming and taking pictures), and I am only recently out of college, so maybe I will feel more pressure to get a mobile phone in the future, and since I'm not opposed to getting one working at some point, I probably will. I just haven't gotten around to it yet.
I've never actually commented on Hacker News as far as I remember, but I'm the same JacobK as <https://codeberg.org/jacobk> and as jacobk on Libera.Chat. I'm not a bot.
I read this: "I am sick and tired of having all of these costs externalized directly into my fucking face", as: "I am sick and tired of doing something for the world and the world not appreciating it because they want to do their own things".
Lack of appreciation is underrated as a problem because it's abstracted away due to the mechanics of the free market ("if you don't want it, just don't buy it"). Yet, markets are quite foundational in humanity's activities and a market is place where everybody can put a stall. Now, Drew can't have his stall and he's angry. But anger is rarely part of the solution.
His path is that of anger, unfortunately, with some hate added in too:
> "If you personally work on developing LLMs et al, know this: I will never work with you again, and I will remember which side you picked when the bubble bursts."