GPT-[0-9]+, DALL-E, "AI".
ChatGPT: Produce a blog post in the style of [Given Name] [Surname]
From NCSA Mosaic 2.7, circa 1993, one of the first "web browsers", authored in part by one of Silicon Valley's favourite VC, the "a" in az16.com.
#
# Agent Spoofing -- Who will you be today?
#
# "I don't think [t]he[y]'ll be too keen" -- Originally from Monty Python, run
# into the ground by Tommy Reilly
#
# NOTE! There is a hard limit of 50 agents! Mosaic will not read anymore!
#
# This file should be located in your HOME directory under the filename:
# .mosaic-spoof-agents
#
# This file consists of lines that begin with a # (comment), a - (a seperator
# for the sub-menu off options), a + (this is special...it denotes that the
# agent following it [no spaces] should be the default agent to use) and
# anything else (considered an agent spoof line).
#
- This is a seperator...as long as the first thing is a dash
# This is a comment line
#
# This is the "example" agent. If you don't like it...comment it out.
# if you want it to be your default, put a + infront of it.
#
Bond_James_Bond/007 (BMW; Z3 Roadster) DOHC_inline_16-valve_4-cylinder
Gandalf/White (Shadowfax; Wind) White_Staff
Elric/Emperor_of_Melnibone (Hellsword; StormBringer - Devourer of Souls)
The sad reality is that "tech" companies and their service providers routinely use HTTP headers for their own manipulative, deceptive and user privacy-hostile commercial purposes. They use UA headers in ways that were not contemplated in 1993 and probably never intended. (Note I am not referring to spoofing, I am referring to data collection, tracking, e.g., "browser fingerprinting", and "surveillance capitalism" in general.)
Sending a UA header generally offers me, the end user, no benefits, so I do not send one.
Why not ban those Microsoft IPs or at least rate-limit them to a limited number of HTTP requests in a given time period.
Also looks like this website does not use sitemaps. At least, there is nothing listed in robots.txt. IMHO, there is no need to spider/crawl if there are sitemap XML files listing all the content URLs.