"I’m told that OpenAI’s newish o1 model is genuinely helpful and creative when it comes to thinking through open problems in the sciences..."
"Likewise, although my knowledge of Italian is not great, I can read it well enough to confirm that the translation it offers is good enough to use for research:"
Using a translation for research that you couldn't perform yourself seems extraordinarily substandard for a historian.
The only part I agree with is a simple search to identify sources that may be relevant that you had not considered. i.e. Primary sources to be examined directly -- the way historians have done it for millennia.
I don't think history should be filtered through model hallucinations. It seems an invitation to mistakes.
The reference to a "medallion, or seat of humors, or badge of office" is obviously something being held. The historian specifically says it is not any of these, but a "urine flask." A historian obviously should not take an LLM model as factual.
Later, the author writes, "After all: when you get down to it, o1 talking about a panopticon and Foucault in the above snippet is very, very similar to what a first year history PhD student might produce."
This is exactly the point. A mediocre average of writing that a first year student would produce. Sure it could be used for "I hadn't considered that," but it surely should not be used for any factual interpretation.