'Obviously ChatGPT' – how reviewers accused me of scientific fraud
nature.com
nature.com
It probably won't be too long before someone creates a tool to create fake Git/Github entries to "prove" a document was written by a human after it was actually ChatGPT-generated.
Edit. Btw, you can use Git with Word's docx-format out of the box. Docx is a zip folder with XML inside; git-diff will show text changes. Further, there's git-merge-driver for merging various file formats (https://github.com/Praqma/git-merge-driver).
I'm probably just not smart enough but I can't figure out how to truly prove your (human) authorship. To some extent though I'm also not sure how much it matters. If the science is sound, the text is well written, and the paper is well presented, why do I care who wrote the text? I didn't handwrite the graphs or the tables of data, that was a script. Nor did I typeset every letter, that was LaTeX. The text isn't even any different in nature, like the difference between an original oil painting and a print of the same. It's ultimately just a means to communicate ideas from my mind to yours. Perhaps it is ethical to disclose the use of generative AI, but I'm not convinced that would be any better in the minds of the reviewers in question here. I just don't see a substantiative difference if the end product ultimately is high quality.
For what it's worth, I wasn't a great student (good at learning, yes, good at going through the motions to get good grades, not so much) and despite what some might call an academic approach to most problems I'm not really a fan of academia itself. There's a lot of stuffy old practices, like p-value being the metric for so many things despite the knowledge that it can be "hacked", and weird, unknowable financial or political motives that you'll never be privy to, which in my opinion compromise the integrity of the work more than use of generative AI does while also being harder to detect. I'm certainly not the first to point these issues and I won't be the one to solve it. I do wonder though if the whole thing is kinda broken, ChatGPT or not.
The paranoia we are seeing now will slowly fade. It doesn't make sense to adopt extreme measures, especially since the truthiness of any claim as to creative originality becomes more dubious with every day.
Even if you hand-wrote and mailed a copy of your manuscripts one page at a time to the US Copyright Office, that would not suffice, since from the moment ChatGPT was made public, you could have been using it, then transfering your thoughts to paper.
Just live your life.
I’ve had the experience so much on various forums I now have to write “This is not GPT” on most posts to try to preempt the trolls (surprisingly this works).
There is a need to be able to source validate the work, and see the creation of things in order to validate their originality already, and we are barely into the AI era.
This obviously leads to a deep breakdown in trust.
I suspect the next step will be some kind of Real ID that allows everything you post to be validated and sophisticated technologies to try to counter AI, but it will be in vain as the vast majority of content on the Internet will become AI source generated. It’s an arms race and one side has geometric math advantages.
Perhaps eventually, in weird comfort, it will become stylish and trendy to collect “human written” things, like the vinyl records people collect where fully half of the people collecting have no ability to play them.
So too, I’m sure we will go full circle with this technology, which will eat and destroy credibility for original work and cast doubt in anything that can be generated, as it becomes so good it cannot be separated from human efforts.
https://consequence.net/2023/04/half-vinyl-buyers-record-pla...
This is what worries me the most about AI (and large language models in particular). You can summarily dismiss someone's work by simply claiming it was written by ChatGPT.
It's not solely being judged on the basis of the writing style, and the author alludes to this in the first paragraph: "I have just been accused of scientific fraud. Not data fraud — no one accused me of fabricating or misleadingly manipulating data or results. This, I suppose, is a relief".
I think there is an art to communicating complex ideas clearly, and some people value that ability, and don't like having their rhetorical ability discredited. There are certain scientists who I appreciate almost as much for their ability to communicate science to the public as I do for their ability to conduct scientific research. Richard Feynman for example had an uncanny ability to convey scientific concepts.
There's a separate argument about potentially unknowingly copying someone's exact words. But beyond that, it's a lazy criticism that's common on the internet. You could mentally replace "it's GPT" with "it's lame". Ok - but why does the person think it's lame.
What bothers me is the idea that a talented digital artist could spend days crafting a photorealistic painting only to have someone rob them of the credit they're due by attributing the work to Midjourney.
It bothers me that a gifted writer could spend hours painstakingly deliberating over the best way to convey an idea only to have the credit for the prose given to OpenAI.
This is basically the sentiment that the author of this article was expressing: "That hurts, because — like many people — I find writing a paper to be a somewhat painful process. I read books on how to write — both to be comforted by how much these books stress that writing is generally slow and difficult, and to find ways to improve. My current strategy involves willing myself to write and creating several outlines before the first draft, which is followed by writing and a lot of revising."
People generally don't enjoy working hard on something and having the credit stolen by another entity (regardless of whether or not that entity is a machine). I don't have a problem with people using Github Copilot to write code for example, but if I personally write my code by hand it's infuriating if someone gives the credit to Github's AI.
I try to impress you with my ability to look and create photo like images. It takes me a few days to do each one. Camera takes a few seconds to do the same thing. Should I be offended when you tell me I must have used a camera to make mine? How much "credit" should my skill get? What value is my skill in the age of cameras?
So do you think we should just shut down museums, put the Mona Lisa in a shredder, and just discontinue art as a whole since it all apparently has no value? Also, what happens when AI and robotics can do everything that a human being can? Are we an obsolete species with no value when that time comes? I'm legitimately curious about how you see the world.
You misunderstand me.
> So do you think we should just shut down museums, put the Mona Lisa in a shredder, and just discontinue art as a whole since it all apparently has no value?
There are different types of value: scientific value, historic value, economic value, sentimental value, costly signaling value. Also wealth portability / wealth hiding / tax evasion value. Art has more than one type of value.
> I'm not going to argue against misanthropy, but I am curious about the implications of what you're saying.
When you are unwell do you seek medicine? What does medicine say you are? Are you not a biochemical machine?
> Also, what happens when AI and robotics can do everything that a human being can?
Despite using machines for transport and moving stuff many of us use treadmills and lift weights ditto many still play chess despite machines mastering that game. Why might that be?
> Are we an obsolete species with no value when that time comes?
Today our minds exist in one form. In the future, might our minds exist in other forms? If you had the power to upgrade various parts of yourself with better parts would you reject this? Are we the end state of evolution?
I (and many others) value art not only for its economic, historical, and scientific value, but because I admire the ability of exceptional human beings. I want to see Michelangelo's statue of David and be awestruck at how a human being could carve the 17 foot (5 meter) tall likeness of a man out of a solid block of marble with just a hammer and chisel. If you believe human produced art is valuable then you should care about verifying whether or not a human actually produced it. You don't have to value the same things I do, that's just my opinion.
> many still play chess despite machines mastering that game
This is a perfect example. Magnus Carlsen suspected that his opponent Hans Niemann cheated by using AI to assist him in a chess tournament. This was a scandal precisely because people value the human element of chess. If the provenance of work didn't matter, people wouldn't care whether or not Niemann was using AI. Just like the author of the article didn't enjoy having her work credited to a chatbot, Niemann probably didn't enjoy having his work credited to a chessbot (if he's indeed innocent).
> When you are unwell do you seek medicine? What does medicine say you are? Are you not a biochemical machine?
I'm not sure what this passage accomplishes besides a redefinition of terms. Despite all my talk about the value of communication — I'm not actually interested in pedantic quibbling over English semantics. If it really bothers you, feel free to mentally replace all instances of the word "machine" with "non-biological machine" in my writing.
> Today our minds exist in one form. In the future, might our minds exist in other forms? If you had the power to upgrade various parts of yourself with better parts would you reject this?
No, I'm not a transhumanist. I believe in using technology to help people with differing abilities (e.g. I'm fine with cochlear implants), but I'm not planning on amputating my limbs to replace them with robotic prosthetics. I'm one of those millions of weird humans who will probably tune in to the Super Bowl today, despite the fact that machines with wheels could probably outclass the human competitors.
Say you learn the feat was done using some clever use of mechanical 3D tracing or casting and chemicals that rapidly dissolve the rock and took no effort at all just knowledge of the trick. Now you would no longer be awestruck? Value for you is in the difficulty not the result?
Btw: Highly recommend “Tim's Vermeer” documentary about old mechanical art technique being reverse engineered:
> If you believe human produced art is valuable then you should care about verifying whether or not a human actually produced it.
Do you not question whether the enormous amount of time and effort various historical works cost was well spent? What if that time and effort was instead spent researching something useful? Does the principle of compound interest not apply to useful knowledge? What happens with compound interest over long time scales? What might our lives be today if instead of placing value on paintings / carvings / etc those before us placed more value on discovery and preservation of useful knowledge?
> This was a scandal precisely because people value the human element of chess.
It was a scandal because many use costly signals (ability to play sports / games or create or acquire stuff like “art”) to rank each other in hierarchies. What matters is the ranking. If the result mattered, anything that improves the result would be welcome, not rejected as “cheating”. Are science research / publications such an activity? For some, yes. For others, no. How do you see it?
> Just like the author of the article didn't enjoy having her work credited to a chatbot, Niemann probably didn't enjoy having his work credited to a chessbot (if he's indeed innocent).
AI assisted writing when it looks like AI assisted writing is often “empty” -- it has the right look but lacks substance. If the author of the article was told her work looked like a high schooler did it that also may have upset her.
If instead the author was told it was the work of a genius and not hers that may actually be accepted as a compliment -- if she maintains that high quality in the future what else can that be but an honest compliment on the quality of her work?
>> When you are unwell do you seek medicine? What does medicine say you are? Are you not a biochemical machine?
> I'm not sure what this passage accomplishes besides a redefinition of terms.
You said above: “I'm legitimately curious about how you see the world.” The passage above reveals how I see the world. Many see a sharp difference between “human” and “machine” yet the medicine we rely on assumes we are machines. I think many know this but do not realize the implications. To think about it can be uncomfortable.
>> Today our minds exist in one form. In the future, might our minds exist in other forms? If you had the power to upgrade various parts of yourself with better parts would you reject this?
> No, I'm not a transhumanist. I believe in using technology to help people with differing abilities (e.g. I'm fine with cochlear implants), but I'm not planning on amputating my limbs to replace them with robotic prosthetics.
Your attitude I suspect is because you do not see you already are upgraded from what you would be otherwise. Your mind has been upgraded with language and is full of useful concepts and techniques that are not your own. To aid communication and memory you rely on speech and writing for information. You are surrounded by technology that upgrades you (and your life) but it is invisible to you and you do not think about it much.
No, if I found out that the sculpture was 3d printed instead of sculpted by hand I wouldn't be as impressed. The value for me is derived both from the difficulty and the result. The result itself is impressive — if I saw a 17 foot tall statue towering over me I would be a bit impressed regardless of the methods used to produce it, but I also find it impressive imagining a man operating under the constraints of his tools, and the skill those constraints require.
> https://youtu.be/WPL7D0Ha1kQ
It's a cool thought experiment to imagine the Dutch Masters using a camera lucida to trace their paintings. If Vermeer traced his paintings I would indeed be less impressed with his work. I feel like we're going in circles restating the same points over and over again.
> It was a scandal because many use costly signals (ability to play sports / games or create or acquire stuff like “art”) to rank each other in hierarchies. What matters is the ranking. If the result mattered, anything that improves the result would be welcome, not rejected as “cheating”.
In the example we're discussing what is the meaningful "result" of a game of chess? It's literally a board game where you move small objects across a tiled surface for entertainment. To me you sound like someone saying "if people cared about the result of a foot race, they would let the runners drive cars". You've gone so far down the rabbit hole of defending AI that you're trying to undermine the idea of sports and games.
> Do you not question whether the enormous amount of time and effort various historical works cost was well spent? What if that time and effort was instead spent researching something useful?
How much time and effort have you spent trying to persuade me that art, sports, and games are frivolous? Maybe your time would be better spent researching something useful rather than writing these comments.
> Are science research / publications such an activity? For some, yes. For others, no. How do you see it?
I must have said this at least 5 or 6 times at this point, but I don't mind someone using AI to help them write a scientific paper. I just want creators who don't use AI to be credited for their work.
> If instead the author was told it was the work of a genius and not hers that may actually be accepted as a compliment -- if she maintains that high quality in the future what else can that be but an honest compliment on the quality of her work?
It could be considered a way of dismissing someone's effort and talent. If for example reviewers only questioned women about whether or not they were plagiarizing geniuses, I don't imagine they would find that very flattering.
> Many see a sharp difference between “human” and “machine” yet the medicine we rely on assumes we are machines. I think many know this but do not realize the implications. To think about it can be uncomfortable.
The idea that you can use the English word "machine" to describe a human being is not the deep philosophical insight that you perceive it to be. There's nothing uncomfortable about it, because you're just playing a language game. A doctor would have no problem treating your illness without referencing the word "machine". This would maybe be an uncomfortable truth to an undergrad taking their first philosophy course.
> Your attitude I suspect is because you do not see you already are upgraded from what you would be otherwise. Your mind has been upgraded with language and is full of useful concepts and techniques that are not your own.
I'm fully aware of the fact that human beings are already being upgraded. I specifically gave you the example of cochlear implants in my previous comment to indicate my awareness, which are a much more concrete example than your abstract "language, concepts, and techniques" example. I know these concepts feel revolutionary to you, but they're not to me.
We've gone astray here but the topic of human upgrades came up because I asked you a simple question: if you only value the results of a process, rather than the human who derived the result, then what happens when humans are no longer required to produce any result? Do you propose that the only way we can stay relevant in such a society is to upload our minds to AWS where hardware can make us more intelligent than our wetware? I don't know about you, but I don't plan on becoming a brain in a vat in some datacenter, and I definitely don't plan on becoming a file on someone's Digital Ocean droplet. We might just have different preferences.
“Labor Theory of Value” vs “Marginal Utility Theory of Value”: you are familiar with these yet you still hold that view? The output of a less efficient process to create something comparable is more valuable _because_ a less efficient process was used?
> If Vermeer traced his paintings I would indeed be less impressed with his work.
The excellent https://youtu.be/WPL7D0Ha1kQ documentary presents evidence that he did use tracing. In contrast to you that evidence made me more impressed with Verneer’s work.
> In the example we're discussing what is the meaningful "result" of a game of chess?
I answered that. Above I said “many use costly signals (ability to play sports / games or create or acquire stuff like “art”) to rank each other in hierarchies”.
GPT4 explains: ”From an evolutionary standpoint, games and sports can be seen as extensions of primal competitions for resources, mates, and status. Competing and displaying skills in a controlled environment can serve as a non-lethal way to establish dominance and hierarchy within a group. This is similar to how animals engage in play-fighting or displays of strength to determine pecking orders without resorting to serious conflict.”
What evolutionary purpose do you think games and sports serve if not the purpose above? It seems to me you like the “labor theory of value” because for you creating things is like a sport or game where ranking is important and difficulty of the task matters and innovation is “cheating” for the purpose of ranking.
> How much time and effort have you spent trying to persuade me that art, sports, and games are frivolous? Maybe your time would be better spent researching something useful rather than writing these comments.
Our chat is like a dialogue I have with myself (or with chatGPT) when I debate views to make up my mind about them: A joint search for truth. When a smart person (as you obviously are) changes my views on an important topic I am obviously better for it. And even if you do not change my mind, the thinking exercise is valuable. Thank you!
> I just want creators who don't use AI to be credited for their work.
Why would creators who do not use AI not get credit for their work? When you say the above you mean they should get extra credit? And would you give them even more extra credit when they did all their research without the help of any machines? Why does it matter if machines helped? Why not judge the research publication on its own merits? You want extra credits to handicap those using better tools because research is a game in which the difficulty must be maintained otherwise it is “cheating”?
> The idea that you can use the English word "machine" to describe a human being is not the deep philosophical insight that you perceive it to be. There's nothing uncomfortable about it, because you're just playing a language game. A doctor would have no problem treating your illness without referencing the word "machine". This would maybe be an uncomfortable truth to an undergrad taking their first philosophy course.
Your mind perhaps rejects the idea that why you think I am playing a language game using a metaphor when I am not using a metaphor. For you there is nothing uncomfortable accepting that living things are machines? I think you have not thought about the consequences of accepting this view: https://chat.openai.com/share/2fdd8863-6a3b-4a09-9d9c-b2f196...
> if you only value the results of a process, rather than the human who derived the result, then what happens when humans are no longer required to produce any result?
Our lives will likely change much like our lives of wandering around from place to place hunting gathering changed and then lives of subsistence farming changed, etc. If you accept the view that living things are machines (not as a metaphor but in reality) the concept of computational universality may give you a clue about our likely future. GPT4 explains: https://chat.openai.com/share/4dd5e6b3-6136-408a-a05a-10261d...
> I don't know about you, but I don't plan on becoming a brain in a vat in some datacenter, and I definitely don't plan on becoming a file on someone's Digital Ocean droplet. We might just have different preferences.
Assuming it is possible: How do you feel about having observations of what you say and write and do being used to create a model that can predict these things about you with such accuracy that those that know you can not tell the difference? Will that model in a “computational sense” not be you? Say, if given a matching body, that model can replace you such that those that know you cannot tell the difference. Will that also not be you? If you accept the reality that you are a machine, why will that not also be you? Because you are the atoms that make you you?
This is one of those comments that appears quite witty at first glance, but crumbles upon examination. First of all, I didn't say I value meticulously crafted art such as Michelangelo's statue of David because a _less efficient_ process was used to create it. I said I value it because of the _skill_ involved in creating it. As a human being, I naturally admire other skilled human beings. The reason why Michelangelo had to cultivate such a skill is obviously because of the state of technology during his lifetime, but that's completely beside the point. I'll take your comment in good faith and assume you truly misunderstood.
Second, macroeconomic theory is best left in my opinion to goods and services that are fungible. So for example, you could use macroeconomic theory to explain the price of a commodity such as pure gold, because any pound of gold is just as valuable as any other pound of pure gold. Art is inherently non-fungible though. Not only are two similar pieces of art not equal in value, but two identical pieces of art are not identical in value (e.g. a forgery or replica isn't as valuable as the original). Even the crypto bros understand the difference between fungible and non fungible tokens. You could argue about why it is that humans value the non-tangible aspects of art, but it's a key component of our psyche. Valuing these aspects doesn't make you a Marxist, or an adherent to the Labor Theory of Value. Some of the most hardcore marginal utilitarians would find value in the original Declaration of Independence as opposed to a forgery for example.
> The excellent https://youtu.be/WPL7D0Ha1kQ documentary presents evidence that he did use tracing. In contrast to you that evidence made me more impressed with Verneer’s work.
I would be more impressed by the person who supposedly invented the camera lucida almost two hundred years before any written or physical evidence of it appeared.
> I answered that.
No you provided an answer to the wrong question. You said in an earlier comment "If the result mattered, anything that improves the result would be welcome, not rejected as “cheating”." In this case the result being the outcome of a game of chess. What you are describing in your most recent comment is the motivation that drives people to sit at the chess board in the first place, which is a completely orthogonal topic.
What I'm trying to get you to explain is how AI could possibly "innovate" once the competitors are sitting at the board. This is why I said you sound to me like someone suggesting that it would be "innovative" to allow cars in a foot race. Follow your own logic via reductio ad absurdum and see where it leads you. Forget AI, if we wanted to "innovate" in competitive games we'd allow adults to compete in children's sporting competitions, surely that would "improve the result"? I can't believe I'm explaining the concept of games to a fellow human being. Games are not primarily a science problem to be solved (or "innovated" upon) by a computer, they're a form of leisure. I've never had to ask this before, but are you a human being?
Also, what about games that have been completely solved by computers such as checkers? We literally know what a perfect game of checkers looks like. If we used your method of "innovation" every game of checkers would be identical, because each participant would simply play the predetermined perfect game as calculated by a computer. When computers inevitably solve chess do you propose that we discontinue the game altogether since there's no longer any innovation left to be had?
> It seems to me you like the “labor theory of value”
You're vastly over-applying this principle. When you discover a new theory it's tempting to apply it to everything. The profession of economics is not in the business of deciding whether or not AI should be allowed in board games.
Thank you for the compliment. It's a nice change of pace here on HN where everyone is convinced of their own intellectual superiority.
> Why would creators who do not use AI not get credit for their work? When you say the above you mean they should get extra credit? And would you give them even more extra credit when they did all their research without the help of any machines? Why does it matter if machines helped? Why not judge the research publication on its own merits? You want extra credits to handicap those using better tools because research is a game in which the difficulty must be maintained otherwise it is “cheating”?
Two things:
First, let's not conflate two separate issues here. Something can be a truly abysmal piece of work and I can still credit the creator of it. We often use the word blame to describe credit given in the negative sense, but it's the exact same concept. See for instance git blame: https://git-scm.com/docs/git-blame. I think you internally believe this to a certain extent as well, which is why you consistently cite ChatGPT whenever you use it (despite supposedly believing it's no more worthy of credit than a font or a spellchecker).
Second, I've said this several times at this point but I don't have a problem with a researcher using AI, and I don't think they should be "handicapped" for using it. The criteria a scientific journal uses to evaluate a paper need not be the same one that a layman uses to evaluate it. I'm fine with scientific journals accepting papers from authors who use AI if it advances the state of science, and I'm ok with them not even being aware of the tools used to produce a given paper to avoid bias during the review process, but if I read a truly well written paper as a layman I should be able to congratulate the person who wrote the prose after it's published, and that requires being able to establish provenance. Do you really have a problem with someone saying "I wrote this paper, not ChatGPT"? Does that really offend you, or somehow hinder science?
> Your mind perhaps rejects the idea that why you think I am playing a language game using a metaphor when I am not using a metaphor. For you there is nothing uncomfortable accepting that living things are machines? I think you have not thought about the consequences of accepting this view: https://chat.openai.com/share/2fdd8863-6a3b-4a09-9d9c-b2f196...
I never said you're using a metaphor. I've already clarified that when I use the word machine I'm using it in the commonly accepted colloquial sense of the term (i.e. non-biological machine). Rather than discuss the actual issue at hand (what happens to humans in a world where machines render us obsolete), you would instead prefer to broaden the term machine itself so that it encompasses literally anything. If you want to truly be this pedantic anything can be considered a machine. Earth is just the machine that transports us around the sun every 365 days. The Moon is the machine that regulates our tides. The universe itself could be conceptualized as the machine made up of galaxy clusters. Do you see how unproductive it is to play games with semantics? For someone who supposedly doesn't care about communication, and is purely interested in science, you spend a lot of time quibbling over semantics.
> Our lives will likely change much like our lives of wandering around from place to place hunting gathering changed and then lives of subsistence farming changed, etc. If you accept the view that living things are machines (not as a metaphor but in reality) the concept of computational universality may give you a clue about our likely future. GPT4 explains: https://chat.openai.com/share/4dd5e6b3-6136-408a-a05a-10261d...
All of your GPT links lead to 404 errors for me, so there's not much for me to respond to here. This is the problem with outsourcing your thinking to OpenAI. Your external "brain" is owned and operated by a private company, and can be thwarted by HTTP errors.
> Assuming it is possible: How do you feel about having observations of what you say and write and do being used to create a model that can predict these things about you with such accuracy that those that know you can not tell the difference? Will that model in a “computational sense” not be you? Say, if given a matching body, that model can replace you such that those that know you cannot tell the difference. Will that also not be you? If you accept the reality that you are a machine, why will that not also be you? Because you are the atoms that make you you?
There's a metaphysical argument to be made here about whether or not destroying your physical body and recreating yourself as a file on a computer constitutes death, but I have more pressing practical problems.
I'll preface this by saying that I normally don't categorically dismiss opinions, but I truly believe that people who advocate in favor of mind uploading haven't fully thought through the implications. Imagine a world in which a data breach doesn't just mean your personal information is being leaked, but your literal brain is being leaked to the public. Computers could speed up or slow down your subjective perception of time, so a hacker could duplicate your brain, and subject your digital clone to an eternity of existence inside a digital nightmare. In a mind uploading scenario, billions of people could be erased by simply reformatting a hard drive. The delete key becomes more troublesome than an ICBM in this situation. We've literally fought wars over whether or not humans could own other humans, and you want to compress people down to files that could be stored in a USB thumb drive in someone's pocket? Like I said, I really don't think people who are in favor of this have thought through it.
> I think there is an art to communicating complex ideas clearly, and some people value that ability,
Yes. There is also an art to fine penmanship and the ability to craft a professional manuscript by hand with just ink. This used to be valued now we use mechanical fonts and layout engines etc. Times change, technology changes.
The utility / novelty of the ideas vs how they are conveyed: which of those matters for a science publication? If both why not also got back to handwriting and ban the use of fonts / layout engines to increase the challenge required to create good manuscripts? Is science a costly signalling status contest, a process that should be as exclusive as possible? Is science a search for truth, a process that should broadly accessible even to those with poor language skills?
Absolutely, and I personally don't design my own fonts. But if I were a professional typographer like Matthew Butterick[1] for example, and someone accused me of using Times New Roman when I actually created my own fonts from scratch, I'd probably be a little perturbed.
> Is science a search for truth, a process that should broadly accessible even to those with poor language skills?
It's absolutely a search for truth, and like I said elsewhere, I'm not opposed to someone using a large language model to help them document their findings. You can continue using ChatGPT to help you write [2]. I have no issue with that. You're arguing against a straw man.
I'm saying if someone had poor English language skills for example, and they spent years acquiring English language proficiency, they'd probably be annoyed if you discredited their years of mastering the language by simply saying "you're not fluent in English, you just used ChatGPT to write everything". That doesn't mean someone else who doesn't have the time or resources to master English can't use ChatGPT as a writing aid. Those two things aren't mutually exclusive.
[1] https://practicaltypography.com/about-matthew-butterick.html
A closer analogy with AI for science writing would be to accuse you of getting help from a computer to create the new font and not doing it by hand. When you create fonts goal is the font. When you publish research the writeup is important but goal is useful and new results. Because accuracy and clarity of the writeup matters whatever serves that best should be used.
> You can continue using ChatGPT to help you write [2]. > https://news.ycombinator.com/item?id=38637667
The example you quoted states the output comes from ChatGPT. It was used to explain some concepts and it did a far better job than I could do for the time it took.
> I'm saying if someone had poor English language skills for example, and they spent years acquiring English language proficiency, they'd probably be annoyed if you discredited their years of mastering the language by simply saying "you're not fluent in English, you just used ChatGPT to write everything".
Will the situation be the same when AI improves so much that it helps writing much like mechanical fonts regularly help our writing today? Say I took years to develop penmanship skills such that my hand writing looks like mechanical fonts. Should I be annoyed if you say “Your hand writing looks like a mechanical font!”? When it comes to science writing should pride be placed in penmanship skills? Should pride be in English language skills? What matters for science writing?
Today AI writing has a mixed reputation. Sometimes the output is fantastic. Sometimes output is fantastic nonsense. To tell someone their manuscript looks like ChatGPT wrote it is likely meant as an insult.
In the future do you not expect AIs will be used to judge writing (like Latex style sheets guide journal submissions today)? Will writing not be rejected unless it meets high standards for clarity etc? Perhaps the best way to meet those standards will be to get an AI to help you? How do you suppose the future of science writing will go?
Ok fine, we'll go with your formulation of the analogy, because it doesn't affect my overarching point at all. If I spent hours tweaking my font manually with the pen tool in a vector graphics software suite, and someone accused me of using ChatGPT to simply spit out an SVG file, I'd be annoyed.
> When you publish research the writeup is important [...] Because accuracy and clarity of the writeup matters
Yes, exactly. If it were just a matter of uploading data, then scientists would be publishing their results strictly as csv files and excel spreadsheets. You're admitting that the writeup itself is important. Whatever small iota of importance you assign to the writeup, why can't a human being take credit for that achievement (regardless of how marginal you think it is)? Do you want to argue about the relative percentage of importance we assign to the writing vs the data? If I gave you the percentage you're looking for would you be content? I feel like we have nothing substantive to debate here.
> The example you quoted states the output comes from ChatGPT. It was used to explain some concepts and it did a far better job than I could do for the time it took.
Out of curiosity, why did you attribute the work to ChatGPT at all? I was under the impression that you don't value the craft of writing, and don't care about who receives the credit for it. So why admit to using AI at all? Isn't this the same as using the HTML input element's spellcheck feature, or using someone else's font according to your philosophy?
> Say I took years to develop penmanship skills such that my hand writing looks like mechanical fonts. Should I be annoyed if you say “Your hand writing looks like a mechanical font!”?
If you're a professional calligrapher and someone says you didn't produce your letterforms by hand, but instead used a robot that mechanically held a pen to produce the output, then yes you would have a right to be annoyed.
> In the future do you not expect AIs will be used to judge writing (like Latex style sheets guide journal submissions today)? Will writing not be rejected unless it meets high standards for clarity etc? Perhaps the best way to meet those standards will be to get an AI to help you?
That's not the future, that's the present. Half of job applications these days have a disclaimer stating that AI will be used to analyze the application, and writing is already (and has always) been rejected if it's not intelligible. The best way to meet those standards for you might involve using an AI to help you write...and that's fine. I'll repeat this again — I don't have a problem with people using AI to assist in their writing. The only thing I'm lamenting is the fact that it's becoming functionally impossible to take credit for your writing without someone assigning all of the credit to some tech company's AI product. I think you have this preconceived notion that there's something we have to disagree about, and you're desperately searching for the point of contention. I'm not really sure what your goal in this discussion is. Are you really trying to convince me that humans can't take credit for their writing? This is such a bizarre "debate".
My goal in this discussion is to make the case that AI assistance for science research writing is much like the use of mechanical fonts and layout engines for that purpose.
> Are you really trying to convince me that humans can't take credit for their writing?
A scientist was accused of using ChatGPT to help write up their research and wrote an article about it. In my mind the novelty and usefulness of the ideas matters when you publish research and the use of AI assistance to help you write is useful much like mechanical fonts and mechanical computation and layouts are useful.
You agree with this view yet you think there is inherent value in writing without AI assistance whereas I feel our energy is better spent on the ideas being communicated. My time / energy is finite and when I allocate it to X I can not allocate it to Y.
We used to take pride in the letter shapes our hands create and we still teach children hand writing however in practice handwriting no longer matters. We use mechanical fonts and machines for writing. The legibility of communication for all has greatly improved and our time is best spent on aspects that do add value.
> This is such a bizarre "debate".
We are sharing our views and explaining them. I do not see a debate.
AI chatbots are tools, fonts are tools, and layout engines are tools. Tools assist people. If your goal was to define those terms, then consider your goal achieved. This is why I sometimes despise philosophical discussions because they often accomplish very little besides defining terms.
> We used to take pride in the letter shapes our hands create and we still teach children hand writing however in practice handwriting no longer matters.
What I'm more interested in is to what extent different tools assist people, and how can we assign credit for work in a world where technology can mimic human work. For example, when I wrote this comment entirely on my own, but used someone else's font and HN's CSS to layout my response, I did a moderate amount of work and deserve a moderate amount of credit.
If I had instead simply copy and pasted your comment into ChatGPT and said "ChatGPT please write a response to this comment", then I would have done less work and deserved less credit. I'm not really sure what substantive discussion can be derived from such rudimentary assertions. It feels like we're discussing how blue the sky is.
> You agree with this view
I never said I agreed with this view.
> yet you think there is inherent value in writing without AI assistance whereas I feel our energy is better spent on the ideas being communicated. My time / energy is finite and when I allocate it to X I can not allocate it to Y.
I think if you're not able to communicate at the level you would like to, then it makes sense to outsource the work to a machine. Similarly to how if you had trouble seeing you might outsource the work to a screen reader, but if you don't have trouble seeing you might find more value in using your eyes to visually view a computer's display.
I personally find that introducing an intermediary (an AI chatbot) between myself and the person I'm communicating with reduces the accuracy of my communication, and I'd rather just speak to my audience directly. That may not be the case for you and that's fine.
> our time is best spent on aspects that do add value.
I think human writing can add value, and perhaps we'll just have to agree to disagree. I frequently find quotes that perfectly encapsulate an idea in a way that I find worthy of credit, and when the author of that quote is a human, I think it's worthwhile to award that person the credit for it. I found a great quote on HN today ironically by the CEO of OpenAI: "But I always want it to be a project that, if successful, will make the rest of my career look like a footnote". Everyone in the comments quickly attributed the quote to Sam Altman so that he could receive the credit for his writing [1]. Now of course in a scientific paper the importance is distributed between the communication and the data itself, so like I said before, are you simply looking for me to assign a weighting function to the writing vs the data? Are we really discussing a hypothetical importance weighting function?
Also, can I ask you a question (you ignored my last one about why you chose to credit ChatGPT for your writing by the way)? Are you writing these comments yourself or using a chatbot, and if you aren't using a chatbot why haven't you allocated your valuable time/energy elsewhere, since you apparently don't consider communicating by hand to be a valuable usage of time?
[1] https://news.ycombinator.com/item?id=39341637#:~:text=my%20f....
Some are paid to post to move public opinion, advertize, etc but your messages do not appear that way. Also you are using an 80 day old “throwitaway1123” account. Do you create accounts and then sell them? What sort of “credit” are you hoping for?
> I personally find that introducing an intermediary (an AI chatbot) between myself and the person I'm communicating with reduces the accuracy of my communication, and I'd rather just speak to my audience directly. That may not be the case for you and that's fine.
I also prefer to speak directly but that style can be misunderstood / come across rude so I use GPT4 to save time explaining my thoughts and also to critique my thoughts. This is out of respect to my audience so I do not waste their time with trivial thinking blunders that a machine can catch. I think this practice will be expected in the future much like spell and grammar check are expected today.
> Now of course in a scientific paper the importance is distributed between the communication and the data itself, so like I said before, are you simply looking for me to assign a weighting function to the writing vs the data?
What makes a science paper is the novelty and utility of the research it is about. To communicate that research is as important as the legibility of the fonts being used and now machines can help us with both.
> Are we really discussing a hypothetical importance weighting function?
We are discussing how to think about a researcher who was accused of using AI to help with her research writing. I think her work should be judged based on just the manuscript. You think her work should be judged not just on the manuscript but also whether machines helped her write it. In response to that I say: the labor theory of value is intuitive till you realize its consequences.
> Also, can I ask you a question (you ignored my last one about why you chose to credit ChatGPT for your writing by the way)?
If a passage is directly pasted from ChatGPT and is based on “its thinking” I want you to know it so you are not confused where it comes from since it may not be 100% my view so it gets treated like an information source.
If instead I ask chatGPT to improve my wording then it is like using spell check grammar check. Do you credit your mechanical spell check and grammar check for checking your writing? Don’t you assume these tools are being used and there is no “credit” involved? If the writing is important I may be annoyed if spell check and grammar check were not used.
> Are you writing these comments yourself or using a chatbot and if you aren't using a chatbot why haven't you allocated your valuable time/energy elsewhere, since you apparently don't consider communicating by hand to be a valuable usage of time.
The comments I shared are my thoughts. When GPT4 is used as a source I sanity check what it says and indicate it was used as a source. As in a science paper the thinking and the information shared is the valuable aspect. The superficial surface style of words and style of the passages is about as important as the style of the fonts. I am happy to let a machine handle it if it can. Ditto if language translation was required. We are not writing poetry. What matters is you understand me and I understand you.
To finish I will quote myself from a previous comment: “Our chat is like a dialogue I have with myself (or with chatGPT) when I debate views to make up my mind about them: A joint search for truth. When a smart person (as you obviously are) changes my views on an important topic I am obviously better for it. And even if you do not change my mind, the thinking exercise is valuable. Thank you!”
This feels like a bit of a non-sequitur to me because I don't see how it has any bearing on the issues we're discussing. But since we're on the topic, your account is 8 years old and you've got less than a quarter of my karma. Maybe lay off of the ChatGPT responses ;)
> What sort of “credit” are you hoping for?
Just because I believe authors have the right to insist on being credited doesn't mean I believe they are required to invoke that right immediately (or ever).
> I also prefer to speak directly but that style can be misunderstood / come across rude so I use GPT4 to save time explaining my thoughts and also to critique my thoughts. This is out of respect to my audience so I do not waste their time with trivial thinking blunders that a machine can catch.
I've found that ChatGPT is more likely to be the source of the thinking blunders, rather than the solution. The clearer your thinking is on its own, the less utility you'll get out of using ChatGPT as a crutch.
> You think her work should be judged not just on the manuscript but also whether machines helped her write it.
You're misrepresenting my argument. I think her work should be judged based on its quality, but if she chooses to claim the credit for her writing she should be entitled to do so. There's no way to argue against people being able to say "I wrote this, not ChatGPT", so you have to invent straw men to argue against instead. You're arguing with phantoms of your imagination.
> The superficial surface style of words and style of the passages is about as important as the style of the fonts. I am happy to let a machine handle it if it can. Ditto if language translation was required. We are not writing poetry. What matters is you understand me and I understand you.
I think there's a reason why we had to invent literal artificial intelligence in order to produce even a facsimile of human writing whereas spell check and digital fonts have existed for decades — it's because writing well is difficult! Writing isn't the only component that matters when communicating. Obviously the underlying ideas are important, but to dismiss it altogether is naive. The importance of writing exists on a spectrum. On one end of the spectrum is poetry, where the prose itself is more important than the ideas, and on the other end would be something like a csv file or spreadsheet where there's almost no writing at all. Scientific papers fall somewhere along that spectrum, and the only way to determine where a given paper falls would be to develop a weighting function, which is not something I'm interested in. If writing is supposedly so trivial that it's comparable to a font or a spell checker, why did it take decades of sustained work by an army of AI researchers to get to a point where even you still don't fully trust it to completely represent your ideas, and have to add disclaimers every time you use it?
I might end this conversation here because I've written so much that HN made me break some of my comments into separate replies, and I've got work to do. Thank you for this discussion!
This is just another example of the abysmal state of affairs that is modern peer-review.
> So much of science is built on trust and faith in the ethics and integrity of our colleagues.
This is where things went wrong. We need a culture of "show me," not "trust me," the core of critical rationalism: Establishing the convention that checking each other's work is the only way to advance our understanding of the world.
Figuring new things out is an error-prone process. Sometimes, these errors are not known to a researcher; sometimes, they are known but deemed non-critical; sometimes, a person has ulterior goals that would be endangered by acknowledging and correcting the error. I don't judge. We've all been there.
However, things have been swept under the rug for far too long. If large-scale attempts can only replicate 50% of the studies investigated, published results from psychology cannot and should not be trusted without further checks (https://journals.sagepub.com/doi/10.1177/2515245918810225). The problem may be less acute in other fields, but perhaps only due to a lack of scrutiny (https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...).
Absent an established process to verify a publication's central claims, substitute measures are used, such as "sounds like ChatGPT to me." This is an effect, not a cause, of a culture that values "trust me" over "show me."