Hacker News Reader

Comment by A_D_E_P_T | Hacker News Reader

Something you've gotta understand is that the majority of English-language scientific journal articles are written by authors who aren't native English speakers/writers.

> https://ncses.nsf.gov/pubs/nsb202333/publication-output-by-r...

In the past, just after they submit a poorly-written paper, the sleazeballs at Wiley/Elsevier/Springer would "encourage" said authors to employ their "Author Services" who would edit and rewrite their paper for them, and resubmit it for publication. This didn't come cheap.

Today, everybody just uses LLMs. LLMs are masterful translators of language and concepts. It's a win-win for everybody but Author Services.

Admittedly, word choice and sentence structure are much more limited. (This is not necessarily a bad thing. LLM-written scientific papers can be clearer, and are often more tidy, than human-written scientific papers.) I don't like seeing LLM-written text in journalism, literature, and other media, but I don't mind it so much in scientific literature -- if there's no fraud involved, that is.

View on HN

Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of choice didn't e.g. add in any nuance that isn't correct or intended.

I do think it's possible they do, I myself am a foreign speaker who would consider doing something like this, and I think I could notice if that happens. But then I think I'm also well beyond the level where this would be a necessity for me, rather than just a convenience and an additional polishing step.

Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.

I don't think modeling this as Author Services = good, LLM = bad (or vice versa) makes for a fruitful conversation, and that (modeling it as such) wasn't my intention either. I'd definitely expect Author Services to do a better job though, even just based on this much description.

> I'd definitely expect Author Services to do a better job though, even just based on this much description

Yeah, me too, but the claim was if "paper authors have the necessary language skills to ascertain" the correctness of the translation. Regardless of how good the LLM or Author Services are, the problem remains the same.

Right, that's true. Also scary in its own way. I'm not too big a fan of natural languages - something I voice quite frequently these days I notice :)

The main worry, in my opinion, is an issue of responsibility. If the author services makes a mistake, a person will be blamed for it. But when an LLM makes a mistake, people tend to shift the responsibility from the person that used the LLM to the LLM itself.

People don't simply want to shift the effort of a task away from themselves, they also want to shift away the responsibility for doing it wrong.

> If the author services makes a mistake, a person will be blamed for it

Wouldn't the paper authors be "blamed" (held responsible) for it, rather than the "Author Services"? Ultimately, they're responsible for the quality of their paper.

Just like I'm responsible for making sure my taxes are paid correctly, even if I use an account, tax lawyer and financial advisor for the whole shebang. If anything is incorrect, I'll be held responsible.

Yes, but this will only happen after a mistake occurs.

What I mean is that if it's your responsibility to do something right, it acts a deterrent that motivates you to make sure of it.

LLM's allow people to defer this responsibility to the LLM, so they may avoid making sure of things because they let the LLM take upon their responsibility.

If anything, wouldn't it be the opposite? When you offload something to a human, at least you can say "it was their fault!" and point to an actual name, or similar.

But you can't use a tool and then say "it was their fault!" and point to the screen, it's not another entity like the "Author Services" is.

It's the same thing, but normally there wouldn't be "a human" because this chain of delegations stopped on you. Now you can just open ChatGPT.

I believe both arguments assume honest contributors, which may not be completely true. I have no reference but vague recollection of doctored data, made up results (basically cheating) due to publication pressure or just dishonesty/lazyness plaguing the reliability of scientific publications. Some may even be happy of the twisted reality or halucinations producing novel sounding results advancing the scientific career similar to those occured in the past. Reproducibility crisis is their friend to get away with it for a long enough time to have the personal gain harvested.

The reason I "assume honest contributors" is because assuming anything else produces trivial conclusions. Obviously someone who just wants to pump out something, anything, won't care about the accuracy of delivery, nor will anyone be impacted by the lack of it (as the paper was rubbish from the get-go). I just don't find that whole can of worms relevant here.

If I really think about it, I guess I can see it being relevant like how typesetting software is, for making a paper feel more serious than it really is. Not really the angle I was going for though.

Absolutely. I think, even for non scientific journals the LLMs have a net good effect on expressiveness and clarity when instructed like that.

> LLMs are masterful translators of language and concepts

No, they are not. They will - speaking generally - try to produce what seems like a typical translation; which is often the same thing, but not always.

They're pretty good at translation, to be honest. Transformers was initially invented/discovered in order to build something better for doing translations with Google Translate, so it's hardly a surprise they are pretty good at it.

> It's a win-win for everybody but Author Services.

Having work that can be edited by a human who understands the concepts –or at least the words— shows a great depth of usefulness to the work itself.

Running your rough draft in one language through an LLM so it appears, at surface glance, to be similar to a publication that a team of humans had been involved in doesn't actually provide value to the author or the reader. Now the author has a paper they cannot say they wrote (or indeed, say who wrote it) and the reader cannot assume that what they read was intended to mean what it now means in English.

If anything, people who write papers in another language to be published in English should be leaning more than ever on human editors and quality assurance measures, not slop machines that hallucinate with good grammar.

In my (non-English-speaking) country, paying for an editing service before one submits a paper was very common. Now it's all AI edited.

That's not the interesting part though. The interesting part is that the biggest editing service provider here explicitly states it's okay for the clients to send the papers through AI first then buy their service for 'additional human editing.' They even offer cheaper price for papers that are already edited by AI.

The irony is not lost on me.

I think you overestimate the extent of Author Services' involvement. It's near nil; there's no back-and-forth; you just send them your article and they send back a version with fewer grammatical/textual errors and correct formatting for the journal. It's a lot like an LLM, just slower and considerably more expensive.

Also, you severely underestimate the ability of the authors to edit and verify the correctness of their own work. They wrote it in the first place, after all. They know, better than anybody else, what's important to convey.

What's more, this editing step happens before peer review, so if the paper is junk or has glaring errors it'll probably be turned down by most respectable publications. Even the marginal publications won't publish stuff that's really bad.

You're pointing out issues with one departmental service provided by one publishing company. I'm pointing out issues with using any LLM.

If you don't like Author Services, use someone else. Involve a coauthor. This is not even a remotely hard problem to solve.

Yes, it is not hard at all to solve. Just use an LLM.

Don't LLM's have enough insight in at least the very concepts here? o3 is considered PhD level on this. Maybe not pure math solving since LLM's suck on that, but I wouldn't be quick to doubt the capacity of understanding the advanced concepts and typical wording. Also, this is hopefully not passing through LLM's completely unvetted, but as a crutch to get 80-90% there with minimal effort.

I find it quite insulting that you seem to think that non-native english speakers are incapable of reading the outputs of LLMs to asses if it still means what they intended to say.

Telling me you're insulted by my comment makes me question whether it is worth the time to reply. In the spirit of goodwill, let me provide some context that your emotional response might not have given you time to consider:

I work in a multilingual healthcare field and my output is often translated into different languages. Forms. Posters. Take-home advice following surgery. We provide all of this in every language where more than about 5% of customers speak that, so English, Vietnamese, Korean, Spanish, Tagalog and Mandarin.

In addition to English, I speak and read one of these other languages fluently and have since I was about 9 years old, but I don't live in the culture and don't understand the conveyed meaning of translated health-related phrases.

Do you think I use an LLM or an editor that does? No–because that would be silly and could convey information incorrectly to the audience who can only speak that language.

If you want to be quite insulted, turn on the news and get a realistic perspective on what is going on in the world. The people hurt by text going through LLMs is going to be those in extreme poverty and minorities subjected to machine generated translations without human review. You're fighting on a site where most of us would likely be on the same side of so many issues. Let's discuss and not make this facebook full of thoughtless responses.

Especially because it's so much easier to understand text than to produce it. I can read difficult authors in a foreign language and understand perfectly but there's no way I could write like them.

This just tells me you don't work with multiple languages very often.

I have spoken a second language fluently since about 9. I produce work that is translated into that language regularly... by a translator.

Being able to read words does not means I understand the meaning they convey to a person who only speaks that language. These are scientific papers we're talking about, conveyed meaning is valuable and completely lost when a non-native speaker publishes machine generated output that the writer could not have written themselves.

These are scientific papers we're talking about, typically written by non-native speakers to a primarily non-native audience. Scientific writing practices have evolved over a long time to convey meaning reliably between non-native speakers. You are supposed to write directly and avoid ambiguity. To rely on literal meaning and avoid idioms used by native speakers. To repeat yourself and summarize.

Based on what I've seen, LLMs can write scientific English just fine. Some struggle with the style, preferring big pretentious words and long vague sentences. Like in some caricature of academic writing. And sometimes they struggle with the nuances of the substance, but so do editors and translators who are not experts in the field.

Scientific communication is often difficult. Sometimes everyone uses slightly different terminology. Sometimes the necessary words just don't exist in any language. Sometimes a non-native speaker struggles with the language (but they are usually aware of it). And sometimes a native speaker fails to communicate (and keeps doing that for a while), because they are not used to international audiences. I don't know how much LLMs can help, but I don't see much harm in using them either.

This just reminds me to never assumes someone's reality. I speak more than two languages. And I disagree.

Papers are read by all type of people, I don't know why you assume scientific papers which are almost all written in English are read solely by native English speakers.

People have been doing science in broken English, French, German, Arabic, Latin and more for as long there has been science to be made.

Sounds like you would be a perfect person to help clear up misunderstandings in texts being translated if your skills are as keen as you describe.

You mention being reminded not to assume someone else's reality by our conversation--I would encourage you to also be reminded of the common fallacy where people wildly overestimate their own abilities, especially when it comes to claiming to speak/read/write multiple languages with knowledge akin to a native speaker.

It is unfortunately very common to mislead oneself about abilities when you haven't had to rely on that skillset in a real environment.

I would venture that you don't regularly work with multiple languages in your work outputs, or you would have likely received feedback by now that could help provide understanding about the nuances of language and communication.

If you disagree with the assertion that people generally have an easier time understanding language (correctly or not) than producing it, that's one thing and that's fine. But if you consider it a claim outright, and find it incorrect, then that's gonna need some beyond-anecdotal supporting evidence, or you should ask for some from the other side. Digging into each others' backgrounds is not this.

Keeping to anecdotals and opinions though, I only speak one foreign language sadly, that being English, but this effect is very familiar to me, and is also frequently demonstrated and echoed by my peers too. Even comes up with language loss, not just language learning. Goes hand-in-hand with reading, writing, listening, and speaking being very different areas of language ability too, the latter two being areas I'm personally weak in. That's already a disparity that a cursory read of your position says shouldn't exist (do correct me if I'm misinterpreting what your stance is though).

And all this is completely ignoring how even the native language output one produces can be just straight up wrong sometimes, and not faithful to intentions. I've been talking about it like there's a finish line too, but there really isn't. This is why things like mechanized math proofs are so useful. They are composed in formal languages rather than natural ones, enabling their evaluation to be automated (they are machine-checkable). No unintended semantics lost or added.

> If you disagree with the assertion that people generally have an easier time understanding language (correctly or not) than producing it, that's one thing and that's fine.

I disagree with the assertion that a person should rely on an LLM as part of their ability to publish in a language they don't understand well enough themselves to complete without involving a word machine.

> Digging into each others' backgrounds is not this.

I spoke from experience and it was then skewered by someone cosplaying the duolingo owl on the internet. You can take it up with them if you have an issue.

> And all this is completely ignoring how even the native language output one produces can be just straight up wrong sometimes, and not faithful to intentions.

How does the inability you point out of even a native speaker to clearly and effectively communicate sometimes not simply make it more obvious that a person less familiar with the language should involve a person who is?

> How does the inability you point out (...) not simply make it more obvious that a person less familiar with the language should involve a person who is?

I think that's a perfectly obvious point that the person you were replying to, you, and me, are all on board with and have been throughout. Inviting their or your attention to this was not the purpose of that sentence.

> I spoke from experience

Great.

> and it was then skewered by someone cosplaying the duolingo owl on the internet. You can take it up with them if you have an issue.

But my issue was/is with you. I wanted you to stop engaging in the use of combative and emotionally charged language. I understand that you feel justified in doing so, but nevertheless, I'm asking you to please stop. It dilutes your points, and makes it significantly harder to engage them. I further don't think you guys were disagreeing nearly hard enough to justify it, but that's really not my place to say in the end.

> I disagree with the assertion that a person should rely on an LLM as part of their ability to publish in a language they don't understand well enough themselves to complete without involving a word machine.

Thanks for clarifying - it genuinely looked like you were disagreeing with what I mentioned too.

> But my issue was/is with you. I wanted you to stop engaging in the use of combative and emotionally charged language.

You seem very intelligent, I truly believe your time and energy would be better spent doing literally anything else than providing me feedback on my commenting etiquette. Please, I implore you to do more with your time that will provide value! You genuinely seem smart.

(See how that felt? That's the effectiveness of telling someone on the internet you want them to behave differently. It's really pointless.)

> See how that felt? That's the effectiveness of telling someone on the internet you want them to behave differently. It's really pointless.

I mean, I think this was pretty alright? I appreciate the advice too, and even generally agree with it. This was just my extremely poor attempt at deescalation, because I thought it might work out nevertheless.

Why?

I don't have academic paper publishing peers with bad language skills, but I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers.

All otherwise perfectly smart capable people, they just happen to have this as a gap in their skillset. And no, they don't notice if transformative details get added in or left out.

> I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers.

Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved?

If it is the former, I would not want to work somewhere that people get into petty catfights over editing and have no abiity to write a sentence or understand linguistic cues. I don't remember working anywhere I would describe in the way you do in your second paragraph.

> And no, they don't notice if transformative details get added in or left out.

I guess I don't have to tell you not to select them as the people to review your work output?

Again, all the examples I'm reading make me think it would be beneficial for folks to include competent team members or external support for projects that will be published in a language you don't speak natively.

> Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved?

Can't tell you for sure (would require me to have comprehensive knowledge of the language skills around the company). I do know a few folks with proper language skills, but they're a rarity (and I treasure them greatly). Could definitely be just my neck of the woods in the company being like this.

> If it is the former, I would not want to work somewhere [like that where] (...)

Yeah, it's not great. The way I solved this was by simply not caring and just talking to them in proper English, hammering them until they provide me (and each other) with enough cross-verifiable information that thus definitely cannot be wrong (or will be wrong in a very defendable way), with an additional serving of double-triple checking everything. Some are annoyed by this, others appreciate it. Such is life I suppose.

> I guess I don't have to tell you not to select them as the people to review your work output?

I don't really have a choice. I think you might misunderstand what it is that I deliver though. I work with cloud technologies, so while I do sometimes deliver technical writing, most of my output is configuration changes and code. When I speak of language barrier issues, that's about chat, email, and ticket communications. I think that's plenty bad enough to have these kinds of troubles in though, especially when it's managers who are having difficulties.

> I don't really have a choice.

When does your employment contract end?

Not a fixed contract, so when either side terminates it (I understand the question was rhetorical). Where I live, opportunities are not so plentiful, though I am working on polishing up my CV to compensate. Benefits are decent though, can WFH all the time, so that's also a consideration. Most everywhere they're doing the silly hybrid presence thing now, which would suck (back and joint issues don't mesh too well with having to move around in the office and travel back and forth every (other) day) - maybe moreso than the linguistic landscape that at this point I'm fairly used to.

This hesitance to switch is definitely put to the test a lot these days though :)

Reply on news.ycombinator.com