Comment by defrost | Hacker News Reader

Comment by defrost | Hacker News Reader

Earlier in the press:

Geoscience AI in crisis? (17 June 2024) https://geoscientist.online/sections/viewpoint/geoscience-ai...

    Paul Cleverley raises concerns about big-data artificial intelligence projects in the geosciences

Geologists raise concerns over possible censorship and bias in Chinese chatbot (24 June 2024) https://www.theguardian.com/technology/article/2024/jun/24/g...

    Tests on Qwen, part of GeoGPT’s underlying AI, reveal geoscience-related questions can produce answers that appear to be influenced by narratives set by the Chinese Communist party.

    For example, when asked how many people have died in a mining operation in Ghana run by the Shaanxi Mining Company, Qwen says: “I’m unable to provide current or specific information about events, including mining accidents, as my knowledge is based on data up until 2021 and I don’t have real-time access to news updates.”

    The same question posed to ChatGPT, the chatbot developed by the US company OpenAI, produces the answer: “The Shaanxi Mining Company in Ghana has experienced multiple fatal incidents, resulting in a total of 61 deaths since 2013. This includes a significant explosion in January 2019 that alone claimed 16 lives. ”

Eight days ago on HN: https://news.ycombinator.com/item?id=40773876 (1 comment | 2 points)

Is it any surprise that models reflect their RLHF training, and thus cultural and governmental underpinnings...? Just look at the old Gemini cultural flubs for a western-centric example. It will be very difficult (impossible?) to remove underlying bias from an AI model, since you'd be hard pressed to get any two people "aligned", let alone entire communities, states, countries & the world.

It is not a surprise models reflect their source training data and RLHF. The surprise is the DDE project choosing to use a Chinese government censored model (based on Qwen) for a world-wide audience.

Are miner deaths relevant to a geology-focused tool? The example of its use was to explain the fossil record.

Not relevant to surface geochemistry results, no.

Not relevant to airbone geophysics results, no.

Not relevant to to drill exploration sampling results, no.

Not relevant to Technical | Economic Feasibility studies, no.

Not relevant to mining environmental impact studies, no; although deaths, injury and illness in surrounding populations is.

Relevant to ongoing minesite safety records and operations, yes.

It's a question of scope; a geoscience aware AI that can answer questions about resource availibity, resource extraction and associated risks, costs and impacts that includes the kind of information found in, say, https://www.spglobal.com/marketintelligence/en/campaigns/met... would be expected to aware of mining deaths.

Just as people want to be aware of nuclear risks for power and deaths in lithium processing from accidents and associated radioactive waste.

It's being touted as a Geoscience tool, not just a chatty geology database.

Social geoscience (impact of geology on society) is very much core to geoscience and geoethics. Ironically, UNESCO and IUGS authored the paper below calling for a change in how we perceive geoscience..that was cited by the DDE project.

https://www.escubed.org/journals/earth-science-systems-and-s...

It’s a bit unfair. I think a proper comparison would be whether the geoGPT gives answers for other companies/countries screw up. This makes it sound like a conspiracy when it could be just poor bot.

It's an odd example, to be sure, part of the reason I pulled that from the Guardian article.

It's really pitting two general knowledge AI DB's against each other on mining knowledge, it's unclear whether this is a smoking gun on censorship or incomplete data entry.

It does highlight a knowledge void though.

The Deep-time Digital Earth (DDE) GeoGPT uses Alibaba’s Qwen LLM. It is no conspiracy, the fact is that Chinese AI has to abide by Chinese Censorship Law’s.

It does give answers on other companies and countries.

Check out Leonard Lin’s excellent independent technical evaluation of Qwen’s Chinese government censorship.

https://huggingface.co/blog/leonardlin/chinese-llm-censorshi...

As Lin points out, Qwen is Chinese Government censored so why bother using it in a global context when there are equally as good models (if not better) that are not aligned - so won’t issue Chinese government propaganda or refuse to answer.

It seems odd that the International Union of Geological Sciences (IUGS) which is non-political and non-governmental, has allowed DDE to build such a system for the world’s community of geologists with such Chinese government bias and censorship. Maybe they did not realise what was being developed using their name.

Reply on news.ycombinator.com