UK government trial of M365 Copilot finds no clear productivity boost | Hacker News Reader

UK government trial of M365 Copilot finds no clear productivity boost

theregister.com

60 pointsdijksterhuis13 comments

I generally distrust AI results, have seen too many misleading results. So my daily productivity killer are increasing number of popups to use Copilot. 10 seconds of clicking it away and 3 minutes lack of focus for hating myself working in a company that uses Microsoft crap.

I have found that it is useful for specific tasks but those tasks aren’t necessarily time savers. For instance, asking it to rephrase a sentence doesn’t really save me time, but it does give me some alternative phrasing that I might not have thought of.

It is particularly good, as most AI is, at text summary. Whether that is summarizing what I’ve missed in a meeting, action items, or even finding a needle in a large haystack of a document.

On the other hand, I find it wholly lacking in the search/discovery of documents in SharePoint and surfacing important emails. There is no way for me to tell it that “such and such executive call” has no importance to me, or that “help please” is the most important email in my inbox.

"For instance, asking it to rephrase a sentence doesn’t really save me time, but it does give me some alternative phrasing that I might not have thought of."

But the real question is - does it matter? What benefit did you get from using the non-alternate phrasing vs the alternate phrasing?

It's not the same trade off as performing computation via a calculator to do more high value work vs doing it by hand without it.

We also trialled M365 Copilot. It’s absolutely useless.

Governments struggle to be productive on the best of days, I wouldn't trust Governments to have trained staff on how to be more productive with it in the first place.

I would normally agree. But as I’ve said in another comment - we trialled 365 Copilot. The only thing that was useful about it was meeting summaries in teams. But there are many alternatives that do a better job.

As something of an AI skeptic the thing that bothers me most is the wholesale gullibility and credulity of leaders. Incredible claims IMO require incredible levels of evidence, not just "tech bros with vested interests tell us this, it must be so!".

A level of general experimentation by companies and other organizations is clearly warranted, and it should be marked accordingly. This just isn't happening in a lot of places: "become 10x more productive or bye-bye" is just totally ludicrous stance to take.

So it's good to see some reports coming out where some attempt at actual measurement and assessment of efficacy has been made.

> As something of an AI skeptic the thing that bothers me most is the wholesale gullibility and credulity of leaders. Incredible claims IMO require incredible levels of evidence, not just "tech bros with vested interests tell us this, it must be so!".

If this would have been true, Microsoft would be an obscure SW vendor.

Dont worry, OpenAI's unconfirmed office suite will be much better.

https://www.computerworld.com/article/4021949/openai-goes-fo...

Nothing says that they are close to AGI and that ChatGPT5 was a raging success more than an office suite. They already have study mode, so definitely you know they are full of ideas and totally not stagnating and unprofitable with huge compute debt.

https://openai.com/index/chatgpt-study-mode/

On one hand I cant wait for this fad to end, on the other hand, I cant imagine how bad the next one will be. I almost miss my middle managers running around with stupid blockchain ideas.

> I almost miss my middle managers running around with stupid blockchain ideas. reply

I tend to agree with you, I miss the blockhain times.

LLMs allow the generation of symbols and characters (calling it "articles", "essays" or "source code" is an exaggeration) too quickly for us to edit and almost too quickly for us to have a second thought and assess what we are reading. When your eyes have seen so much slop in such quick succession, your brain starts normalizing it and you are more likely to stop noticing.

There goes your good taste, there goes your critical thinking.

And of course, you get into these discussions now: "I pasted this prompt in Claude and my feature was done in 5 min, why did it take you two days?". I shudder thinking what further enshittification the future holds.

"When your eyes have seen so much slop in such quick succession, your brain starts normalizing it and you are more likely to stop noticing.

There goes your good taste, there goes your critical thinking."

Yes exactly!

One thing I'm learning to appreciate is true and proper deep thinkers who take the time to break down a product/technology into its characteristics - to determine what the benefits and trade offs are and having the imagination of where they fit in the world - is extremely, extremely rare.

It takes a rare mix of visionary thought and discipline to do it well.

Reply on news.ycombinator.com