Modern English usage does not pay much attention to the traditional ideas of being grammatically correct.

The trend today is to omit needless punctuation. In the case of "Hello, world!" vs. "Hello world!", there is no one who would be confused by either version. The comma does not change the meaning at all, so you can include it or omit it as you prefer.

But take a look at one example of modern usage. If your name is Mike (like mine), which would you be more likely to see at the beginning of an email:

Hi, Mike,

or:

Hi Mike,

Pretty sure it will be the latter. In fact I can't remember the last time I saw this salutation with a comma.

(Edit: after writing that, I noticed that my last sentence is an example of what I'm talking about. I think the "grammatically correct" version would be "In fact, I can't remember..." - but many writers today would leave out that unnecessary comma.)

Edit 2: Hello, downvoter! ;-)

Gosh, does anyone write like that any more? It seems so archaic.

See, the thing about the comma is that it isn't a question of being "grammatically correct" or not. The purpose of the comma in modern English is to show where you would pause when speaking.

Which way would you pronounce it? "Hello world" just like any other two words in a row? Or "Hello <pause> world"?

The way you pronounce it is what should inform you about whether you write the comma or not.

It's cute. "Hello" is just greeting the programmer, as if it were one person to another. "Hello, world" is like a new baby introducing itself to a planet. It's both tiny and momentous; the contrast is humorous. That feels friendly.

Especially viewed in its original context, introducing a new programming language (C) to a planet that wasn't quite as crowded with programming languages as it is today.

Since then it's just tradition. Feel free to leave it out. But it's iconic for a reason.

Traditions are traditions, and an industry as young as ours needs to hold on to the very few we have!

Moving goalposts around on the Hello World code golfing world has some 'consequences' - there's languages with built in hello world that try and get it printing with minimal code, and removing the comma, for example, would invalidate that hack.