The reason that "Hello, World" is a C thing is that C is the source of the tradition around that phrase. In Kernighan and Richie's "The C Programming Language" printing "Hello, World" is the very first exercise.
The first program to write is the same for all languages:
Print the words
hello, world
Suggesting that this was already an established tradition when the book was written in 1978