I'm Swedish too, and grew up with the same understanding as the parent poster. Only in recent years have I understood that khaki can refer to both a military type green or sand like beige color.
I asked my wife, who knows nothing about this thread, how she would describe khaki.
"Oh, beige-green, sort of."
:-)
According to NE thesaurus:
"ljus, gul-brun-grå färg som påminner om (väg)damm el. sand: kakifärgad
HIST.: sedan 1904"
Swede here. I asked my wife too, her reply is that the color "khaki" is a shade of green - "khakigrönt". But the specific style of pants - "khakibyxor" - are generally beige.