Those are good questions. I'll take a stab at an answer. It's long, I can't quite figure out how to succinctly sum it up. Sometimes writing a longish post helps organize my thoughts, maybe you will find it interesting...
I'm a music producer, of sorts, but I don't work in the music business per se, so my knowledge of actual costs is limited. I do occasionally work with people who are active in the industry, and so I try to at least keep up with what's going on.
(I usually work either for companies, or on low budget independent projects that don't make money, but are more products of love and passion:)
It's a little hard to get your mind around. The music industry proper is not that big, and what really goes on is not discussed in general forums very often. It's a bizarre little business world unto itself.
One thing to keep in mind is that the music market as an economic structure is an almost entirely "fat head, long tail." The vast amount of revenue is derived from the output of small number of artists (maybe 100 at any given time). The percentages are crazy. The percentage of recordings that even generate any significant revenue, never mind profit, is on the order of .001% (I'm guessing, but it's very low). There is definitely revenue that comes in from the long tail part of the market, but it's relatively small.
This "winner take all structure", which is inherent to a mass media form of entertainment, tends towards record labels being conservative about who they sign, and who they take a chance on promoting. If they do decide to seriously promote an artist, and support the whole business infrastructure needed to create a modern, mass media entertainment "product," they will bet big, and not hedge on production costs.
It's hard to understand in part because the way commercial music is actually produced has changed forms many times over the last 100 years, and the current model is relatively new, and not widely known by the public at all.
Currently the production model for commercial music is that a small team will be put together (or maintained) who work under a name "producer." (Dr. Luke being at the absolute top). These are they craftsmen/women who actually produce the music.
There will be writers, musicians, engineers, sub-producers, and associated creative and business types, who will all work together with the producer, artist, and label to create the actual song product you might hear on the radio.
There are often multiple team members working in parallel, in multiple studios, on the same project, with the best material generated being chosen for continued work. (Sort of the way a bigger technology company would have multiple teams on R&D).
Musical material from outside producers and writers is also be brought in under the production umbrella. For example, sometimes an artist and producer get some traction organically, like you describe. The producer can be brought on board a team to continue the work.
There are lots of independent entities who are producing musical "product" that are used as kind of a farm system for the major labels and producers. A lot of modern music is created from a "beat" which is really just the basic music of the song. There are actually various markets at work for "beats." that are always busy. There are a small number of independent "songwriters" who have a major contribution to the process.
Out of this process, which doesn't necessarily take a long time, weeks, maybe months, a marketable "record" (a single song) is hopefully produced.
I would estimate the costs of this production phase at $70-90 thousand.
From there, a label might be looking at roughly $1 million in marketing costs, videos, and the like. Radio promotion alone can cost hundreds of thousands (getting the song on the radio). Similar funds will be used to get the song visible wherever they can, Spotify, Youtube, anywhere funds can be used to increase exposure.
Even after all this, most songs don't catch on and make significant money. But the hope is that all of this promotion gets it heard by enough people that actually like the song enough to buy it, stream it, tell friends about it, play it at parties, companies license it for commercials, etc...
This is nuts, but it is what is happening on the commercial end of the music spectrum.
The irony of the internet is that it is trivially easy to place a song on the internet at no cost. Anyone in world has access to your music! This was simply impossible in the old days. Also, the costs of the gear needed to produce music have dropped dramatically.
It is possible to produce hit records mostly using a laptop. It however remains a labor and talent intensive process. It is pretty rare that I come across music created by relative unknowns that would be viable commercially.
Which is not to say there isn't some great music out there from independent artists. Spotify does have some aspects which give exposure to artist further out on the long tail. For example, it's recommendation system is amazing, and works differently than those typically found in aggregating services like iTunes, Pandora, or Amazon.
Those services tend to reinforce the "fat head" part of the market. Pandora, in my experience, will usually draw from the most well known tracks by a given artist to generate a playlist.
Spotify on the other hand, especially its "Discover Weekly" feature, presents to me 30 tracks every week that are incredibly close to my tastes, yet I have usually not heard the particular song. I usually have not even heard of the artist! I have found some amazing music like this. Currently, I'm on a big listening tangent of Swedish Prog, which is mind blowing. (If you like adventurous rock music check out Anekdoten:)
I've read that the algorithms they use to generate personal recommendations integrate your personal taste based, on tracks you play, and put in to playlists, with similar info from other users. This is information that is truly based on what actual other human beings like to listen to, and so far it seems relatively free of the influence of label marketing money (at least if one's taste lean away from popular music).
Anyhow, the costs of marketing a song as part of a for profit project are probably higher than ever, as the competition for attention in the digital world is fierce.
An irony, to me personally, is that I think the quality of music that is popular these days, in some subjective sense, is poor, despite more resources being put towards the production than ever. It all reflects a high degree of production values and craft however.
This is related to fact that the technological and business structures under which modern pop music have fundamentally changed how pop music sounds in ways that are shocking to me. (As someone who has been creating music in various capacities for over 30 years, yikes!).
Almost every single song in the top 100 charts is produced through a "cut and paste" style of production, in a computer, using "virtual" instruments. It has only loose relationship to how music has been produced historically, which is based on the muscular efforts of actual people, moving molecules, which generate sound.
Something akin to the change to CGI effects in movies.
The effect on the music I find to be disturbing, as it leaves even "classic" genres of modern music, like Rock, strangely static, and lacking in feel. An "uncanny valley" effect.
BTW, I make do record my own guitar based rock music, heard by virtually no one. If you want to take a listen, you'll find it here. I think it's pretty good:)
https://sunshinefortheblind.bandcamp.com/