MusicLM: Generating Music from Text
google-research.github.io
google-research.github.io
It is going to be interesting to see artists revert to being a no-revenue passion again instead of being well compensated. That “well compensated” did not last long for them and I feel for the struggle. At the same time, a part of me feels art has no place being motivated by money anyway. Perhaps this change will restore the balance. Artists will need to get real jobs again like the rest of us and fund their art as a side project.
I hope an open model for music generation will be soon to follow, it will really be a big deal.
Art should be a communication way, a hobby... All the people should have time to do art, do love or whatever they want... Also, all the people, who are able, should do productivity work, to maintain the physical bodies.
Huh? Music already is a near-zero revenue passion for the vast majority of artists, has been since mainstream access to recorded music if not even earlier, and the well compensated minority's superstar status, royalties for hit songs and sellout tours aren't really threatened by DIY tools that generate nice-sounding, genre-appropriate noise a bit more efficiently than a DAW and sample pack.
Right but those will become more and more available, dull, and not original (because feeding from the same « repository »).
So to differentiate and attract/inform/delight/surprise/tell better, they might still have to go to specialized people. Or find a way for those models to, like, hallucinate in different ways.
Virtually anyone today can make their own jewels or even setup a shop on Etsy. At low cost, low price.
Still, there’s a huge market for high quality and original/custom-made jewels. Especially when you want something with a story within it. And kind of a human relationship to the provider too.
The teacher in me is also optimistic. May be all this AI generated will be great for teaching and democratizing art for students who cannot afford to learn or have easy access to learn!
> Music generated on the fly for any given scenario, that is unique every time.
> Generating custom royalty-free samples which can be used by actual artists as part of a greater work.
> A way to learn about musical styles, motifs and timbres by creating many custom examples
> A way to experiment by generated many hundreds of examples, clashing together many different types of sounds together to find inspiration
There is a big backlash against AI by current artists and while I understand their pain, I wish we were able to all just stand in awe at the possibilities we’ll soon be having.
Unlimited dynamic catered experiences, accessible by all regardless of income. I just cannot imagine standing in the way of that future. I can’t imagine standing in the way of raw unbridled creation in the hands of everyone. It is just incredibly beautiful.
I seems to be missed on your comment, the AI will take on the real job first. The real job you were planning to reserve as survival means for the artist... are we not all now, creatives out of a job and without a backup plan?
As someone who has experienced both the career paths of a full-time musical artist and technical employee in the tech ecosystem, I find this comment saddening, because it revolves around a highly prevalent way of thinking in our society that needs to die.
Why shouldn't artists have their own career? Why wouldn't you consider this a "real job"?
My experience as an individual artist is that it feels very much like being the founder of a company. You’re dealing with technical, administrative, logistical, marketing issues every day, and you’re required to wear many hats until you generate enough revenue to use external actors/hire a few people.
With all due respect: best-case scenario, these lines might be fuelled by a form of untold jealousy; at worst, they reveal how deeply pessimistic and capitalist our worldview has become. Because such situations have long been observable in modern society, we listeners confuse this longitudinal data with empirical truth, when in reality it’s very much preserved by corporate interests and power dynamics within the music industry.
Artists have long been the last agent of the music business to be paid, and usually get the smallest slice of the cake. This is mostly preserved by a handful of actors who benefit most from this situation. Enter: 1. Labels (including, but not limited to, majors) exploiting the complexity of music rights (master, publishing royalties, ancillary rights, etc.) and imposing harsh contractual conditions on artists. 2. Digital streaming platforms (DSPs) built on vastly unprofitable, VC-money-ridden business models and making up for it by squeezing every possible bit of revenue they can.
The ever-persistent myth of the romantic, tortured artist who needs to live on skid row and experience dire living conditions to create the purest piece of art, devoid of any mercantilism is another problematic piece of the puzzle.
Empirically, extreme precariousness throughout the art world leads to mental health issues I’ve experienced first-hand and observed with many colleagues. When you don’t have the physical or mental resources to produce art, because you’re living from hand to mouth and focused on making a buck on the side, it directly and negatively impacts your practice. A steady, albeit small, income goes a long way.
But because gig-economy mechanisms and atomization of the artistic practice vastly profit the aforementioned actors (you could also throw in social networks, with their black-box, engagement-driven algorithmic updates), it stays the same.
Bottom line, it doesn’t have to be this way. That the ad-tech bubble is about to burst, or that some startup relied way too much on free VC money to prolong its ill-thought-out business model should not have such a drastic impact on the vast majority of artists. And more importantly, citizens like you and I should have no part in perpetuating these toxic mentalities.
Van Gogh was commercially unsuccessful during his lifetime and suffered from severe depression when he cut his left ear off; and here we are arguing this social model is the right one.
My 2 cents.
It is fundamentally another piece in the collapse of materialism and capital. Soon, all work will be machine-renderable, and what then? We will all be out of “jobs” and will need to rediscover what it means to be humans. A pursuit of experience, meditation, exploration, and significantly less tedious creation — it will be a radically different experience, and far more fundamental to our true nature, than what we experience now.
Me personally? I love to paint, write, and make video games. I love nature and would rather be hiking in remote terrain than pretty much anything else. But there is a lot of work to be done to make what I described in the previous paragraph a reality, and I have skills that can contribute to it, so I do that instead of what personally want. None of us are entitled to simply do what we want while most of us suffer and struggle. Some artists are skilled and lucky enough where they can make money. That bar is going to go a lot higher now with AI.
Artists should be compensated, but for whatever reason, fate would have it that work of the soul might involve acceptance of poverty. It isn’t about capitalism, it is about purity. I think we need universal basic income so that people can pursue their passions without concern of the finances. I also think that passionate people don’t care much about riches; and if they do, that occupies space in a mind that could otherwise be used for their art, and so someone else who properly uses that space will be more capable than them. This is why parasites can take so much from the top artists; they just don’t have the capacity left for that kind of bullshit.
The term “get a real job” might be harsh, but it simply means “get a job capital is interested in paying for”. This criteria constantly changes and AI is pushing it radically. Most people trudge through the shit show of that reality, just because someone calls themself an artist doesn’t make them exempt. I don’t agree with this reality so I would like to see it changed, but we cannot be offended by speaking about the way something is. This is the way it is. If we don’t like it, let’s change it.
You said you were surprised that the creative jobs are being automated first. Are you expecting the "real jobs" to also be automated, then?
If aliens came here and someone told them this was our music, they'd be justified in destroying the planet to make way for an interstellar bypass.
>instead of being well compensated. Hello? What % of pro artists are "well compensated"?
Later we had mass commercialization which "democratized" art but artists got squeezed by the intermediaries / distributors that enabled the mass market. As digital mass reproduction became a thing, this ushered zero marginal cost dynamics which is essentially still an unresolved stalemate.
The "AI" generation of all sorts of combinatoric works without attribution or payment comes to compound the challenges. You now have arbitrary amounts of derivative art that can be reproduced at zero cost.
Never a dull moment in homo sapiens land.
The whole point is that you won't be.
Like a lot of demand for small-time live music died with the record player.
It takes non-trivial investments of time to develop artistic capability, often with the aid of tuition or materials/equipment which also cost money. This is common knowledge and I frankly don't understand the point of ignoring it.
There's debate here, of course, about the "death of the author" [0] and whether work should exist independent of its author's intentions and intentionality - but perhaps our social understanding of that question will evolve, at least for works created after the AI explosion.
As a crude example that buoys me somewhat, there are numerous virtual youtubers who one might discount as avatars of a corporate music product - but their fans love them not only for their musical works in isolation, but for the personality and personal journey that the character (and, by extension, their voice actor, as the lines fade quite quickly) streams to the world, baring a pseudonymized version of their lives for all to see. There's real human intent and struggle and perseverance there, and it's appreciated, and that's really cool. Even if the human is hidden, the humanity shines through.
Even with the knowledge that something like this is inevitabely coming, it's pretty crazy to see the examples. The fact it can create fitting music from painting descriptions (including ones such as or Guernica) is unexpected. And it seems to generate almost naturalistic vocals with reasonable-sounding lyrics, with no embedded structure for lyrics or anything.
The paper itself is pretty cool. While GPT-3 was just "GPT-2, but bigger", this seems to have required some clever combination of models. Wonder if similar ideas will be needed for longer-form text or video generation.
Perhaps both ycombinator threads on MusicLM can be merged together.
If not, I will just post my comments here. I've always seen music as the "final frontier". With images and text you can afford tiny errors and noise that isn't that noticeable by the human eye, but with music everything has to be impeccable and if a note is slightly off, you can instantly tell. Machine learning models that will be able to create outstanding music, imo, will mark the end-game of creative AI endeavors.
Anyone want to share their experiments with MusicLM, feel free to join community-fan discord and subreddit:
BTW, I really hope this gets integrated in software like Ableton similarly how image generators get slowly integrated into Adobe Creative Suite.
These are pretty good, even the vocals are OK, although I wonder how much if parts of the prompt like "it may be used during a festival during two songs for a buildup" are actually adding much to the mix and the music's association with the painting descriptions is as loose as I'd expect.
Transformer models are quite a bit more computationally intensive than the LSTM this used, and GPT adds attention mechanisms; but the basic approach is loosely comparable and the LSTM model can be easily trained on a single machine.
I wonder why that never went anywhere.