Experience synesthesia: Google tool lets you 'hear' colors and shapes | Hacker News Reader

Experience synesthesia: Google tool lets you 'hear' colors and shapes

artsandculture.google.com

59 pointslazycrazyowl43 comments

As a musician and artist who experiences mild synesthesia, this is throwing me off. I don't associate the colors illustrated in the example with the sounds I'm hearing.

I don't have anything other than personal experience to back this up, but I would be surprised if it's not a completely subjective experience—one person's synesthesia probably doesn't match another person's, per se.

Just curious for any musician's out there, when you're playing in the key of D dorian, for example, what colors go through your mind? F major? E minor?

All these have specific colors that tend to dominate my imagination.

Exactly. Now, what would be really cool is if you could build a questionnaire, similar to a personality test, that would match you up with the right color/sound scheme. So at the end of the questionnaire, you can verify the results by listening to the Kandinsky piece and see if we got your associations right.

On your subject of an online questionnaire for synesthesia, I'm almost certain there was a post on HN related to this, but I don't find it in my bookmark library. But I think I found the same article with a Google search. https://journals.plos.org/plosone/article?id=10.1371/journal... is a paper describing an online survey of colors versus letters, the url for the survey is in the paper, is still registered, but doesn't seem to be responding to me at the moment. Over 6500 people had taken the survey at that time. A strange correlation appeared when they looked at the data. They accidentally tracked down that that a Fisher Price toy with colored letters matched the correlation. And people who took the survey but were not the age where they would have been exposed to this toy did not show the correlation. You can find lots of related posts with a Google search for Fisher Price synesthesia or perhaps other plausible key words. If you could use similar methods and look for correlations between notes and colors then there might be some toy that young children experienced for many many hours that might explain some of this.

>I don't have anything other than personal experience to back this up, but I would be surprised if it's not a completely subjective experience—one person's synesthesia probably doesn't match another person's, per se.

From what I recall from the literature (it was a while ago, so details elude me), that while there does seem to be some relatively common experiences (many people associating a particular number with a particular colour), there's definitely a lot that's very idiosyncratic. I recall some people speculating that some of these associations could be shared because of common books/teaching materials, that maybe many children grew up with the same books showing letters and numbers in various colours. But that's just speculation as far as I'm aware.

Agreed, violins are green not red (not a joke, I 'm synaesthetic)

For me they're green or purple

This is exactly what my wife said when she went to that site. Synethesia is personal in its details.

I've wondered if I have synesthesia. When I play music I don't literally "see" colours anywhere except in my mind's eye, but I definitely associate certain notes with certain colours. It's not something I think about actively or even notice unless I'm paying attention, but it's definitely there.

And it's only really when I'm playing music (I play guitar), not when I'm listening to it. It's less about keys than it is about specific notes. A is red, B is blue. C is more yellowy orange. E is... I'm not sure, just dark, maybe black? G is kinda dark blue, maybe turquoise.

This is the hardest I've ever thought about it and the first time I've ever tried to write it down, but those associations are definitely there and have been there for as long as I can remember, and I've been playing guitar for at least 15 years.

Do I have synesthesia?

> I would be surprised if it's not a completely subjective experience—one person's synesthesia probably doesn't match another person's, per se.

I was wondering this too. Since they used machine learning instead of medical data, it means that each synesthesia case may be unique enough to throw off a synesthesia-color/shape catalog.

Kandinsky might have ended painting abstract art because normal art was ruined by his synesthesia, just as his abstract art is incomprehensible to someone without his kind of synesthesia.

Question, is the association for you with a singular sound or with a series of sounds (as found in the simulator)?

Not to be a bit evasive, but...both?

Part of it is very abstract and arbitrarily associative. As well, it can certainly morph. I wouldn't say there's any specific rhyme or reason to it.

If you ever watched the movie Soul—that spoke to me pretty well, even though they didn't focus specifically on synesthesia, per se.

Key of D is typically black for me. For dorian F-natural is a soft shade of red. The B natural is purple.

F major is red. E-minor is a desert sand color.

You also have perfect pitch? Is this common for people with synesthesia?

I don't have perfect pitch so it's not that I hear a note and see the color it's more like when I think of a note, chord, or mode I see a color in my minds eye. When I'm playing I am aware at a high level the notes I'm playing and their colors.

It's totally over my head. The final screen needs a button for "don't get it" or "annoyed at the pretentiousness".

There was always a bit of cynicism and criticism on Hacker News, but lately it is at a completely different level. Almost every comment on every submission is negative, if not of the work itself then of some tenuous political implication, and usually not a very original or interesting one. Overall the attitude of valuing progress and working towards invention is completely gone, replaced with the dominant cynical, pessimistic and conservative tone of academia and legacy media.

Maybe people here are annoyed at google (and other big tech companies) for their recent (and not so recent) actions, and just don't care about stuff like this? Imagine if any other group of people, who continually screw other people over and have enough money to get away with it, came together and made something like this. Imagine they continued to not address any of the things people are upset at them for not addressing, and instead just said, "but look, we did an art."

I can only speak for myself, but I strongly believe I'm not alone in my refusal to look past google's bad actions because they made something neat. I'd much rather they start providing real customer support for their products and stop trying to ruin the open internet.

Also,

> conservative tone of academia and legacy media

Can you explain what you mean by this? I have heard people deny there is a liberal bias in these things, and I have heard people say that these things indeed have a liberal bias, but only because "reality tends to have a liberal bias". But I have never heard anyone actually claim it's biased the other way. I know it's off topic, but I am just so curious. It feels like that Slate Star Codex piece about people subjected to the same things coming to exact opposite conclusions, with each person being totally clueless about how the other person could disagree. (I cannot remember what he called it.)

Maybe not the best choice of word, but I think that people who call themselves "liberals" actually have very conservative goals, in the sense of being against change. They might come around to a type of 'yimby' when it comes to building apartments or whatever, but e.g. the opposition to meritocratic selection processes in guise of equality among identities is actually about preserving inherited wealth and social capital. This manifests in lots of areas as a default dislike of any major technological change, because that usually implies the risk of mobility between socioeconomic strata.

I agree. Curious, I started looking at the specific comments and commentator's post histories to see when they became active. As the tone changed, like minded people became comfortable and emboldened.

(By the way, your use of the word conservative, while correct, is likely to be misunderstood by some.)

The negativity might be warranted because we are actually running out of time with the climate emergency. It's a pessimism rooted in something very real and something very terrifying.

So sorry if I seem a bit negative. I'm going to work on my garden as soon as it warms and enjoy my family.

P.S.

It's not academia that has conservative views, but the workplaces most people here occupy as they crush any radical idea quickly and efficiently.

synaesthesia is such a cool thing to experience. I am blind and still synaesthetic. It all started out with me realizing as a child that every digit (I later found out it works for small numbers as well) has a dedicated colour. Helped me a lot learning to deal with numbers in my head. I (as most synaesthetics do) originally thought this is the way it works for everyone. Took a while until I figured out that you're being treated weirdly if you mention numbers and words having colours.

With music, I tend to get a carpet of all sorts of visual effects. I cant even begin to name specific colours or patterns. It is not very defined, as most of my visualisations are these days. Edges tend to blur from year to year once you finally lost your sight completely. But the myriad of colour and shapes is still there.

[1] https://en.wikipedia.org/wiki/Solomon_Shereshevsky

Quote:

"For example, when thinking about numbers he reported:

    Take the number 1. This is a proud, well-built man; 2 is a high-spirited woman; 3 a gloomy person; 6 a man with a swollen foot; 7 a man with a moustache; 8 a very stout woman—a sack within a sack. As for the number 87, what I see is a fat woman and a man twirling his moustache."

Can I say you are in good company?

While I don't experience synesthesia, colors and places resonate emotion, both good and bad, to the point where I don't go certain places. I cannot stand being in large conurbations or in heavy traffic. It's emotionally draining. Moreover, I cannot deal with hot, sunny days. I find them somewhat disturbing, and always get excited when there is overcast and/or rain for days on end. Trying to get my partner to see the benefits of the PNW over Texas is not working very well.

Editing to say that people who prefer overcast/rain are pluviophiles. Nothing better to me than a misty day. I find this delightful to relax or nap with:

https://www.youtube.com/watch?v=3gCiCxLkXOk

My best friend feels exactly as you do. While I agree on the urban aspect, nothing makes me feel better than living in a desert. Yes, it makes my breathing and allergies better, but the real reason is the sense of well-being I get in hot, dry, sparse environments that I get nowhere else. It really is sublime.

Interesting. We appreciate the same things, but in different environments. I desperately want to move to the forested mountains of the PNW for the lack of sun and heat, but my partner is opposed. Allergies are a thing for me, too, and Texas is horrible for me along with the 9 months of pregnant humidity (pun intended) that often reaches close to 100%.

When we honeymooned to Washington state a number of years ago, I felt like I had come home and it was my first time visiting the area. The forests and mountains are fantastic.

Same, I feel absolutely alive in a hot, dry desert environment. I wake up with no issues, I fall asleep immediately and feel my best creatively in a desert. I feel like I’m sleep walking in any place with a damp, cold environment.

I often "see" and get the feeling of motion from music, but it is way more abstract than this. Those videos showing a moving shape or character in time with a baseline, or percussion are familiar to me, in this sense, if that helps to understand what I am trying to convey.

Smells will also invoke shapes, and maybe odd colors when they are very strong.

I get the impression this work was done by someone who has not experienced synesthesia. Or perhaps a very different kind.

Smells and sounds triggered other impressions for me as a kid. For a long while, it grew more dormant. Muted.

An acid trip with musicians one weekend in my early 20's brought most of it back. And the experiences have endured since then.

Almost seems like my mind was congealing in some basic way, interrupted by the trip. Epic trip, BTW. Great people, trust and all the basics were there and experiencing music with those people under such an influence was amazing. I treasure that weekend. Have never seen reason to repeat it.

Down deep, I know that was one of those, "all things in moderation" tests. I plan on passing. And am glad I had the experience.

I have mild-strong synesthesia and it's fascinating to see how someone else sees sounds - and also how physically uncomfortable it made me.

Is this super laggy for anyone else in FireFox?

Is this really synesthesia or just association? Blue is water sounds, yellow is brass instrument, red is slow groove?

I have mild synesthesia for letters, numbers, shapes and scale degrees. At least in my case (and this is wildly personal and subjective) I'd agree that synesthesia basically is association. It's kindof hard to put into words, like for example

> Apple

I can see plain as day that the text above is black font on a light tan background. But the "A" is obviously yellow, each "p" is pink, the "l" is lightish grey, and the "e" is green. The word as a whole takes on the yellow from the A it begins with, but is tinted based on the other sounds it contains. I don't "see" these supplementary signals so much as strongly feel them, like it's a side effect of the language processing that's going on when I read.

I don't know the underlying mechanics that trigger this association, but it's quite consistent. I'm guessing that it's a side effect of how my memory of language is organized, some processing trick my mind is using to make the storage more efficient. I was surprised to learn later in life that this is unusual, like, "doesn't everyone see the association in their mind's eye like this?" Apparently not. :)

The mind is a fascinating thing.

> I don't know the underlying mechanics that trigger this association, but it's quite consistent. I'm guessing that it's a side effect of how my memory of language is organized, some processing trick my mind is using to make the storage more efficient.

Your description seems closer to what I read about synesthesia than the Google Art experiment. From my reading, it seems to be directly linked to the way the brain process stimulus rather than to memory. Brain scans show that for people experiencing synesthesia some stimulus activate parts of the brain unrelated to their processing. Some drugs (LSD notably) seem to be able to trigger this effect temporarily for people not usually experiencing it.

What you describe echoes with my experience.

I've had conversations with family members that confirmed I was somewhat unique in experiencing this way too. Most didn't relate, though one brother is waaaay more synesthetic than I am.

I think I got my synaesthesia on more from playing Rez (also inspired by Kandinsky) than from this.

Rez = ?

This is Rez: https://youtu.be/9_E5hCau8Hk

So: if you mention ‘machine gimmick’—errrr, ‘learning’—enough times then you get to reinterpret Kandinsky with an orchestra on Google's time.

So from now on I'm waiting for a new Therion: with machine learning, a band, an orchestra and two choirs. Let's put the advertising money to good use.

Transforming colors or shapes into sounds is trivial, and there are an infinite numbers of ways to do it (e.g. for each RGB value assign a tone, or set of tones, and bam! there ya go.) So it's not like that by itself this project very interesting. Or maybe I'm missing something?

The point is that some people experience that naturally, and those who don't are intrigued. This is a way of simulating that experience for them.

How? Anyone can program something where you point-click and the color you see becomes paired with some sound. How is this any more insightful that that?

Anyone can program it. Somebody did.

It's difference between writing a video game and playing it. People enjoy the experience differently from knowing that it could be written.

Just me or it doesn’t work in Firefox for iOS? I hear no sound at all.

Makes me miss flash.

Reply on news.ycombinator.com