Rethinking Design Tools in the Age of Machine Learning
medium.com
medium.com
A company called The Grid collected about $5 million from users (and another sum in the same ballpark from VCs) with the promise of an AI that designs websites for you. Years later, they still have very little to show.
It seems like most of those tens of thousands of aspiring web creators who paid $96 to be "founding members" of this AI revolution have quietly licked their wounds and given up, because there doesn't even seem to be too much active discussion about the product anymore. (The Reddit forum is dead, with a few messages of the "Can we sue?" kind.)
If the end product is something that is basically a computer program, like a website, then I expect it to fail. My conjecture is that code and a text editor is already the optimal tool interface for creating computer programs. You can't create a superior tool interface, because code and a text editor is already optimal.
When it comes to creating art, there seems to be a great potential for better tool UIs. A guitar and hands is not the optimal way to create music. We don't seem to have found the optimal tool for creating music yet (if there is one).
My company is working on just that, creating the next generation of music instruments to allow anyone to create music.
Here is our first prototype where we've changed the UI of the guitar, but also left in traditional elements to allow beginners or experts to play music by switching from what we call "Magic Mode" to traditional guitar: https://www.youtube.com/watch?v=vkM4sr2socQ
1. It looks like there are limitless possibilities in that area of technology.
2. Magic Mode reminds me of Omnichord. It looked like with the Omnichord, you could do certain things more easily than with an analog instrument, but other things became harder or impossible. So it was a trade off.
3. In addition to allowing anyone to create music, it would be especially interesting if you could create new possibilities for expert players. For ex: some kind of augmented drum could allow you to create and play more intricate rhythms than would normally be possible, perhaps by having ML extrapolate and combine patterns in ways that are too complex to even write (I used to play drums). It seems like the ear can hear patterns that are beyond what can be played or even created by a person.
One of our investors is Matt Bellamy, the lead guitarist/vocalist of Muse. Generally when he wants to write a song, he goes for the piano to figure out the chord progression. It's much easier to compose on the piano because all the notes are in front of you in sequential order (black and white keys). After he figures out the chord progression, he'll switch to the guitar.
If he was to start composing on the guitar, there is a higher amount of cognitive load that is based on the physics of vibrating strings and his ability to physically shorten them by pressing your fingers on metal/nylon.
With our instrument he gets best of both worlds, a guitar interface, but every button is a chord in any key/scale. Basically, you now have a chord encyclopedia (like omnichord), but once he figures out the chord progression, he can switch to traditional mode and do the crazy stuff that he does all on one device.
At Huula, I'm a firm believer that ML can automate various parts of web designs. We just released a new experiment CSSToucan[1] to auto color texts on web pages with Recurrrent Neural Networks. It learns to color texts on web pages without a single line of color theories in the code. All learned from the data. Hope to see more and more ML powered design tools emerging!
In this case:
>> Like any other map, we can add textual labels — street signs if you will. This allows us to navigate the design space verbally with commands like: “take me to a maple leaf.” Once there, we could say something like: “take me a bit closer to an oak leaf.”
The change here would be subtle. Rather than a two-way conversation between tool and user, instead it's more like the user is telling a story and the tool is making suggestions along the way:
start:
"Create a leaf"
tool inserts a suggestion:
"Create a [maple,oak,cherry,...] leaf"
user updates the story with the clarification:
"Create a maple leaf"
tool suggests:
"Create a [red,green,...] maple leaf"
user:
"Create a green maple leaf"
user starts typing more:
"Create a green maple leaf on a"
tool inserts suggestions:
"Create a maple leaf on a [branch,wreath,...]"
Hope this is helpful.