473 karma · joined April 17, 2018
Skydio, SymForce, Riffusion
training frontier music generation models and building the most powerful and fun music creation product in the world
looking for a world-class infra person
{ hayk at riffusion dot com }
Riffusion is a 10 person team making a music platform where you can create any music you imagine. We are aiming for the greatest song crafting, editing, exploring, and listening experience ever made. We train frontier diffusion transformer models that make incredible full length songs, and have built a powerful product around it. I'm one of the cofounders.
We just launched into public beta -- check it out at https://www.riffusion.com/
We are looking for elite team members
* ML research scientists in large scale generative modeling
* AI infra engineers who have deployed and trained at scale
* Product builders who have worked on world class apps
Reach out at { hayk at riffusion dot com}
We're building the best music creation and listening experiences ever made, by making music interactive in new ways. We're a small team of world class engineers, researchers, and musicians working out of Hayes Valley. It's hard to believe we get paid for what we do.
Looking for:
* ML Research scientists and engineers who are deep in large-scale generative models and love music
* Devops expert deep on multi-node training and infiniband (will pay well for a short term contract)
Message me: { careers @ riffusion.com }
At the core, we’ve been training music foundation models that are high quality, diverse, controllable, and an order of magnitude faster than anything done before. This required a ton of deep investment across the whole stack from compact representations to architectures to deployment infra. The result is fast enough to enable totally new workflows for this stuff.
The primary input is to capture/upload a photo or video with optional personalization, then the app will generate original lyrics and music based on the context. The next time you see your dog or kid do something funny, you can turn that into a vibe in seconds. We’ve been having a blast creating music in moments that we never would have before.
The most remarkable part is the editing, as sampled in the web demo. In the edit view, you can go deep adding multiple prompts on a timeline and controlling their strengths and interactions. You can mix genres into something totally new or make transitions like going from an operatic aria to death metal. The near-real time editing flow is something that to me feels totally new, like trying GarageBand when it first came out.
There’s a ton more we’re excited to launch in the coming weeks as we work to create more musicians in the world. Excited to hear what you all create!
Looking for world class ML research engineers and scientists to join our core team building the future of music. If you love creativity, large scale diffusion models, fast infrastructure, and early startup energy, our team may be a fit. We're building something new to inspire hundreds of millions of people. Competitive comp, in-person culture.
Contact: hayk @ riffusion dot com
Skydio is the leading US drone company and the world leader in aerial autonomy. Our mission is to make the world more productive, creative, and safe with autonomous flight. Today tens of thousands of our robots are flown at scale to inspect critical infrastructure and provide life-saving intelligence, and we have one of the best robotics teams on the planet in support of these goals.
Our X10 drone features NVIDIA Orin and Qualcomm 865 processors with 85 TOPS of compute capability, along with onboard 360 trinocular camera coverage. We develop real-time 3D reconstruction and semantic AI so that our robot can understand the world around it and make intelligent decisions.
We’re looking for infrastructure engineers and a director for our autonomy team who love making code for flying robots and for making our engineering team more productive. Experience with Python and C++, performance optimization, systems infrastructure, first-principles thinking, 3D visualization, communication and logging. full-stack web apps, and build systems all welcome. Very strong software engineering and computer science fundamentals required.
I lead the autonomy team and you can reach me at { hayk at skydio dot com }.
Skydio is the leading US drone company and the world leader in aerial autonomy. Our mission is to make the world more productive, creative, and safe with autonomous flight. Today tens of thousands of our robots are flown at scale to inspect critical infrastructure and provide life-saving intelligence, and we have one of the best robotics teams on the planet in support of these goals.
Our X10 drone features NVIDIA Orin and Qualcomm 865 processors with 85 TOPS of compute capability, along with onboard 360 trinocular camera coverage. We develop real-time 3D reconstruction and semantic AI so that our robot can understand the world around it and make intelligent decisions.
We’re looking for infrastructure engineers for our autonomy team who love making code for flying robots and for making our engineering team more productive. Experience with Python and C++, performance optimization, systems infrastructure, first-principles thinking, 3D visualization, communication and logging. full-stack web apps, and build systems all welcome.
I lead the autonomy team and you can reach me at { hayk at skydio dot com }.
Skydio is the leading U.S. drone maker and world leader in autonomous flight. We create the world’s most intelligent flying machines to make the world more productive, creative, and safe with autonomous drones. Today our products are used for everything from capturing amazing video for consumers, to inspecting infrastructure, bridges, to enabling situational awareness for first responders in critical situations. We just raised a Series E at a $2.2B valuation.
We're looking for deep learning engineers to work on image-language understanding, open vocabulary detection, few-shot learning, and obstacle avoidance.
I lead the autonomy team and you can reach out directly to { hayk at skydio dot com }.
Read more at --> http://bit.ly/skydio-autonomy
We're now catching up and did a full rewrite to go from a hack to a quality software project. It also includes an extensible CLI tool and an interactive streamlit app for common tasks, MPS backend support, stereo spectrogram encoding, a test suite run by CI, and more. These tools provide a solid foundation to build upon.
One thing that's very important though is the language pre-training. The model is able to do some amazing stuff with terms that do not appear in our data set at all. It does this by associating with related words that do appear in the dataset.
https://colab.research.google.com/drive/1FhH3HlN8Ps_Pr9OR6Qc...
Meanwhile, please read our about page http://riffusion.com/about
It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself
This has been our hobby project for the past few months. Seeing the incredible results of stable diffusion, we were curious if we could fine tune the model to output spectrograms and then convert to audio clips. The answer to that was a resounding yes, and we became addicted to generating music from text prompts. There are existing works for generating audio or MIDI from text, but none as simple or general as fine tuning the image-based model. Taking it a step further, we made an interactive experience for generating looping audio from text prompts in real time. To do this we built a web app where you type in prompts like a jukebox, and audio clips are generated on the fly. To make the audio loop and transition smoothly, we implemented a pipeline that does img2img conditioning combined with latent space interpolation.
Skydio is the leading U.S. drone manufacturer and world leader in autonomous flight. We leverage breakthrough AI to create the world’s most intelligent flying machines to make the world more productive, creative, and safe with autonomous drones. Today our products are used for everything from capturing amazing video for consumers, to inspecting infrastructure, bridges, to enabling situational awareness for first responders in critical situations. We’ve raised over $340M and are in a strong cash position to achieve our vision.
We are hiring across multiple teams here at Skydio.
Autonomy- If you’re interested in being a core member of a world-class engineering and research team that is defining the future of a major emerging industry, dive in → http://autonomy.skyd.io/
Software Engineering- Our Software team is working on our cloud platform that allows our customers to manage their fleet of drones from anywhere as well as schedule fully autonomous missions. https://bit.ly/skydio-software-eng
Embedded Software - Our Embedded Software team acts as the brain stem of our autonomous drones, bridging the gap between the algorithms and the sensors/mechanical hardware. http://embedded.skyd.io/
Wireless - Our Wireless team builds a robust wireless network solution, extending the range of our drones and providing high-speed and secured communication.
Open roles: https://skydio.com/careers
I lead the autonomy team and you can reach out directly to { hayk at skydio dot com }.
Skydio is the leading U.S. drone manufacturer and world leader in autonomous flight. We leverage breakthrough AI to create the world’s most intelligent flying machines to make the world more productive, creative, and safe with autonomous drones. Today our products are used for everything from capturing amazing video for consumers, to inspecting infrastructure, bridges, to enabling situational awareness for first responders in critical situations. We’ve raised over $340M and are in a strong cash position to achieve our vision.
If you’re interested in being a core member of a world-class engineering and research team that is defining the future of a major emerging industry, dive in → http://bit.ly/skydio-autonomy
We’re looking for strong software engineers to join our autonomy infrastructure team. Please contact us if you have experience and interest in some combination of internal tool infrastructure, devops, data analytics, low-level performance optimization, and 3D visualization.
Open roles: https://skydio.com/careers
I lead the autonomy team and you can reach out directly to { hayk at skydio dot com }.
Skydio is the leading U.S. drone manufacturer and world leader in autonomous flight. We leverage breakthrough AI to create the world’s most intelligent flying machines to make the world more productive, creative, and safe with autonomous drones. Today our products are used for everything from capturing amazing video for consumers, to inspecting infrastructure, bridges, to enabling situational awareness for first responders in critical situations. We’ve raised over $340M and are in a strong cash position to achieve our vision.
We’re looking for strong software engineers to join our autonomy infrastructure team. Please contact us if you have experience and interest in some combination of full stack, internal tool infrastructure, devops, data analytics, low-level performance optimization, and 3D visualization.
If you’re interested in being a core member of a world-class engineering and research team that is defining the future of a major emerging industry, dive in → http://bit.ly/skydio-autonomy
I lead the autonomy team and you can reach out directly to { hayk at skydio dot com }.
The main comparison is that in SymForce one writes symbolic expressions in Python, and generates runtime code. In GTSAM and Ceres you write C++ code. This is really nice for exploration and code maintainability. SymForce also has a powerful system of automatically competing tangent space Jacobians that is more elegant and powerful than what the alternatives offer. Finally, SymForce has the benefit of a fresh implementation using the other libraries as a reference, and I think we were able to get better performance by greatly improving memory access patterns and designing functional APIs with solid abstractions (in the C++ optimization library). You can see this with the benchmarks described in detail in the paper.