rushingcreek

I'm not saying that the schema can't change from time to time, I'm saying that having any fixed schema at request time is not an ideal user experience because it may not be clear what is supported and what is not supported. From first principles, it's much better if the app schema can be created dynamically at request time so we can guarantee that we can fully serve the user's request exactly as they asked it.

Yes! We have a blog post here on how we designed these models and widgets: https://www.phind.com/blog/phind-2-model-creation.

Now that models have gotten much more capable, I'd suggest to give the executing model as much freedom with setting (and even determining) the schema as possible.

Yes, because it seems that Zillow exposes those specific filters as a part of the input schema. As long as it's a part of the schema, then ChatGPT can generate a useful input to the widget. But my point is that is very brittle.

Yep, this is a big problem as well. If the user doesn't know what features will or won't work, they lose confidence overall.

Thanks for the feedback. I think that our main differentiator going forward will be this generative UI on the fly for answering questions as opposed to search alone.

Totally agree that it's too slow with conventional approaches, which is why we're training custom models for this that we can run fast

I think this is very interesting, but it is reminiscent of what we built with Phind 2 where the answer could include dynamic, pre-built widgets.

The problem with this approach is precisely that these apps/widgets have hard-coded input and output schema. They can work quite well when the user asks something within the widget's capabilities, but the brittleness of this approach starts showing quickly in real-world use. What if you want to use more advanced filters with Zillow? Or perhaps cross-reference with StreetEasy? If those features aren't supported by the widget's hard-coded schema, you're out of luck as a user.

What I think it much more exciting is the ability to completely create generative UI answers on the fly. We'll have more to say on this soon from Phind (I'm the founder).

The most interesting thing by far is the ability to include video clips of people and products as a part of the prompt and then create a realistic video with that metadata. On the technical side, I'm guessing they've just trained the model to conditionally generate videos based on predetermined characters -- it's likely more of a data innovation than anything architectural. However, as a user, the feature is very cool and will likely make Sora 2 very useful commercially.

However, I still don't see how OpenAI beats Google in video generation. As this was likely a data innovation, Google can replicate and improve this with their ownership of YouTube. I'd be surprised if they didn't already have something like this internally.

Yes! There is a model called ACE++ from Alibaba that is specifically trained to replace masked areas with a reference image. We use it in https://phind.design. It does seem like a very esoteric and uncommon task though.

The native FP4 is one of the most interesting architectural aspects here IMO, as going below FP8 is known to come with accuracy tradeoffs. I'm curious how they navigated this and how the FP8 weights (if they exist) were to perform.

I spun up an H100 on Voltage Park to give it a try in an isolated environment. It's really, really good. The only area where it seems less strong than gpt-image-1 is in generating images of UI (e.g. make me a landing page for Product Hunt in the style of Studio Ghibli), but other than that, I am impressed.

Not sure why this isn’t a bigger deal —- it seems like this is the first open-source model to beat gpt-image-1 in all respects while also beating Flux Kontext in terms of editing ability. This seems huge.

Congrats on the launch! I'm very bullish on how powerful <10B-param models are becoming, so the on-device angle is cool (and great for your bottom line too, as it's cheaper for you to run).

Something that I think is interesting about AI note taking products is focus. How does it choose what's important vs what isn't? The better it is at distinguishing the signal from the noise, the more powerful it is. I wonder if there is an in-context learning angle here where you can update the model weights (either directly or via LoRA) as you get to know the user better. And, of course, everything stays private and on-device.

hey swyx! hope you’re doing well too. we are exploring some other avenues to the core Phind business and this is one of them. i’m excited by the rate of improvement of the underlying models in this space and it reminds me of the early days of the AI search space when we started working in it in 2021.

The text editing should work well in the chat -- did you have a chance to give it a try?

Thanks, this is good feedback! I'll take a look.

Founder here. Sorry to hear that. Let me know if you are still getting errors.

Yes, I think the same analogy applies. Given a binary choice of a developer not releasing any code at all or releasing code under this type of binary "open-code" license, I'd always take the latter.

I am personally in favor of true open source (e.g. Apache 2 license), but the reality is that these model are expensive to develop and many developers are choosing not to release their model weights at all.

I think that releasing the weights openly but with this type of dual-license (hence open weights, but not true open source) is an acceptable tradeoff to get more model developers to release models openly.

It doesn't seem to have open weights, which is unfortunate. One of Qwen's strengths historically has been their open-weights strategy, and it would have been great to have a true open-weights competitor to 4o's autoregressive image gen. There are so many interesting research directions that are only possible if we can get access to the weights.

If Qwen is concerned about recouping its development costs, I suggest looking at BFL's Flux Kontext Dev release from the other day as a model: let researchers and individuals get the weights for free and let startups pay for a reasonably-priced license for commercial use.

I think this is a separate issue. No model provider currently obtains express permission from the content they train on. But some model providers, like BFL, can choose to give back to the open-source/weights community even when they don't have to. I think this outcome is strictly better than them choosing not to give back, which they totally could have done.

This is awesome, and kudos to BFL for releasing the weights. The financial sustainability of open-source is hard to get right, and giving academics this model for free while charging a reasonable licensing fee for startups is something I think makes sense if it allows BFL and others to continue releasing open-weight models.

Congrats on the launch!

I love this idea. We experimented with building an AI coding agent that we showed to a small set of users and the most common feedback was confusion over what exactly the agent did. And so, I think that something like this can solve that problem, especially as AI performs increasingly complicated edits.

Phind was fine-tuned specifically to produce inline Mermaid diagrams for technical questions (I'm the founder).

Very cool, congrats on the launch!

That would be great! Can you email me at (my first name) at phind (dot) com?

Thanks for sharing. It just worked well for me: https://www.phind.com/search/cm753jbx800013b60odargu2h.

Thank you, I hear you. We'll find a way to make it so you can try some pro features, like automatic additional searches or Phind-405B, for free.

This is supported! You can use the https://www.phind.com/search?q=my+question syntax.