heliophobicdude

Any word on if this using their diffusion architecture?

This looks way better than mermaid. Is there support for this in GitHub PR code blocks like mermaid?

It's super fast to iterate with? Curious what mistakes are you seeing and how were you using it?

I use the instant edit feature with directions on exact code changes.

Wishing they release the Gemini Diffusion model. It'll quickly replace the default model for Aider.

Might be private repo? I get 404 as well

Question for the researchers, can dLLMs be pinned down with a seed? Can they be made 100% deterministic?

I think the lede is being buried. This is a great and fast InstructGPT. This is absolutely going to be used in spell checks, codemods, and code editors.

Instant edits feature can surgically perform text edits fast without all the extra fluff or unsolicited enhancements.

I copied shadertoys, asked it to rename all variables to be more descriptive and pasted the result to see it still working. I'm impressed.

I've been let off the waitlist. So far, I'm impressed with the Instant Edits. It's crazy fast. I can provide a big HTML file and prompt it to change a color theme and it makes careful edits to just the relevant parts. It seems to be able to parallelize the same instruction to multiple parts of the input. This is incredible for refactoring.

I copied a shader toy example, asked it to rename all the variables to be more descriptive and it edited just the variable names. I was able to compile and run in shader toy.

Thank you for sharing this. I'm amazed! Are there any known emergent abilities of it? I ran my evals and seems to struggle in very similar ways to smaller transformer based LLMs

It's a great language for Application-level software on a single platform (could be Linux).

It has some challenges that it needs to solve to do great as a cross platform "general-purpose" programming language.

It's paradoxically high level with its syntax and ergonomics but is tied down to the same cross platform headaches like in low level languages (e.g. cpp). Linking across cross platforms requires lots of careful thought and testing. Unlike cpp, it's not super portable. It requires a hefty 30 MB runtime for some features of the language to work. Try static executable hello world.

That being said, it's possible. You can build cross platform applications with Swift, but you'd still have some of the same kinds of portability issues like in cpp but with nicer syntax and ergonomics.

> I would actually argue that axing Image Playground would improve Apple Intelligence; its mere existence greatly lowers the expectations for how good the whole thing is.

I was expecting a discount. In the onboarding it says "Prices are the same. You won't pay more for autonomous rides."

If that's the case, I would rather give my money to a human driver than just Uber.

Do you still also need to compile a mac build on a Mac?

Is the date April 2 or February 4?

We should NOT give up on scaling pretraining just yet!

I believe that we should explore pretraining video completion models that explicitly have no text pairings. Why? We can train unsupervised like they did for GPT series on the text-internet but instead on YouTube lol. Labeling or augmenting the frames limits scaling the training data.

Imagine using the initial frames or audio to prompt the video completion model. For example, use the initial frames to write out a problem on a white board then watch in output generate the next frames the solution being worked out.

I fear text pairings with CLIP or OCR constrain a model too much and confuse

Back in the old days, top level arrays were a security risk because the array constructor in JS could be redefined and do bad-guy stuff. I cannot think of any json parsing clients that are vulnerable to this.

It looks like there is no SfM used here atm. Removing SfM is something I worked on removing in my similar project https://github.com/afloresescarcega/splatinit

I honestly haven't tried! I'm trying to do this per frame of a live video. I had not considered COLMAP for performance reasons.

Kinda! My ultimate goal is to be able to join multiple 3D reconstructions from a set of iPhones all in different positions! Like hemisphere. The single-view image to 3D is pretty accurate but across lots of iPhones.

Thank you so much for open sourcing this!

I've started a project to initialize each pixel in an image into a mesh of GS. I used a depth map to unproject them into space.

I've been very curious what a few training iterations would do to optimize my scenes. The original 3DGS implementation is not accessible on my hardware at the moment! I really look forward to your training implementation!

Thank you Paul! I noticed the diff editing format mentioned. Does Opus perform better with whole formatting like GPT-4?

So like Apple Vision Pro's EyeSight but for a Waymo?

This is reported to be opening up globally?

Wondering if I could then play PCVR with game streaming on the Apple Vision Pro

Some that I follow are pre-ordering multiples but intending to return all but one. Some did it to secure a launch day model or specific fit.

If anyone is up to the challenge, Apple provides an excellent sample project available for download with low level access and operations on the LiDAR points here and shading all for an iPhone app:

https://developer.apple.com/documentation/avfoundation/addit...

If anyone is interested in this field, we need your help with real-time reconstruction.

Currently a huge challenge is in real-time reconstruction. Approaches involve estimating point clouds from images then optimizing splats on those. Other approaches are using SLAM like LiDAR to have distance of points to the camera but then still optimizing on that.

Optimization is producing good results but takes iterations that are not suitable for real-time.

Pixel-wise splat estimation with iPhone LiDAR could produce good results but need help and expertise

The most prominent critical feedback has been the weight. I would like to know what straps the users used. Also if the double strap one is better for weight

Ah, I believe you meant micro-OLED. The names are unfortunately very similar.

I'm more excited about Samsung's micro OLED with no color filters that will push the brightness and/or reduce power consumption.

AFAIK, Apple's Sony micro OLED does have color filters. So there is definitely an opportunity for another display upgrade.

Hard edges are a challenge right now.

Thinking in 2D for a second, to get a nice crispy edge, you need a long and opaque splat to mark the boundary. Sometimes the long splat could wisp off leaving fuzzy artifacts.

Take this example: https://www.shadertoy.com/view/dtSfDD

Peyman Milanfar [1] suggested using bump functions instead. Bump functions would allow you to specify cut off intervals but still make the whole function smooth and continuous (good for my gradient optimization freaks)

1: https://x.com/docmilanfar/status/1719584410348204233

Great work!!

Question for the authors, are there opportunities, where they exist, to not use optimization or tuning methods for reconstructing a model of a scene?

We are refining efficient ways of rendering a view of a scene from these models but the scenes remain static. The scenes also take a while to reconstruct too.

Can we still achieve the great look and details of RF and GS without paying for an expensive reconstruction per instance of the scene?

Are there ways of greedily reconstructing a scene with traditional CG methods into these new representations now that they are fast to render?

Please forgive any misconceptions that I may have in advanced! We really appreciate the work y'all are advancing!