__jl__

Has anyone compared the updated Copilot with Cursor? The main updates I am wondering about are model selection and multi-file edits. I used copilot before these features, changed to Cursor and now I am wondering how much Copilot has closed the gap.

Gemini is surprisingly good at this. Look at example 5 here: https://developers.googleblog.com/en/7-examples-of-geminis-m...

They also have a colab notebook with more examples linked in the article.

I really like the idea of ColPali and products building on it but I am still unsure about the applications for which it makes most sense. We mostly deal with reports that are 80-90% text, 10-20% figures and tables. Does a vision first approach makes sense in this context? My sense is that text-based embeddings are better in mostly text contexts. Layout, for example, is pretty much irrelevant but plays into vision-based approaches. What is your sense about this?

I think Gemini Flash 1.5 is the best closed-source model for this. Very cheap. Particularly compared to GPT4o-mini, which is priced the same as GPT4 for image input tokens. Performance and speed is excellent. I convert each pdf page to an image and send one request per page to Flash (asynchronously). The prompt asks for markdown output with specific formatting guidelines. For my application (mainly pdf slideshows with less text), the output is better than any of the dedicated tools I tested particularly for equations and tables.

Thanks for sharing your experience.

We also get pretty reliable JSON output (on a smaller scale though) even without JSON mode. We usually don't use JSON mode because we often include a chain of thought part in <brainstorming> and then ask for JSON in <json> tags. With some prompt engineering, we get over 98% valid JSON in complex prompts (with long context and modestly complex JSON format). We catch the rest with json5.loads, which is only used as a fallback if json.loads fails.

4o-mini has been less reliable for us particularly with large context. The new structured output might make it possible to use mini in more situations.

There is another big change in gpt-4o-2024-08-06: It supports 16k output tokens compared to 4k before. I think it was only available in beta before. So gpt-4o-2024-08-06 actually brings three changes. Pretty significant for API users

1. Reliable structured outputs 2. Reduced costs by 50% for input, 33% for output 3. Up to 16k output tokens compared to 4k

https://platform.openai.com/docs/models/gpt-4o

Came here to say the same. "indiscriminate" doesn't really make sense. It's very deliberate.

However, there is one scenario: Scraping of web data. In that case, AI labs might know what is model generated.

Meta just claimed the opposite in their Llama 3.1 paper. Look at the conclusion. They say that their experience indicates significant gains for the next iteration of models.

The current crop of benchmarks might not reflect these gains, by the way.

I disagree. Not saying the other benchmarks are better. It just depends on your use case and application.

For my use of the chat interface, I don't think lmsys is very useful. lmsys mainly evaluates relatively simple, low token count questions. Most (if not all) are single prompts, not conversations. The small models do well in this context. If that is what you are looking for, great. However, it does not test longer conversations with high token counts.

Just saying that all benchmarks, including lmsys, have issues and are focused on specific use cases.

They already invested 1 billion in 2019, I think. So maybe they already own part of OpenAI and $10 billion brings their share up to 49% at a $29 billion valuation. Maybe I am wrong about the details here but these numbers do not mean that Microsoft is undervaluing them.

Is there still any verification involved or is it just blue check mark for money? Does providing your payment details count as some form of verification and is that information linked to the Twitter user?

The iPad Pro is an amazing piece of hardware! But the last iterations make zero difference for most consumers. The hardware improved rapidly over a couple of years. Both the 2017 model (move from 9.7 to 10.5 screen) and the 2018 model (new design without home bottom, USB-C, Gen 2 Pencil) were huge steps. Magic Keyboard from 2020 was a great addition as well and works with the 2018 model. For my own usage, the 2018 model is pretty much feature complete from a hardware perspective. Software makes the big difference now.

I am still amazed by my 2018 iPad pro four years later!

Can you point to newer products that reflect these improvements?

For Zotero users: Try the add on zotfile (zotfile.com). It allows you to extract PDF annotations and save them as notes in Zotero (among other things). Full disclosure: I am the developer.

First big news company, I think. And they send out free google cardboards to all Sunday home delivery subscribers. Pretty cool and interesting to see how that plays out.