mfalcon

I was asking myself the same question the other day, some basic search thing just didn't work.

I'm doing something similar to the OP but I'm not a CTO but a "manager". Maybe the title is "we're all managers now". The CTO role is bigger than that.

"Outputs are disposable; plans and prompts compound."

I agree with this and it aligns with the general opinion about what is the true value the SWE's bring to the table.

Another argentinian here, nice username! :P

That sounds nice. Sometimes I listen to a really great fragment of a song (the final minutes of Descending from Tool) and I think: "It'd be great to hear songs that have this kind of fragment".

I don't know if that's possible or not, but it'd be much better for me than the recommendations from popular streaming services.

Wow! I was thinking about doing something similar (extracting audio features to find really similar songs) some weeks ago. I love the idea, it'd be great to have something like this to discover music too, but it'll be difficult to discover new music because it won't be present in the collection.

And you probably won't finish that too :)

You don't. You have to separate concers between deterministic and stochastic code input/output. You need evals for the stochastic and mocking when the stochastic output is consumed in the deterministic code.

At least your content is going to be included in the LLMs for years to come. Kind of a legacy:)

Yes, I've been working in the machine learning space for a while and I agree that it's an approach that should've been followed years ago. But I also think that now the problems of not following it are bigger, because now we have "AI as a service" so everyone can have AI in their systems.

The Moat is the people.

I work in a company that sells chatbots. We use LLMs to understand the intention of the user: start a flow, set a slot value, cancel a current flow...

After that, all the logic to handle the identified action and the response is implemented using normal backend code.

We sometimes use an LLM for the response when the user asks for a FAQ or maybe for rephrasing an output sentence.

OpenAI, Anthropic, Google(DeepMind), Mistral, Groq, Meta(AI).

I think that the description fits what a data scientist usually does. LLM knowledge could be considered as part of the stack.

Amazing contribution Hamel, thanks for sharing. I though it was behind a paywall as you said, very glad you decided to open it. Looking forward to start with this course.

One big issue with hybrid is that you lose optionality because you no longer can live far from the office:

- You can't live far away from the office

- You can't take some weeks/months to work from abroad

- You lose flexibility on your schedule when in the office

Of course no approach is perfect but I personally wouldn't say hybrid is a nice balance, it's better than going to the office every day but nowhere near as being 100% remote.

I wouldn't say "not at all" because I think the industry is going to change at least a little bit, so with change comes inestability.

I think that my ability regarding problem solving will be valuable in the future, maybe if most of software engineering becomes fully automated I could build some valuable product easier, just thinking, writing and using feedback loops for the continuous iteration of the product. That would be awesome, I like programming but I love problem solving.

If the above is not possible, I've some investments and savings that could help me take some time to adapt to the new environment and find something where I could provide some value.

And if the above is not possible there are some possibilities:

- we live in leisure while the machines do the work for us -> great!

- we don't exist anymore, the machines won -> bummer, we had a nice ride.

- I'm no longer useful to the new environment -> I prefer not to think about this :P

Not worried, I am confident that I will always find a way to provide value.

I've been building data projects for real estate for a while and applying LLM's for conversational products but I've never mixed both of them. I think something like a conversational buying/renting assistant could be feasible but I don't think it's something interesting to monetize.

I'm making two LLM's negotiate the exchange of a product, price is the main issue but I'm trying to make them negotiate another issues too in order to avoid the "bargaining" case.

I've tried several models and gpt4 is currently the one that better performs, but OS LLM's like Mixtral and Mixtral-Nous are quite capable too.

https://github.com/mfalcon/negotia

I love Ollama's simplicity to download and consume different models with its REST API. I've never used it in a "production" environment, anyone knows how Ollama performs? or is it better to move to something like Vllm for that?

I think there is a lot of ground to cover in "RAG". Most of the demos or tutorials online seem to simply use a vector database to retrieve similar documents according to a cosine distance.

I'm now working on a "hybrid" search combining lexical and semantic search, using an LLM to translate a user message into a search query to retrieve data.

As far as I know, there's not a "standard", the field keeps moving and there are no simple answers.

Antifragile, Meditations.

I was thinking about budgeting lately (I don't do it already) but the extreme inflation in my country, Argentina, makes it a bit difficult.

I had the idea of introducing some kind of "randomness" to the amount of money I can spend on non essential expenses in order to avoid the lifestyle creep. One month maybe you have enough money available to spend it on several eatouts and another month you can't.

Thanks! Really helpful. I've a 3090 at home and my idea is to do some testing on a similar config in the cloud to have an idea of the amount of requests that could be served.

I've been trying Ollama locally. I've yet to know how it'll behave in a production setting.

Here too, Meditations and "I am that".

I use ChatGPT a lot for simple documents transformation (csv to json for example).

And when the documents are too big to fit in a prompt, I ask chatgpt to build a simple Python script to do it.

Hey man, thanks for the article. I like it that it is concise and simple. One thing that's not clear to me is about the inference stage: where does this inference process runs? Do you need to run it in a GPU powered instance or could it be runned in a consumer laptop?

It's pretty similar to our current situation, how old is your child?