saqadri

Nice! Let me know if you try out aiconfig. We would love contributions

Must be nice :). I wish I was in that 40%

Thanks! Would love your feedback both on the UX editor as well as the APIs for interacting with aiconfig. We are also working on a local editor so you can have a locally hosted "AI workbook" playground

Please definitely let me know when you get a chance to try it out! The readme has a link to our discord too if you want to get in touch (and you can email me directly too)

We have a bit of context about this in the readme: https://github.com/lastmile-ai/aiconfig#what-problem-it-solv.... The main issue with keeping it in code is that it tangles application code with prompts and model-specific logic.

That makes it hard to evaluate the genAI parts of the application, and also iterating on the prompts is not as straightforward as opening up a playground.

Having the config be the source of truth let's you connect it to your application code (and still source controlled), lets you evaluate the config as the AI artifact, and also lets you open the config in a playground to edit and iterate.

For example, compare how much simpler openai function calling becomes with storing the stuff as a config: https://github.com/lastmile-ai/aiconfig/blob/main/cookbooks/... vs using vanilla openai directly (https://github.com/openai/openai-node/blob/v4/examples/funct...)

Hi HN! OP here - this is our first open source project and we are really looking for your feedback about this direction, because we haven't seen something like aiconfig in generative AI developer tools yet.

Our basic premise is that AI application development should be config-based, so you can track the prompts, models and model parameters being used more rigorously. Having this AI artifact then lets you iterate on it separately from your application code, and also set up evals that provide "test coverage" for the gen AI parts of your application.

We were also inspired by the ipynb format for Jupyter notebooks, and you'll see parallels to that in the aiconfig format.

Please ask any questions, and share your thoughts on config vs. code.

This is cool. This might be a silly question, but what are the scenarios where it's useful for fine-tuning on the edge with small devices? I get inference on the edge, and curious about metrics on that for Whisper, but isn't it better to fine-tune on beefier infrastructure and then deploy it for inference on the edge?

You may not need CRDT per-se, but building a collaborative experience is so difficult. I worked on collaborative systems for a bit, and also have read a bit about how Figma and Notion do it (this is a good read: https://www.figma.com/blog/how-figmas-multiplayer-technology...) -- it's still super hard to get right.

This talk by Karri about Linear's "sync engine" is also a good watch: https://www.youtube.com/watch?v=Wo2m3jaJixU.

What are folks most excited about? I personally would love some of the code interpreter stuff as an API. But, currently trying to fix the breaking changes on their latest pypi package -- maybe before the livestream starts

You can use AI workbooks UI to access GPT3 and GPT4, it's still working: https://lastmileai.dev/workbooks/clk8qctki004sr1mkhv0eel64. (Full disclosure: I work on it)

From the lastmile team here. Would love your feedback! Origin story -- we're a group of ML developers who want to build better developer tools to make AI more accessible to non-ML engineers. We built ML Ops tools in Big Tech for many years (Jupyter notebooks, model management, experimentation platforms, etc), but those were targeted towards ML researchers.

With LLMs and other generative AI models, we want to provide simpler tools to bridge that 'last mile' gap between the core foundation models, and using/personalizing/integrating them in your applications. Hence the name :)

Ah interesting! What models were you running inference on (assuming locally)?

Hugging Face has such a large repository of models! Curious how folks have been exploring HF models to date, and if this notebook-style environment to try out many models in one place would help?

Incredible! The march continues to get more models to run on the edge, much faster than I anticipated. The static quantization and slicing techniques here are pretty cool

There was a free trial period that expired, and there was no free option for additional documents that required multiple signers.

TIL there was such a thing as a passport ranking. I had assumed an EU country would top the list because of the eurozone, so pleasantly surprised by many of the countries in the top 10 (especially Singapore!)

Just today I was forced by Docusign to pay $45/user/mo in order to continue using the service for a single document I had to send out for signatures. Seeing this pop up on HN right after feels really nice. The cloud-hosted version seems to be very simple to use, so nice job on this.

Like some of the other comments pointed out, the key element here is trust -- in the 3rd-party platform collecting signatures, and in the confidence that it cannot be manipulated. These are solvable challenges, but calling that out explicitly in your documentation and website copy will help convert skeptics, or at least convince them to give it a try.

There are several areas where we are seeing LLMs being deployed successfully in production. Often those are in comparatively boring areas, like NLP, classification and intent recognition, for which LLMs work quite well. So far most of this adoption is at companies with existing ML expertise in-house.

We haven’t seen any usage of the more exciting-looking demos of semi-autonomous agents in production yet, and probably won’t for some time.

20,000 sq ft of space for a computer that processed only 1200 cards/min. The tone of the narrator made that sound very impressive, which it was back in 1973.

50 years from now I wonder what things we are amazed by today will sound silly to future technologists.

As an AI startup founder, I worry pieces like this miss the point — we are so early into this new era of computing that it’s premature to call any winners.

It’s also simplistic, because AI-related innovation is happening across companies of every size at every layer of the tech stack. Even Open AI with its state of the art foundation models needs to compete with both open source and large companies coming out with new LLMs.

Pieces like this may also discourage bright people from taking risks and building cool things, because they worry that this is just a big company game. It’s just too early to think like that.

Somewhat related to shadow programs for leadership roles -- at my previous company we had shadow program to give employees a peek into performance review discussions. It was always a black box and created uncertainty for many IC's, but the ones who attended the mock performance reviews really appreciated the insight into the process.

Amazing read, convincingly explains a lot of confusion around the aftermath of the search operation. And kind of mind blowing that IUSS exists primarily to detect submarine movements around the world.

I would love to learn more about the technology — are these wireless transmitters? Undersea cables all around the oceans of the world?

For us the complexity has all been about the infra around the AI stack. For example, having a k8s cluster for running inference for open source models (Stable Diffusion, Bark, etc.). Beyond that it’s been custom abstractions around Open AI/PaLM APIs and vector stores (pinecone and Faiss).

This stack is evolving so quickly but we are convinced it goes back to a distributed systems/infra problems when it comes to complexity, unless you’re training your own foundation models.

I wish the “how much an engineer can focus” section was first, because that’s been by far the biggest bottleneck in most organizations in my experience.

All the other things the engineers can themselves address (e.g. better tooling, etc), but the organization’s culture around protecting makers schedule is impossible to address individually

A few tips that have worked for my teams in the past (used to be an engineering manager, and what you're describing isn't uncommon): 1. Shift perspective: try to believe that everyone is trying to help, and wants what's best for the project and for you. Even if that may not always be true in every circumstance, it helps to believe that, because it makes others' input feel less attacking.

2. Change tactics: try to have some design discussions async (for e.g. by getting comments on a doc ahead of time). This gives you time to process their feedback, and the meeting can be a place to discuss it.

Ultimately the outcome may still be the same in terms of the technical decisions, but I have found perspective and tactics can affect how the team feels about them, and also creates space for good ideas you may not have considered to be heard constructively.

Hope this helps!

It's mind-blowing to me that Alexander Hamilton wrote the vast majority of these. Unreal impact

Why do you think Bing chat has been under-utilized? I also don’t find myself reaching to it as often even though the few times I’ve used it I have been impressed.

I have noticed that Notion having a built-in AI assistant helps with writing because I hardly ever use something generated verbatim — so it’s easier to be in an interface that is editable.

Perplexity is also good for saving and sharing different queries.

Interesting write up! Would love to learn more about the ‘why’. The post describes one (that etcd has low usage), but that alone doesn’t seem to be a compelling enough reason to do this.

It is fascinating how some ancient techniques had remarkable properties even though they didn’t have the tools or science to explain it back then. I recently learnt that ancient concrete has self healing properties because of the reaction of air/water with lime and volcanic ash.

Makes you wonder how far we can get with trial and error.

We did some research into the different ways of evaluating LLMs, but there is a lot of literature on many different approaches, e.g. scores like BLEURT, precision/recall if you have ground truth data, all the way to asking GPT to be a human rater.

Are there evaluation strategies that worked best for you? We basically want to allow users/developers to evaluate which LLM (and specifically, which LLM + prompts + parameters) has the best performance for their use case (which seems different from the OpenAI evals framework, or benchmarks like BigBench).