AI Workbooks – A notebook interface for LLMs, image and audio models | Hacker News Reader

AI Workbooks – A notebook interface for LLMs, image and audio models

lastmileai.dev

196 pointsFlux15933 comments

I notice that the python community seems to love workbooks/notebooks but I have to say as a non-python dev, they drive me up the wall. I just want to deal with “regular” code and not have an entire gui environment with code and text intermingling somewhere in the cloud.

I think AI really needs a de-pythoning in general.

Please don't mix up python and notebook. The linked product is not even python! It's barely a variation of ChatGPT Web UI. And many python people love VS code too! It's a vast community with many different tastes.

That's said - As LLMs move "AI" field from self-hosted to managed, the surrounding API ecosystem seems to move away from Python, even though slowly. For example LangChain has both Python and JS implementations. So De-pythoning is kind of happening.

As a long time (16+ years) user of python and Vim, I personally find the notebook format extremely unergonomical. Maybe it would be a good option for those who haven't spent 10000 hours mastering a text editor and relating tooling, but there exist old school python devs that very much prefer a good old text editor like everyone else.

> I think AI really needs a de-pythoning in general.

Yes, absolutely. The state of the Python ecosystem is what remains after a nuclear apocalypse.

We need more native stuff, ggml.cpp for one is a super important project.

People say that, but ironically most everyone that uses it just goes with the python wrapper for llama.cpp lol. Everything except kobold anyway.

"We need more native stuff, ggml.cpp for one is a super important project."

Please, not C++

Checkout Clerk from the Clojure community [0]. Presumably it would be possible to extend a new language like Mojo (Python-superset) to support that kind of dev workflow to AI.

[0] https://github.com/nextjournal/clerk

The amount of critical production code that's straight up copy+paste from notebooks is pretty jarring too. Lots of pasta and lots of extraneous libraries (like gradio) all over the place

I suggest AI people to look into Mojo[1]. They are building a language that is fully compatible with python, but when performance is needed it can become typed and compiled. Its very early days, but that seems to be best way to depythonize

[1] https://www.modular.com/mojo

Agreed, let's bury the secrets of technological consciousness in even more opaque and hard to read layers of code.

It's a good way of separating useless crap from good code.

If the main code is in a notebook, ignore it completely.

If the code is in python but no ipynb, it may be useful.

If it's a well structured code base, and there are some ipynb files, check it out and determine if it's good software. Then the ipynb files are like only a few lines, but show decent plots and the package may end up being useful. Python can be good, it's just unfortunately less and less likely due to far too many idiots using it.

At least it's not R though.

Hey folks! I'm Sarmad from the LastMile AI team. We'd love your feedback on this as you try it out. Here's a video example of using AI Workbooks: https://www.youtube.com/watch?v=19vRQQNZLFo

Example Workbook from the video: https://lastmileai.dev/workbooks/clj530sqs000znztcmd5qr7v6

It looks pretty nicely integrated. Have you considered providing plugins for Jupyter notebooks? I would love to integrate this into my workflow but all my work is typically done in self hosted Jupyter notebooks.

We are working on that at the moment! We currently have a Python SDK that can be imported into a Jupyter notebook for more advanced scenarios (https://github.com/lastmile-ai/lastmileai-python).

We are working on updating the SDK for more advanced scenarios (e.g. running bulk evaluations, comparing different workbooks programmatically, etc.). Are there any specific workflows you'd like to see enabled in self-hosted Jupyter notebooks?

This looks great! What are some of the verticals you think this will be most immediately applicable in, and are you integrating with any? I am thinking a lot of ads/marketing work can be jump-started with these notebooks.

We'd love to see how people use them to understand that better. Our initial hypothesis is that AI Workbooks fit that sweet spot where a chat interface isn't enough, and a Jupyter notebook is too heavyweight. So most likely this will be helpful for technical users and teams who want to experiment/prototype, and collaborate on the results.

Ads/marketing is definitely one vertical, but even for prompt engineering in other verticals we can see this be valuable.

We're especially excited about the multi-modal usecases where you chain multiple models together, but would like the community feedback direct our product direction.

Any thoughts on integrations with vector stores?

We already support vector stores! If you go to https://lastmileai.dev/models, you can New > "Tune new model", which under the covers creates embeddings for your data (either file upload or scraped from a website). You get a new "Model Fork" from this, which you can then use in a Workbook.

We're working on more complex scenarios with vector stores, including API integrations.

Very nice! We've been using something very similar that we built in Streamlit, and it's been incredibly helpful for enabling non-dev employees to do some pretty sophisticated work with AI.

Genuinely excited to see these types of tools take off.

We agree! Curious what you've seen with Streamlit -- what kind of workflows are people finding most valuable? We are looking for feedback for future product direction.

I’m amazed that this idea never crossed by mind or my twitter feed. I am usually a dissenter of notebooks, but I could see this being a great use case.

Especially for debugging LM programs. Usually I make logs and have to manually set breakpoints in a program to see where the language algorithm or agent is going wrong, but it would be much simpler to break the algorithm down into cells where I can see where a tangent is arising, and iterate over some different prompts or control flow to get a feel for what can happen at a specific juncture.

While building this we were pleasantly surprised how intuitive the interface feels for prototyping. For example, you can insert chat cells in-between other chat cells, which affects the overall messages (and response from the model). You can tweak the model parameters at every cell. And once you're happy with it you've got a workbook saved that you can revisit in the future (including text annotations).

I'd love to see any workbooks you create for your debugging scenario. Please share them to see if we can improve that scenario further.

I think there is some interesting interface exploration to do in "freaking ai, give me what I want even if i'm not yet sure what I want" interfaces.

In our open source chatcraft.org we focused on retrying with different openai(for now) models. https://github.com/tarasglek/chatcraft.org/pull/99#issuecomm...

This looks cool! Is it possible to feed the output of one cell into the input of the next? That would really unlock the value of a tool like this for me.

Absolutely! We are working on that right now. Currently you can copy the output of one cell and manually use it as input for another cell, but we will be making that flow much simpler soon.

Pretty interface, although I remain unconvinced of how I'd actually use it. If I'm just prototyping for myself, LLM providers offer a decent history, and I rarely need to share notebook-style explorations of LLMs with my team. For production use cases at logicloop.com/ai we just add our prompts into code.

What's the use case you're envisioning people using AI notebooks for?

Does it pick up context from previous cells like Wolfram’s Chat Notebooks?

https://writings.stephenwolfram.com/2023/06/introducing-chat...

ChatGPT and PaLM Chat cells do take context from previous cells of the same type (we don't mix messages between them to avoid confusion).

For something like Stable Diffusion it doesn't make much sense where the prompt is going to be isolated.

This was something that we took into account while building and designing workbooks - how will normal users expect a notebook interface to work with chat style models.

I'm curious what the key advantages of AI Workbooks are compared to doing the work on Jupyter Notebooks!

We love Jupyter notebooks, I was tech lead of an entire platform around them in a previous job - but we wanted to differentiate from Jupyter a bit by not being python first.

For AI Workbooks, we thought that the interface is natural language & standard files (images, audio) that people interact with everyday. This makes it much easier for non-developers to get started.

For developers, we also support a python SDK so that you can make a workbook from python or a Jupyter notebook using some simple API endpoints. We're going to keep improving the SDK to improve the integrations too.

I thought this was going to be from lastmiles (a good low level dev on twitch with excellent content on C dev.)

The I realized they would likely hate the idea of working with python, and do everything in C first. So likely not lastmiles, unfortunately.

It's amazing!

This is cool!

Reply on news.ycombinator.com