esafranchik

7 karma · joined April 3, 2023

Building AI-native devtools

esteban[at]statespace[dot]com

submissions comments

esafranchik··on Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items?

e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.

esafranchik··on Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

Two follow-ups:

1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets?

2) How do you measure token use without the agent, prompt, and tools?

esafranchik··on Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

Is the benchmark measuring one-shot retrieval accuracy, or Coding agent response accuracy?

esafranchik··on [dead]

Does this work with any tool calls that make an HTTP request? e.g. calling `curl` directly vs writing a script to make the request, then calling it

esafranchik··on Show HN: Continual Learning with .md

Have you noticed an relationship between recall and the number of files/memories?

esafranchik··on Show HN: I built a tool that turns any API into a CLI for agents

The new API2MCP

esafranchik··on Show HN: I open-sourced the library I use to track ML experiments with GitHub

Hello HN! I built Cubyc to manage my ML research in grad school.

It lets you store all your experiment metadata with cloud-based repo providers like GitHub, GitLab, and Bitbucket. Plus, you can directly use SQL to dive into your runs.

I kept it simple as a headless Python library so it's easy to install and integrate, and doesn't weigh down projects.

Feedback is appreciated!