prabhatjha

This writeup is very informative. Learned about few OSS frameworks that I was not aware of. Amazing engineering work. Kudos to you all.

Congrats to you and team. Looking forward to awesome stuff you all build as part of this partnership.

This article does a good job of describing hard problems with ANN search and how different algorithms solve or don't solve. https://dzone.com/articles/5-hard-problems-in-vector-search-...

Initially I thought we need a dedicated vector database but as we tried to build even a simple gen-ai applicaitons we realized that we need another "regular" database to build a complete application.

Instead with DataStax's Vector Search, we designed a document style API and corresponding clients that give you a Vector Native experience to do CRUD of vector and meta-data as well CRUD of other data models. Here is one client ref for example https://docs.datastax.com/en/astra/astra-db-vector/clients/p...

We will be publishing the github repo in the next couple of weeks. As you can imagine this is happening in real time and we want to cross check few things before we publish the repo.

I will reply to this thread once that has happened.

Thank you.

Nice. What kind of agents are you building? I am especially interested in your data requirements.

You can totally do this in Cassandra. See https://docs.datastax.com/en/astra-serverless/docs/vector-se...

This is interesting because it does not mention Vector database powered by Apache Cassandra or the hosted serverless version DataStax Astra. Here is write up we did on 5 hard problems in Vector database and how we solved them. https://thenewstack.io/5-hard-problems-in-vector-search-and-...

In full transparency: I work for DataStatx and lead engineering for Vector database.

There is a great write up by Jonathan Ellis, Apache Cassandra committer and DataStax co-founder on hard problems with Vector Search. https://thenewstack.io/5-hard-problems-in-vector-search-and-...

Exactly.

What a success story for yet another open source project

This post is misleading. It's not about Kafka. It's about need for a messaging system or not which is a completely different question.

What a great tribute you have written. When I first found about swizzling through a seasoned iOS dev I was blown away. The swizzling capability in obj-c basically helped create my first startup, InstaOps, a long time back which allowed no code change to instrument an app to capture logs and network performance metrics.

This is a fantastic idea -- the kind you see and go why the heck this was not done before. Such a huge time saver.

Agree. You don't want to do Fashion Driven Development (FDD). You would end up chasing every shiny thing that comes out and not really master anything.

Pick any one stack, build some simple projects and then expand on it. You would also want to be able to read well written code from others on github and learn from those.

I am assuming you are talking about our deployment on GCP Cloud Run? We have thought about sending a heartbeat API call. It we notice any user experience friction because of this lag then we will definitely do that. As we said in the blog, this has not been a major pain point as of today.

We supported batch api calls in v0. But as those API calls increased a new instanced would get spun up but boot time was longer. To get around it, we would have to keep more instances running all the time which obviously costs more money.

Thanks for the tips. Having everything colocated in a k8s cluster will definitely help with latency and probably overall infra cost but it will be at the expense of engineering times spent on running a k8s cluster in prod.

Fingers crossed that we keep growing which would mean that we can justify working on v2 architecture.

Totally. We did not have a need for custom models initially. We could load all our models on one VM so there was no need. We were tempted to get on the bandwagon. ;-)

Because we were loading all the models startup time was long which meant that server would return 5xx errors which created more instability. We would had to do some engineering around it with a mix of config and code changes.

The bigger issue was that he had to use bigger machine as we added more custom ML models for our customers. New architecture gives us huge $$ saving and more visibility into performance of each model.

I think that AWS are the "bad guy" here. I am happy to be proven wrong but they don’t have a track record of contributing code to the OSS projects they provide as a service. Whenever they are forced like it was the case with Mongo and now with Elastic they are using their power to have a competing OSS project. There is nothing wrong with this business wise but it's totally against spirit of OSS.

If you look at Red Hat on other hand when they decided that Kubernetes was the way to go for their OpenShift project they put lots of engineering resources for upstream k8s.

I totally agree. Deploying microservices and running k8s sound easy until you actually do it. For example, just see this section of k8s docs about exposing services: https://kubernetes.io/docs/concepts/services-networking/serv... . You need to understand many different concepts first to get this right. However, I think once you cross that hurdle, the traditionally harder stuff like auto scaling, rolling upgrade becomes relatively easier.

However, I would say that it's really early days for K8S and the ecosystem around it. As long as K8S does not try to solve every problem in the world and focus on the problems it's designed to solve, things will get easier and then may be a 60-min video can do some justice. ;-)

This is a great paper. It helped us think more rationally and helped us realize that we were not alone having difficulties running ML in production.

Working for a early stage startup depends on the stage of life, your financial freedom and your ability to hustle.

Early stage startup requires that you work a lot more than 40 hours of week with a big pay cut compared to what you would get at an established company. This usually is not a problem for individual or couple who don't have kids but very difficult for people with school kids. You have less time and less money -- both have direct impact on how your kids grow up. Is it worth taking the risk? This depends on your values and how you define success in life.

Your roles and responsibilities are not well defined. Even for a software engineer, you have to split your time helping sales, customer support and marketing. I personally find this aspect super exciting but I know a lot of people who don't like and wont thrive in this kind of environment.

Most startups fail. Founders and VCs can screw you -- intentionally or unintentionally. Odds are just stacked against you if you define success by financial gains.

I would also recommend Camille Fournier's The Managers Path book for folks in tech. You will be able to relate to this book whether you are an individual contributor or seasoned manager.

Thank you for the link to the paper. We had not come across that one.

I totally agree with this. We learned it pretty quickly that classification does not generalize across domains so we narrowed the problem space by focusing one domain at a time followed by predefined and fixed set of categories so that we can measure effectiveness of our solution as we experimented with different algorithms and deployment pipeline.

Got it. Yes, topic classification does not generalize across domains based on what we see in on our implementation.

Ah -- I thought you were talking about classification of these tweets so that politicians know what their followers are talking about. Sentiment analysis is a very small part of what we do and as you said there are tons of examples on web that use Twitter's data in their model.

Forget about other languages, it's sometimes tricky for English as well. Someone said language is the world's oldest API but it's also the most complicated. ;-)

For couple of our Norwegian and Spanish customers we hit Google translate to translate feedback into English and then feed it through our ML engine to classify. Accuracy obviously is not as good as it should but it gives them good insight.