eskibars

I asked it to talk to a real person: a manager, legal, or compliance employee and it hung up on me

1 year ago, I lived in San Rafael (Marin county, Bay Area). I occasionally needed to go to Palo Alto for work meetings. The fastest public transit option was to take a 40 minute bus to Larkspur Landing, then a 30 minute ferry to the SF Ferry building, walk for 20 minutes, and then take Caltrain for 45 minutes or more and then walk from there. With transfers, at minimum it was a 2.5h journey, but typically 3+h

All to cover a 60 mile / 100 km distance

Wanted to let this group know because this sort of thing is an appropriately sensitive topic in this community, which is also directly their target audience. The AI agent did not identify itself as an AI agent until I asked, which likely violates other laws as well.

I've been a product management lead for 2 commercial open-core companies and people drastically overestimate:

- How much code the community contributes (in both cases, >95% of all code was written by employees hired by the commercial company) - How few commercial resources are needed to support the community (running forums, answering GitHub tickets, etc) - How much financial support is actually forthcoming when there's not some "locked commercial features"

As the paper points out, many of these widely used commercial projects receive a few hundred thousand dollars at most in donations (often much less) but need to employ more developers than that financing can support to maintain a baseline capability to address basic bug fixes (including security fixes) once they become "popular enough" to be known by the masses.

- How well does it work at a specific frequency, if you're just trying to transmit/receive on one specific frequency

- How well does it work on the frequency range(s) if you're working on more than a specific frequency

- How well does it block frequencies that you don't want to send/receive

- How well directional is it to trade off using lots of radiation to blast in many directions vs a higher focus beam using less energy or getting less interference from other directions

- How much physical space do you have in each dimension?

These are just a few examples, but for example you can provide a much "better" connection in almost every sense of the word if you can make your antenna directional (point between the source and destination) only on a specific frequency, and be huge, but most of the time you have some physical space constraints, multiple frequencies to deal with, and the potential that your signal could at least come from some degrees in each the x/y/z axes, and sometimes it needs to be omnidirectional.

Again, these are just examples, but you end up with these types of design considerations that play into larger system design (can you put more transmitters up to encourage directionality, limit frequencies, etc).

There are some well known "base" antenna types like dipole, yagi-uda, circular, and log periodic dipole array if you want to look them up by name and see some of the known tradeoffs and design choices, but virtually any wire can be an antenna and there are an unlimited number of shapes, nearly all of which don't have known radiation characteristics

TL;DR: several factors.

- Fuel prices are up (for everyone)

- Spirit having so many flights out of Florida has hurt them (climate change with heavy storms more common, and ATC staffing particularly bad in Florida, so operations has been harder)

- Leisure travelers have been willing to spend more for luxury travel in the past couple years and Spirit doesn't have a real offering to that market segment

1. Using median is a problem for many projects because many often start off (sometimes for years) with only a few developers before they really take off. The median() number for Elasticsearch was 28, but the max() is 49, which I suspect is closer to the current number.

2. Not everybody uses a pull request flow, and not everyone working on a project is a developer. You could have product managers, technical writers, engineering managers, etc, all working on a project and not opening PRs and you can have users just directly adding commits, especially when a project is young. The original query here for the kong/kong project shows a team size of 6, but if you remove event_type = 'PullRequestEvent' and switch to max(), you get 33.

3. Sometimes, parts of the project are kind of "elsewhere." e.g. Elastic employed a number of Lucene committers to help move Lucene along, Kong employed a number of folks that would e.g. commit/maintain OpenResty plugins which got incorporated in the build system, the docs live in separate repos in both cases, etc.

One other thing I'd be interested in: not just mini PCs but used office workstations. I realized that many offices were selling old workstations that were often just a few years old with things like dual Xeon chips and 64GB of RAM or more with support for a few hundred GB for only a few hundred $. Things like 2ish generation old HP Z400/Z600/Z800 series. They make for great home lab virtualization machines and can often support 2+ GPUs and a boatload of additional peripherals. I'd love to see something like this that lets you find those as well

There's no data dump: I understand why, but it means everything is behind a pretty proprietary offering. And one which "only" appears to provide geolocation, which is fine if that's what you're interested in bit which is pretty orthogonal to the article's apparent goals where the author is interested in wireless security specifically

I later moved to the SF bay area and was terrified by earthquakes and the potential building collapse, but my colleagues from the area shrugged that off.

In the end, these are all kind of similar risks, and we all just grow accustomed to the risk we're familiar with

I've led several product management teams in my career and many engineers, executives, and customers I've talked to have this impression that if they put some bit of software under AGPL, then its nature is such that it's super viral: that potentially the entire product would necessitate becoming AGPL if the AGPL code is used in any fashion in the product. This has been primarily perpetuated from Google's statement on it: https://opensource.google/documentation/reference/using/agpl... and engineers that now there's a lot of lore around AGPL generally causing so much fear in any of big-tech company potentially needing to open source their entire service that it's treated as "companies wouldn't dare include our AGPL software in a competitive offering because then they'd have to open source their entire hosting stack, and nobody is going to be willing to do that."

My understanding that I heard through the grapevine is that this fear was what MongoDB was relying on for so many years and they found out that one of the big tech companies was about to offer a competitive MongoDB-as-a-service, which caused them to make their shift away from AGPL. Other (mostly database) OSS vendors caught wind of it as well through their networks, and that's why various OSS database products didn't go to AGPL but instead to proprietary licenses.

1. There are natural language questions being asked 2. There's ambiguity in any of the query terms but which there's more clarity if you understand the term in context of nearby terms (e.g. "I studied IT", "I saw the movie It", "What is it?") 3. There are multiple languages in either the query or the documents (many embedding models can embed to a very similar vector across languages) 4. Where you don't want to maintain giant synonym lists (e.g. user might search for or document might contain "movie" or "film" or "motion picture")

Whether you need any of those depends on your use case. If you are just doing e.g. part number search, you probably don't need any of this, and certainly not semantic/vector stuff.

But semantic/vector systems don't work well with terms that weren't trained in. e.g. "what color is part NY739DCP?" Fine tuning is bad at handling this (fine tuning is generally good for changing the format of the response, and generally not all that good for introducing or curtailing knowledge). Whether you need a keyword search system depends on whether your information are more of the "general knowledge" type or something more specific to your business.

Most companies have some need for both because they're building a search on "their" data/business, and you want to combine the results to make sure you're not duplicating. But I'll say I've seen a lot of companies get this sort of combination business wrong. Keeping 2 datastores completely in sync is wrought with failure, the models all have different limitations than the keyword system limitations which is good in some ways but can cause unexpected results in others, and the actual blending logic (be it RRF or whatever) can be difficult to implement "right."

I usually recommend folks look to use a single system for combining together these results as opposed to trying to invent yourself. Full disclosure: I work for Vectara (vectara.com) which has a complete pipeline that does combine both a neural/vector system and traditional keyword search, but I would recommend that folks look to combine these into a single system even if they didn't use our solution because it just takes so much operational complexity off the table.

The OP paradedb blog states for example:

> Future-Proof: Thanks to the copyleft provision, cloud vendors cannot easily resell our project without our consent. This gives us confidence in our ability to monetize without fear of predatory competition.

IANAL but my understanding of reading the various legal analyses of AGPL (which I've read several) is this part is not true: cloud vendors absolutely could resell their project without their consent, provided those cloud vendors don't modify the code. From an end user perspective and for an OSI OSS advocate those attributes are likely in your best short-term interest: they promote competition. But in the long-term, if the company launching an AGPL product believes it protects them from this, they're going to eventually find out what Mongo did: that it doesn't. And the result of that is that they'll either need to accept that there will be cloud vendors competing with them on the IP they created or relicense. Commercial interests/shareholders being as they are and many of these companies being VC-backed startups, the latter is frankly a more likely scenario.

https://katedowninglaw.com/2019/09/08/the-great-open-source-...

as a really great read from a lawyer on the various factors and motivations, and which in particular touches on AGPLs lack of ability to protect what many of these companies are worried about most. Specifically:

> That understanding was the genesis for the GNU Affero General Public License 3 (AGPL 3) published in 2007: it was meant to solve for this exact problem by adding language to the GPL 3 which required companies to provide source code for software that users interacted with via a computer network if the software was modified AGPL 3 code (or a derivative work of such modified AGPL 3 code). But even the AGPL’s obligations can be avoided by simply not modifying the AGPL 3 code, which there is often no reason to do, or by building layers between the AGPL 3 code and proprietary code. That’s why a lot of these middleware companies didn’t choose to relicense to AGPL 3 and why MongoDB, who was already using AGPL 3, chose to revise the AGPL 3 to expand the circumstances under which services running on AGPL’ed code must open source the previously proprietary parts of those services.

FYI for folks just skimming this, shards can affect scoring, but they don't have to. 2 mitigations: 1. The default in Elasticsearch has been 1 shard per index for a while, and many people (if not most) probably don't need more than 1 shard

2. You can do a dfs_query_then_fetch query, which adds a small amount of latency, but solves this problem

The fundamental tenant is accurate here that any time you want to break up term statistics (e.g. if you want each user to experience different relevance results for their own term stats) then yes, you need a separate index for that. I'd say that's largely not all that common though in practice.

A more common problem that warrants splitting indices is when you have mixed language content: the term "LA" in Spanish content adds very little information while it adds a reasonable amount of information in most English language documents (where is can mean Los Angeles). If you mix both content together, it can pollute term statistics for both. Considering how your segments of users and data will affect scoring is absolutely important as a general rule though, and part of why I'm super excited to be working on neutral retrieval now

Vectara is a retrieval augmented generation (RAG) as a service platform. We have a ton of IP already: an embedding model that outperforms Cohere and OpenAI, a reranking model that does similarly, a generative model that outperforms GPT 4 for RAG, a prompt engine that can provide citations and markdown/HTML links in RAG use cases, a leading hallucination detection model, well as our own internal high-performance vector/hybrid database. We just raised $25 million in series A and we're hiring a number of roles.

I personally lead product and before Vectara led product management teams at Kong and Elasticsearch at Elastic and am looking for a technically minded product manager with a few years experience in B2B. Location is pretty flexible for the role, but I'd prefer someone in the US to help increase our timezone coverage for US-based deals and working with our many US (mostly west coast) colleagues. I'm based in Berlin now after having been in SF for the past ~15 years.

All positions are at https://vectara.com/careers/#job-listings, but if you're a PM and interested in developer-focused GenAI, you can reach me at shane at [our domain, above]

Obviously everyone's mileage will vary, but I'm happy to see this treatment being more widely adopted

I always start editing in gdocs because it's so much easier to collaborate on than any blog platform, but then you always need to copy/paste the content once final into the blog and nearly every time, it copies some elements of formatting into the rich HTML editor I don't want (fonts, font sizes, etc) while I do want some things (headings, bold, italics). It's usually easy to import markdown to blogs or trivial to convert it to stripped-down HTML that can be imported. One of the teams I worked on built a simple gdoc script to do this

But even in 2013, there were managers in power positions that knew and were talking about how much the subscription models were going to hurt their base. But at the same time market dynamics being as they are, everyone also felt the only way to demand multiples for the stock price was to be seen as a software subscription company instead of a hardware company. Otherwise the revenue multiples on the stock were going to go down.

The stock market reaction leads the consumer market reaction, so executives at public companies are generally incentivized to "sell" a vision that sounds good even to investors even if they believe it will long-term hurt the business. That is, unless the executives plan to stay around a long time (another decade+). Modern companies don't reward that long term tenure so that's dissolved and now all these huge public companies have a slew of upper managers that expect to leave before the fires really take hold and make a pretty penny in the process

Some people are surprised by smaller models having the ability to outperform bigger models, but it's something we've been able to exploit: if you fine tune a small model for a specific task (e.g. reduced hallucinations on a summarization task) as Intel has done, you can achieve great performance economically.

For us, we treat hallucinations as the ability to accurately respond in an "open book" format for retrieval augmented generation (RAG) applications specifically. That is, given a set of information retrieved (X), does the LLM-produced summary:

1. Include any "real" information not contained in X? If "yes," it's a hallucination, even if that information is general knowledge. We see this as an important way to classify hallucinations in a RAG+summary context because enterprises have told us they don't want the LLMs "reading between the lines" to infer things. To pick an absurd/extreme case to show a point, the case of a genetic research firm, say, using CRISPR and finding they can create a purple zebra, if the retrieval system in the RAG bits says "zebras can be purple" due to their latest research, we don't want the LLM to override that knowledge with its knowledge that zebras are only ever black/white/brown. We'd treat that as a hallucination.

2. On the extreme opposite end, an easy way to avoid hallucinating would be for the LLM to say "I don't know" for everything thereby avoiding hallucinating by avoiding answering all questions. That has other obvious negative effects, so we also evaluate LLMs for their ability to answer.

We look at the factual consistency, answer rate, summary length, and some other metrics internally to focus prompt engineering, model selection, and model training: https://github.com/vectara/hallucination-leaderboard

People make mistakes: even the best people. We accept that risk as colleagues/employers/employees because there's a human connection, but that doesn't make it nonexistent. We're now trying to figure out how to deal with that when you're "employing" AI, and whether the risk tolerance is higher or lower alongside the reduced operational cost

In general, if you set your original pricing to yield an acceptable gross margins % and you cut your costs by 10x, you pass roughly that on to your customers if you're being smart

Consumption based pricing is becoming/already the norm for 2 reasons. As a SaaS operator, my storage/compute/etc costs tend to go up with your usage. If I can bill by the metrics that affect my cost, then:

1. It feels more fair to the buyer (pay for what you use) and

2. Investors and finance folks like it because I can build a pricing model with consistent gross margins

This whole thing is hinged on the multitenant SaaS model generally though, and it being so prominent. For example, if I ship self-managed software, I no longer genefally incur hardware usage costs (except my support costs probably go up roughly linearly to usage).

AI doesn't affect any of this reasoning.