malisper

Three consecutive months of decline starts to look more like a trend. Unless you think there's a transient issue causing the decline, something fundamental has changed

From the chart, the percentage of companies using AI has been going down over the past couple of months

That's a massive deal because the AI companies today are valued on the assumption that they'll 10x their revenue over the next couple of years. If their revenue growth starts to slow down, their valuations will change to reflect that

To be fair to OpenAI, their privacy policy[0] does provide some detail. They don't mention Mixpanel explicitly, but OpenAI does mention they share your information with third-party web analytics services:

> To assist us in meeting business operations needs and to perform certain services and functions, we may disclose Personal Data to vendors and service providers, including providers of ... web analytics services ...

OpenAI likely provides this disclosure to comply with US state privacy laws, but it's inaccurate to say they didn't disclose that they won't share your information

[0] https://openai.com/policies/privacy-policy/

Not exactly. Step E in the blog post:

> Gemini exfiltrates the data via the browser subagent: Gemini invokes a browser subagent per the prompt injection, instructing the subagent to open the dangerous URL that contains the user's credentials.

fulfills the requirements for being able to change external state

> Also, ARC AGI reported they've been unable to independently replicate OpenAI's claimed breakthrough score from December

Can you elaborate on this? Where did ARC AGI report that? From ARC AGI[0]:

> ARC Prize Foundation was invited by OpenAI to join their “12 Days Of OpenAI.” Here, we shared the results of their first o3 model, o3-preview, on ARC-AGI. It set a new high-water mark for test-time compute, applying near-max resources to the ARC-AGI benchmark.

> We announced that o3-preview (low compute) scored 76% on ARC-AGI-1 Semi Private Eval set and was eligible for our public leaderboard. When we lifted the compute limits, o3-preview (high compute) scored 88%. This was a clear demonstration of what the model could do with unrestricted test-time resources. Both scores were verified to be state of the art.

That makes it sound like ARC AGI were the ones running the original test with o3

What they say they haven't been able to reproduce is o3-preview's performance with the production versions of o3. They attribute this to the production versions being given less compute than the versions they ran in the test

[0] https://arcprize.org/blog/analyzing-o3-with-arc-agi

Other models aren't able to solve it so there's something else happening besides it being in the training data. You can also vary the problem and give it a number like 85 instead of 65 and Gemini is still able to properly reason through the problem

if the three numbers are a, b, and c, then either a+b=c, a+c=b, or b+c=a

I've been using a math puzzle as a way to benchmark the different models. The math puzzle took me ~3 days to solve with a computer. A math major I know took about a day to solve it by hand.

Gemini 2.5 is the first model I tested that was able to solve it and it one-shotted it. I think it's not an exaggeration to say LLMs are now better than 95+% of the population at mathematical reasoning.

For those curious the riddle is: There's three people in a circle. Each person has a positive integer floating above their heads, such that each person can see the other two numbers but not his own. The sum of two of the numbers is equal to the third. The first person is asked for his number, and he says that he doesn't know. The second person is asked for his number, and he says that he doesn't know. The third person is asked for his number, and he says that he doesn't know. Then, the first person is asked for his number again, and he says: 65. What is the product of the three numbers?

Because Google's revenue is $2M/employee while a startup's will be $0/employee

That's already priced in

The 50k/yr value of the 1% equity already prices in future dilution. Just because the equity is going to be less than 1% in the future doesn't mean it's not worth $50k/yr today

> As a founding engineer, I do almost the same amount of work as the founder (e.g. 90%), and get only 5% or less of the reward.

If you believe you're doing 90% the work of a founder and getting paid 5%, then you should be an actual founder and get paid 20x as much as you be as a founding engineer

If you do the Math, 1% seems fair. The typical YC company raises a seed round at a valuation of around $20M. 1% of that with standard vesting terms equates to $50k/yr.

If the typical founding engineer equity was 5%, that would equate to $250k/yr which would mean most startups would have greater total comp than Google.

I need to do some Math, but I wonder if there's a better strategy than Kelly betting. An assumption made for Kelly betting is the bets are independent of each other. That's not the case in the problem given.

After making a bet, you gain information about the contents of the rest of the deck of cards. I could see it being possible to do better by pricing in that information into your bet.

It's interesting to note none of the email tools I'm most familiar with are mentioned by the author. It's clear the author is a different demographic from me given they said they want to stay under $200/mo. Some of the tools I hear companies use the most are:

  - Customer.io
  - Iterable
  - Braze
  - Marketo
  - Salesforce Marketing Cloud

My understanding is Customer.io is what most startups use these days with larger companies using one of the other four.

This is pretty neat! I was toying around with the problem and it appears you can use generating functions to derive the same sequence of operations. If you start with:

  G(x) = 1 + x + x^2 + ... = 1/(1-x)

The coefficients of this polynomial is the sequence (0^0, 1^0, 2^0, ...)

If you take the derivative of G(x) and multiply by x you get:

  x * G'(x) = x + 2*x^2 + 3*x^3 + ... = x * d/dx 1/(1-x) = x/(1-x)^2

The coefficients of this polynomial is the sequence (0^1, 1^1, 2^1, ...). If you repeat this step, you get a polynomial whose coefficients are (0^2, 1^2, 2^2, ...) and if you do this operation N times, you can get a closed form of a polynomial whose coefficients are (0^N, 1^N, 2^N, ...).

The infinite sum converges for -1 < x < 1. If you set x=1/c, you get the infinite sum

  0^N/c^0 + 1^N/c^1 + 2^N/c^2 + ...

which is exactly the sum we are trying to solve for. This means you solve any infinite sum of the form given by taking the derivative of 1/(1-x) N times while multiplying by x each time. Then plug in x=1/c at the end.

This is accounted for because the second equation looks at sequences of two moves and not just one. After a sequence of two moves there are two possibilities. Either the state has changed or it has not. This is reflected in the left and right side of the second equation.

There is a simpler way to handle loops. If you end up repeating a position, you know the turn player will repeat the same move as before. By iterating over every possible move and every possible response, you can calculate the expected value of a position. Writing out the equations where V(s, i, j) is the expected value of the position when turn player will attempt move i and the opposing player will attempt move j:

  V(s) = max i (min j V(s, i, j))
  V(s, i, j) = (probability move i or move j changes the state) * V(new state) + (probability state doesn't change) * V(s, i, j)

You can solve the second equation for all i and j and then use that to solve the first equation.

Didn't expect something relevant to what I do to show up on HN. This has been a hot button issue for the last two years ever since the Markup did a report on hospitals sharing data with Facebook[0]. Since then the government has explicitly called this out as a problem[1] and a number of hospitals and related companies are dealing with fines and lawsuits[2][3][4]

Given the government is actually enforcing the law for once, this is one of the few times I've seen people take regulation like this seriously.

[0] https://themarkup.org/pixel-hunt/2022/06/16/facebook-is-rece...

[1] https://www.hhs.gov/hipaa/for-professionals/privacy/guidance...

[2] https://www.ftc.gov/news-events/news/press-releases/2023/07/...

[3] https://www.ftc.gov/news-events/news/press-releases/2023/02/...

[4] https://finance.yahoo.com/news/costco-sued-accused-sharing-c...

Note that website information can still be PHI.

From[0]:

> if an individual were looking at a hospital’s webpage listing its oncology services to seek a second opinion on treatment options for their brain tumor, the collection and transmission of the individual’s IP address, geographic location, or other identifying information showing their visit to that webpage is a disclosure of PHI to the extent that the information is both identifiable and related to the individual’s health or future health care

[0] https://www.hhs.gov/hipaa/for-professionals/privacy/guidance...

Website visitor information is still really sensitive. If you book an HIV test online, you probably don't want Google and Facebook to know that.

I used to work at Heap, although I left 4 years ago

> Why use Postgres distributed cluster vs say an incremental store that supports real time data like Materialize

Materialize didn't exist when Heap was founded 10 years ago. Also, Materialize is dependent on knowing what queries you are running up front. Not to mention Heap is dealing with petabytes of data. Materialize only recently introduced multi-node support, so I would be surprised if it's being used at that kind of scale.

> Why use typescript at all?

Heap was originally written in CoffeeScript. It was the decision the semi-technical CEO made. Migrating to Typescript was the best option that allowed Heap to keep their existing codebase.

> Regarding audit tables, are you also keeping audit tables for user and events tables too?

No. Only the distributed metadata had audit logging when I was there

> Doesn’t the database come with audit tables baked into it?

No

> This is a nitpick, but their "Claude 2 on 200k Context Data" graph doesn't actually extend to 200k, only 100k

It does extend to 200k. The chart is logarithmic. You can see the little 2 in the bottom right.

> Kinda astonishing how some startup with almost no revenue that burned through hundreds millions of dollars is still able to acquire anything x)

As of December, Docker was doing $100M+ of ARR[0]. No where close to no revenue

[0] https://www.linkedin.com/posts/asethi_docker-the-phoenix-sag...

> Even research from OpenAI has attempted to use GPT-4 as quasi-ground truth (as a replacement for human evaluators).

The way OpenAI used GPT-4 is fundamentally different than how GPT-4 was used to score the answers to the MIT exam. In OpenAI's case, they had GPT-4 generate an explanation of when a neuron in GPT-3 would fire. They then gave that explanation back to GPT-4 and had GPT-4 predict when the specific neuron in GPT-3 would fire. The scoring was done by computing the correlation between when GPT-4 predicted the neuron would fire and when it actually fired. The scoring was not done by GPT-4 as was done for the MIT exam

In addition OpenAI did have human evaluators score the explanations as well to make sure they were human interpretable[0]

[0] https://openaipublic.blob.core.windows.net/neuron-explainer/...

> 27 bits for just one word seems wasteful

Where did you get that you need 27 bits for one word?

> Then to send any word you only need to send one number, and in binary it would have between 1 and at most 19 bits

Yep! By sorting by frequency, you are able to make it so the majority of words have shorter bit strings. By my calculations, common words such as "the", "of", and "and" will have ~4-6 bits associated with them. That means you can encode a large number of words (googling says those words make up ~1/7 of words based on frequency) with only 4-6 bits each. That's far from the 27 bits you calculated

Ambiguity of a context-free Grammer is undecidable means there are some context free grammers where it is impossible to determine if they are ambiguous or not.

There are context free grammers that we can prove are unambiguous. As an example "A = xAy | ε" is unambiguous. Lojban is one of the examples of grammers we are able to prove is unambiguous.

> If pulling a third party resource is not compliant then loading from any cdn is non compliant

Loading data from a third party can be GDPR compliant, but isn't always. One legal basis for processing personal data is "legitimate interest"[0]. Legitimate interest is incredibly vague. In short, it allows you to process data as long as doing so is necessary or of critical important to your business.

As an example, in order for someone to visit your website, you need to receive and process their IP address. That's just how TCP works. Since you have a "legitimate interest" to process their IP address so they can visit your site, you don't need to ask for consent before processing their IP. Similarly, since DDOS prevention is critical for maintaining your website, you are allowed to process IP address for DDOS prevention as long as you intend to process the IP only for DDOS prevention.

For your specific question, a website loading an external font resource would likely fall under legitimate interest since the font is necessary for the website to function.

Since user analytics is not necessary or critical to a business, you cannot share IP address with a third party if the intent of doing so is so you can perform analytics on your users.

[0]: https://gdpr-info.eu/art-6-gdpr/

Note that despite the claims of GDPR compliance, Beam is likely _not_ GDPR compliant for a few different reasons. Going off of this doc[0], a few things stand out.

First, there's a Javascript snippet you add to your site to set up Beam. That Javascript snippet loads additional Javascript from beamanalytics.b-cdn.net. If you add the Beam provided Javascript to your site, every time a user visits your site, their IP address will be shared with beamanalytics.b-cdn.net. If the user didn't consent to sharing their IP address with beamanalytics.b-cdn.net, you do not have a lawful basis[1] for sharing the user's IP with beamanalytics.b-cdn.net.

Second, there's this notion that because Beam hashes IP address that "anonymizes" the data[0][2]. According to GDPR, this is actually "pseudonymisation"[3]. If you know what hash function is used, you can still tie back the hashed data to the original user. Pseudonymized data still meets the GDPR definition of personal data[3] so applying this hash doesn't actually do anything in terms of helping with GDPR compliance.

  [0]: https://beamanalytics.io/data
  [1]: https://gdpr-info.eu/art-6-gdpr/
  [2]: https://news.ycombinator.com/item?id=35539476#35546091
  [3]: https://gdpr-info.eu/art-4-gdpr/

GPT-4 didn't say there's a 1 out of 5 chance of a bank run happening. On a 5 point scale from very low likelihood (1) to very high likelihood (5) GPT-4 gave a 1. GPT-4 gave the lowest score possible for how likely a bank run was to occur.

Of the six options GPT-4 evaluated, GPT-4 gave the lowest risk assessment code (combination of the probability of the event occurring and how bad it would be if it were to occur) to a bank run.