dbroockman

Another one: don’t program your own AB testing framework! Every time I’ve seen engineers try to build this on their own, it fails an AA test (where both versions are the same so there should be no difference). Common reasons are overly complicated randomization schemes (keep it simple!) and differences in load times between test and control.

CA YIMBY has been making a lot of progress on this at the state level. If this stuff makes you mad it’s a good org to donate to: https://cayimby.org/

This is a rewriting of history. Before we ever had results from the COVID vaccine trials, the FDA declared that the primary goal of the trials was to prevent any symptomatic infection in the first place. From June 2020: https://www.fda.gov/media/139638/download.

And that’s 95.6% efficacy relative to two shots! Relative to no shots it must be like 97%. Wow.

This paper is intended as a sarcastic critique of the push towards conducting randomized trials in academia/medicine, but it’s an obvious straw man. You could write this sarcastic article for any research method - “see, why do research using method X when we already know the answer from other methods?”

The problem is that there are plenty of research questions where RCTs show us that previous non-RCTs were wrong.

I was similarly addicted and quit for similar reasons. https://www.weancaffeine.com worked really well for me - I bought 3 packs and was off caffeine in a few weeks. I was slightly slower those weeks but it wasn't so painful.

Holy shit, this quote is crazy:

> While the company was dodging me, it was also monitoring me. At my request, a number of police officers had run my photo through the Clearview app. They soon received phone calls from company representatives asking if they were talking to the media — a sign that Clearview has the ability and, in this case, the appetite to monitor whom law enforcement is searching for.

Here's a great paper about how rent control is counterproductive: https://web.stanford.edu/~diamondr/DMQ.pdf.

> Using a 1994 law change, we exploit quasi-experimental variation in the assignment of rent control in San Francisco to study its impacts on tenants and landlords. Leveraging new data tracking individuals’ migration, we find rent control limits renters’ mobility by 20% and lowers displacement from San Francisco. Landlords treated by rent control reduce rental housing supplies by 15% by selling to owner-occupants and redeveloping buildings. Thus, while rent control prevents displacement of incumbent renters in the short run, the lost rental housing supply likely drove up market rents in the long run, ultimately undermining the goals of the law.

It's hard to tell exactly what the underlying principle is from just the few questions we could get folks to answer, but the general pattern seems to be about like you describe -- on the environment, founders are pretty liberal; on issues of labor and product market regulations, they're fairly conservative; on everything else, they're fairly centrist. We have all the survey questions we asked that went into this index in the Appendix.

Great points on both. On #2, we wanted to include those, but space constraints :/. Given the interest in this survey we'll probably do another after collecting more suggestions like these. Thanks!

Author of the study they wrote about here. I think the NYT write up is better: https://www.nytimes.com/2017/09/06/technology/silicon-valley.... Study here: https://www.gsb.stanford.edu/faculty-research/working-papers....

The same logic holds for founders in other industries but they don't seem to like taxes/redistribution much (although we need and are getting more data on that). Would have been nice to ask about taxation on capital gains, but survey space was limited (you folks are busy!) and that is a relatively small share of federal receipts so not as important substantively, even if it speaks to this theoretical question.

Yep, that is exactly what the we find and argue. I think the NYT article about our work does a better job highlighting that central finding: https://www.nytimes.com/2017/09/06/technology/silicon-valley....

Author of the study they were writing about here. That is the original expectation I had coming in, but I don't think what we find is totally consistent with that (although some is). We talk about that in the paper. Study here: https://www.gsb.stanford.edu/faculty-research/working-papers....

Seems like disaster was only avoided because there was good visibility. If there was fog that day, it sounds like this wouldn't have been avoided.

These kinds of councils have nearly no influence; they're photo ops. But leaving does signal that Silicon Valley $ isn't going to play along with Trump, which Republican Members of Congress are going to care a lot about as they think about how to raise money in the future.

> This is really unfortunate. If you're in the "Trump is a mad man" camp, don't you want voices of reason like Kalanack's to be in his ear?

I see the point, but this business council isn't going to meaningfully influence policy. The policymaking apparatus is huge and this is a tiny, tiny piece of it. However, it does send a signal to Republicans in Congress that the tech $ they want will not play along with Trump. That's a quite important signal.

Fun fact: in Tokyo prices are 3x lower than in the bay area, in large part because new housing is legally much easier to build.

If you want to make the bay area more affordable, check out the great work these folks are doing to legalize building new housing: http://www.sfyimby.org.

Good thing voters are incentivized to vote correctly like they were incentivized to answer correctly in this survey. Oh, wait.

The city doesn't have enough room to add many more jobs. This is like saying a restaurant that fills its tables every night must be in a bubble because it hasn't grown past its capacity. Right? (Not saying it shouldn't have more room (building). But as a descriptive matter...)

It's standard practice for states to salt their voter lists with fake names. If, e.g., a non-political solicitation shows up to the fake name, the sender is caught. Same thing with FEC data -- politicians have to list their donors publicly, but it isn't legal for other politicians to solicit those people, and the FEC enforces it with salting.

What's funny is the best study would involve picking some people at random and giving them a basic income -- something YC could easily do. Theory can only get you so far on this one (although would be necessary to interpret the results of an empirical study).

Full transparency could have negative unintended consequences. For example, if surgeons knew their success rate were public, they would be incentivized to take easier cases. Who would take a difficult case if they knew it would constitute a bad mark on their record almost for sure?

Fundamentally, the issue is that it's impossible to observe for any given patient if that patient's outcome would have been better with a different surgeon. This is the same challenge we face with evaluating drugs: many more people who take aspirin survive than those who take anti-cancer drugs, but this likely reflects the kind of person who is taking each (people with headaches vs. people who have been diagnosed with cancer). To solve the problem there's no way around randomized trials. So, one idea would be to randomly assign patients to surgeons.

(Transparency might still be better on net, but important to keep these issues in mind.)

Here are some guesses.

On (1), the mental model I have is more like an understanding of who the super-intense supporters are. 90% of people might say they support background checks for gun purchases, but it's the 10% who support it who are voting on the basis of that issue, in part due to organizations like the NRA. I think our findings are consistent with that, in the sense that politicians seem to believe they don't need to learn what public opinion is in order to get re-elected; they can focus on other forms of information-gathering.

On (2), I suspect there is something there, but it's hard to make sense of Democrats from that angle, as Democratic primary voters, for example, are more liberal than the average person, yet Democratic politicians don't seem to see their districts as more liberal than they are. My guess is the key group is something more like "the people who choose to write in and go to meetings."

There's two assumptions many in the civic tech space seem to share that I don't think are so obvious (not that I think they are wrong, just that I think they're not obvious):

A1: The world would be better if politicians paid more attention to public opinion or knew more about it. I don't think this is so clear. Citizens support many policies not in their interest, don't know everything that experts do, etc. Most democracies reflect a tradeoff between popular control on the one hand and expert judgment / elite control on the other. I don't think either extreme is the best, and don't know where we are on that continuum relative to the ideal. But for what it's worth, my sense is that citizens actually understand this to some extent and ofter defer to legislators' judgments: http://stanford.edu/~dbroock/papers/broockman_butler_legisla...

A2: In order to increase citizen engagement, we should focus on lowering the cost of acquiring information about politics, the cost of participation, or the cost of providing information to representatives. I think this misses the real challenge: increasing people's motivation to participate and the benefits to participation. It's never been easier to participate in myriad ways -- the cost is epsilon. But if people see the benefits as zero, which many seem to, they still won't engage. I don't think the cost side of the equation is where the action is.

Their age doesn't predict their accuracy. With that said, I like the idea of asking how they think various age groups think. Maybe they think their district is not as young as it is, maybe they think younger voters are more similar to older voters than they are, or maybe both.

This is my personal favorite interpretation of our results -- they could know if they cared to know, and so the fact that they don't know suggests that they don't care. With that said, this paper is at odds with that: https://www3.nd.edu/~dnickers/files/papers/Butler_Nickerson....

Author here. Not sure why this is showing up today but happy to answer questions!

FWIW, we have data from 2014 across many more issues and the basic story is the same. Haven't finished writing that up.

None of these studies are randomized trials. All we've learned is that aggressive kids prefer violent video games to non-violent ones. It's useless.

The question is whether the lack of competition reflects a natural monopoly (i.e., something inherent to this market given spread out suburban geography), like for electricity service, or other forces. It's probably some of both. At the very least, increasing competition won't be easy in the medium term. So, I disagree with the notion that net neutrality is a trivial issue ("just a symptom").