HNHacker News
TopNewBestAskShowJobs

scottrogowski

755 karma · joined October 19, 2012

scottrogowski.com
submissionscomments
scottrogowski··on California population declines for first time in more than a century
I agree so strongly with this sentiment. There is always such a defeatist attitude on these threads that assumes California is somehow full and unchangeable. Outside of the three metro areas, it's basically empty! Separately, in the metro areas, there's a lot of room for infill density.

Three years back, I made the rational choice to leave because I could barely see a way to home ownership - not to mention having a family and sending children to a good school.

California can solve all of it's problems if it wanted to. I think the problem is the number of people in California who have barely left the state and view the rest of the country as this underdeveloped backwater.

Food for thought, California currently:

- Is tied for the 3rd highest unemployment rate https://www.bls.gov/web/laus/laumstrk.htm

- Has the 5th highest Gini coefficient https://en.m.wikipedia.org/wiki/List_of_U.S._states_by_Gini_...

- Has the LOWEST high school graduation rate https://worldpopulationreview.com/state-rankings/high-school...

The first step to solving a problem is admitting you have a problem.

scottrogowski··on On incels, dead bedrooms and the hard problems of loneliness
I have a pet theory for this phenomenon. If you look at recently contacted peoples or those who have little contact with the outside world, endemic warfare is common. In these societies, well over 20% of men are killed in the ongoing conflicts. Evidence points to endemic warfare being common for most of human history and prehistory. Looking at our closest evolutionary cousins, the chimpanzees, it is the same if not more brutal.

I think, on average, that men are more polyamorous than women partially because this fit our environment. It allowed women in a clan to be in child-bearing partnerships even if gender ratios were grossly imbalanced. While we are now in a very different environment where almost all men survive, our relative sexual and coupling urges have remained the same.

Through most of the modern-era, post feudalism, this was solved through socially enforced monogamy. Now of course, with loosening restrictions, polyamory is gaining an acceptance it probably hasn't experienced since earlier eras of endemic warfare.

To be clear, the lack of violence in modern life is a strictly _good_ thing. But I'm not sure where it leaves us for the future.

scottrogowski··on Ask HN: Mobile phone addiction help?
Could you expand on that? I'm not familiar with the world of rooting android. Does something exist that would allow you to specify exactly which apps are installed (and remove the web browser and play store)?
scottrogowski··on Ask HN: Mobile phone addiction help?
I'm with OP in their diagnosis of the problem. Other commenters seem to focus on "tricks". The problem with these tricks is that they can all be bypassed and you always go back to normal. Would you suggest someone who wants to quit smoking just keep cigarettes in their pocket and just snap a rubber and on their wrist when they want one? Of course not, that is just asking to relapse. Self-control is a finite resource. Eventually, you stop using the rubber band and just continue smoking as before.

I've long thought that what we need is a forked Android distribution without a web browser and with a whitelist of apps that can be installed. Something of an "addiction-free phone". I'd be curious to hear if anyone is working on anything like this. It's certainly something I would want. To be honest, I'd pay a lot for it.

scottrogowski··on Why are there so many Canadians in India?
This is almost certainly wrong but is worth a mention given that the author can't figure it out. In Southern India, there is a language known as Kannada. It has 43 million speakers, which, fun trivia, is more than the population of Canada (38 million). While "Kannada" and "Canada" are pronounced differently, I wonder whether there could be some clicks from Kannada speakers who think they are selecting the local English dialect. The English 'k' and 'c' sounds are so close together that perhaps Kannada speakers with weak English are trying to sound it out? Would be curious to hear someone from India chime in.
scottrogowski··on Cryptocurrency Mines Consume More Power Than Argentina–But PSUs Can Help
The only reason why Bitcoin energy usage is so high is because the difficulty is automatically adjusted so that the average time between blocks is approximately 10 minutes. Any attempts to improve the energy efficiency, rather than reduce total power consumption, will likely just make Bitcoin mining cheaper and bring more miners online.

The only practical thing that would reduce the environmental impact of Bitcoin would be a drop in the price of bitcoin. I keep thinking this thing is on its last legs but it keeps going like a bad dream driven only by speculative frenzy.

For the record, I'm not against the concept of a decentralized cryptocurrency but most popular cryptocurrencies are very badly suited to actually being currencies and in the meantime just serve to warm the planet.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Yup. Considered it. I initially settled on no because it might compromise the design goal of 1:1 MongoDB/PyMongo compatibility. Now the idea is being revived as an option alongside the two existing backends. I am concerned there might be an issue with some square-peg high level MongoDB operations simply not fitting into the SQLite round-hole but it's certainly something that's being investigated.
scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
O(n) cache size is fine (and IMHO, preferable) for small datasets. For large datasets, you're correct. Cache eviction is important when memory usage gets too high. I didn't have it explicitly on the README as something that needs to get done but now it is. So thank you for pointing it out.
scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
You're right. Fixed. SQLite is more-so the winner but Mongita appears to squeak ahead in id lookups https://github.com/scottrogowski/mongita/blob/master/assets/... (to be fair, it might be the thin translation layer I built). Regular MongoDB struggles a lot and I don't think I'm being unfair to it in any way afaik.

Thank you for pointing that all out.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Thanks for the kind words :)

I like your idea. As I understand it, since SQLite performs better on most benchmarks anyway, why not have an engine subclass that utilizes SQLite itself?

This would be a little trickier than your envisioning. The engine class is the lowest layer but doesn't handle the slow bits like the finds, managing the indicies, etc. So to really get the benefit of SQLite you would have to pull those slow bits in.

I do really like the idea of offering a third option so that you have - Memory (fastest) - SQLite (almost as fast but compromises on perfect MongoDB reproduction) - Disk (slower but is faithful to MongoDB)

Happy to discuss it more if you want to email me.

> How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see)

I used type annotations for a while when they came out but didn't find them to offer advantages over docstrings. The world might have moved over to type annotations though and I've not been aware.

> PyPy? I wonder if you'd get a ~free speedup

I'm getting a lot of suspicion right now from other commenters who are convinced I have my thumb on the benchmarks and I don't want to give them any more excuses :tears-of-joy:. But joking aside, you're probably right and it's not something I had considered.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases.

This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Hey, your first statement is true but your second is not.

The disk engine does store data in memory which is part of the design. You wouldn't want to use a database that doesn't utilize caching.

The last benchmark panel, https://raw.githubusercontent.com/scottrogowski/mongita/mast..., shows cold starts where I test it without cache. So in that, it does hit the disk.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
This is good and helpful feedback. Thank you. I'm humbled by how well this has done so far but definitely feeling the criticism. A title like, "Mongita is embedded MongoDB for Python", while being a statement without ambiguity, doesn't immediately convey the vision for why this might be useful to people. SQLite by contrast is a nice rhetorical anchor that people understand. I'll have to think through whether there is a better way to convey it that fits that goal.
scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
It's a good question and to be accurate, depending on the benchmark, Mongita is about the same speed at SQLite to several-times slower.

There is less happening algorithmically than you would think. Where the tricky slow bits do exist, they have largely fallen into the happy-path of fast data structures in the Python language/stdlib. I also use sortedcontainers for indexes which helped quite a bit (http://www.grantjenks.com/docs/sortedcontainers/).

If you're curious, the benchmark code is in the repo: https://github.com/scottrogowski/mongita/blob/master/benchma...

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Author here. My goal with that one-liner was to convey immediately what it is - an embedded database that implements the PyMongo/MongoDB API.

Is it semantically perfect? Probably not. Will Mongita be useful to Python developers who are accustomed to saving their data in idiosyncratic JSON files because they don't want the overhead of a MongoDB server? Maybe! I hope so.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Thank you! I actually did consider doing exactly what you said. Early on, I decided one my goals for the project would be to make it easy to swap back and forth between Mongita and PyMongo/MongoDB. For me, that meant getting as close as possible to their implementation and using things like BSON, ObjectIds, etc. For that, I sacrificed some performance but I think most people who would use something like Mongita would prefer a faithful reproduction over a few clock cycles.

I could be wrong on that though.

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
Author here. That's exactly it.

I think it would be very difficult to completely beat SQLite with a Python library. My goal wasn't to beat it but to have performance that's within an order of magnitude which I think I've achieved.

In my opinion, the MongoDB interface has a lot of advantages over SQL that make sense in a lot of use cases. Certainly, there are times when a traditional relational database is the right choice. But I do think Mongita fills a niche.

Separately, the JSON1 extension, which the top-level comment refers to, is nice technically but has a challenging interface IMHO https://www.sqlite.org/json1.html

scottrogowski··on Show HN: Mongita is to MongoDB as SQLite is to SQL
That's a good question. I'll try adding it to the benchmarks.
scottrogowski··on SpaceX wins contract to develop spacecraft to land astronauts on the moon
I have a genuine curiousity (because I can't seem to find it in the news reports). The contract is going to the starship design. What exactly are they planning to take that requires such a large ship? The lunar landers were tiny because the Moon has such a small gravity well. For that payload capacity, I assume you would have most or all of a habitable lunar base.

Edit: There is this: https://blogs.nasa.gov/artemis/2020/10/28/lunar-living-nasas.... I wonder whether NASA will now try to include it on one of the two missions.

scottrogowski··on Show HN: I made a 5-key keyboard
Very cool. Reminds me of an excellent radiolab I listened to about Wubi: https://www.wnycstudios.org/podcasts/radiolab/articles/wubi-...

If you can effectively type 70,000 characters with 26 keys, certainly you can type 26 characters with 5 keys. It would be even easier if you made it two handed.

(In other words, I think this could someday be a practical text input method)

scottrogowski··on Ask HN: For those who use Chrome, why do you?
I try to use Firefox (I do) but there are still issues every now and then and I find that I have to use Chrome for something at least once a day.

Most developers develop for Chrome and don't even consider Firefox.

I'm not particularly fond of Google (and growing less fond by-the-week) but the best I can do for now is use Firefox for 95% of things and keep Chrome around for the rest.

scottrogowski··on The Myth of the 'Waterfall' SDLC (2019)
The difference between what mainstream agile is and what the original authors of the agile manifesto were trying to achieve is vast.

This talk from one of the authors is an amazing rebuttal of what agile has become and clarification of what it is supposed to be https://m.youtube.com/watch?v=a-BOSpxYJ9M

scottrogowski··on A third of Covid survivors suffer neurological or mental disorders: study
While I agree that it was an offhand comment, if you look closely at the chart you posted, the weakest correlation is on the bottom right under "mood, anxiety, and psychotic disorders" which is what I believe OP was implying. Under that category, you only see a small difference between the COVID-19 and "other RTI" populations.
scottrogowski··on Israel may have achieved herd immunity against Covid-19
You are failing to take into account the hygiene hypothesis: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2841828/. There is evidence that a lack of exposure to germs is a cause of autoimmune and allergic diseases. While this may-or-may-not be true of exposure to COVID amongst healthy young people, blanket statements like "It’s a disease. It’s not something your body needs to experience" could use a little more nuance.
scottrogowski··on Benzene detected in hand sanitizers
I agree with most of the other replies to your comment. The world is full of counter-intuitives and I'm genuinely curious about whether this is one.

As I said, my guess is that the risk of surface-transmission of COVID is probably still greater than the risk of increased exposure to benzene. But I think given the low risk of surface transmission, it might still be worth a comparison.

scottrogowski··on Benzene detected in hand sanitizers
I'd be curious to see whether we actually see an uptick in cancers in the coming months/years. If so, I wonder whether it will turn out that using hand sanitizer resulted in a net increase or decrease in overall mortality.

Evidence that Covid spreads through surfaces is sparse and that route of transmission is no longer emphasized by the CDC https://www.nature.com/articles/d41586-021-00251-4.

While the IFR of COVID varies, it appears to be below 1% for most regions of the world, https://www.medrxiv.org/content/10.1101/2020.05.13.20101253v.... Could anyone who is more informed than me estimate what the increased mortality would be for those who religiously use hand sanitizer with this level of benzene? My guess is still lower than COVID but I have no reference point.

scottrogowski··on What’s up with these new not-open source licenses?
> you can’t harm competing services providers without harming end users.

> the overt motive is to enhance monetization by preventing licensees from competing with them to sell services: it is to create a moat enabling monopolization and monopoly rents.

Here's the problem: Amazon, Microsoft, and Google are already able to charge something approaching monopoly rents because they have built a moat that others can't dream to cross. Heck, try to even build two services that talk to each other between the moats and you will get hit with massive data-egress charges. And they do this by monetizing free products with no compensation to the developer. If you are Redis, your hosted product, whose core you developed, is less competitive than that of the cloud providers who didn't contribute anything to the project. At the same time, the parent tech behemoths are engaged in all sorts of other bad behavior that only massive tech companies can get away with.

This state of affairs seriously threatens the open-source ecosystem and the source-available licenses are targeted to righting this imbalance. Again, are they perfect? No. If you are making money selling the software without modification, is it reasonable to ask for royalties to go to the developers? I think, on balance, that is reasonable.

If not source-available licensing, what is the better solution to the problem?

scottrogowski··on What’s up with these new not-open source licenses?
https://commonsclause.com/

Second FAQ: "Is this “Open Source”?" Answer: No.

I don't think they are trying to hide anything and if anything are being generous because the source-available licenses do have most features of the open-source model with the narrow SaaS restriction.

scottrogowski··on What’s up with these new not-open source licenses?
This article takes a purist stance driven by a rigid adherence to ideology. Let's look at this another way...

Open-source and proprietary licenses are at two ends of the software development spectrum. The open-source model maximizes ease-of-adoption but doesn't provide much incentive for the developers. Proprietary software provides a lot of incentive but adoption can be slow and burdensome.

Let's assume that a good goal for society is to maximize the rate of innovation in software. To do that, you need a mix of BOTH ease of adoption and suitable development incentives. Source-available licenses are an attempt to accomplish this.

Is this a perfect solution? Probably not. I think better licensing models are still waiting to be discovered.

However, my sense is that these new licenses will accelerate the development of software with limited downside for the user. After all, they are designed only to impact companies attempting to sell a SaaS.

In addition, they have the potential to weaken the tech monopolies which, in my mind, is a Very Good Thing.

scottrogowski··on Satellite imagery shows Northern California kelp forests have collapsed
My point was less on the benefits/drawbacks of individual decisions and more on the presence of democracy and therefore the need to convince individuals of public policy to make it happen.

For the record, I think of democracy as a strictly positive thing even when it seems like democratic societies have erred.

← PreviousPage 2 of 4Next →