bshipp

I saw his talk live in 2014 and the dude is amazing. I loved his summary of building Python libraries from the ground up during legal discovery because he discovered a hidden Python installation on the terminal his opponents gave him that allowed him to parse thousands of documents very quickly.

https://youtube.com/watch?v=RZ4Sn-Y7AP8

Different strokes for different folks. I thought Portainer and Netdata were overkill the first time I spun them up and now I can't live without them.

I'm pretty sure the main reason for throwing out old tools that still function in favor of the new hotness is developer boredom.

That's an excellent point. I think one complex agriculture issue I've encountered that LP would struggle to handle is something like inventory control of perishable goods, where there's variance in incoming quality, degradation during storage, and variance in shelf-life based on age of crop, growing conditions, etc. LP would work great for handling grain storage, where a crop generally maintains a known quality profile over time with limited shrinkage, but I could see a use for a machine learning algorithm that handles perishable goods stochastic variables in a more direct way.

Fair enough. I'm not invested in LP or RL, but my background is in agriculture and LP forms the basis of a lot of cropping/feeding decision trees. I'd be curious to explore new techniques, but it sounds like the discussion here is for a different set of linear programming problems than the ones I see most often, and that there's likely a limited upside toward implementing RL.

I also am curious about this statement. If a problem becomes too complex for linear programming solutions, how easy is it to know that a reinforcement learning solution is actually a global optima and not just local?

If the green squares were sufficient to give me a dopamine hit, my half-abandoned repos with stars, pull requests, and conversations would be a much higher priority in my life.

As it stands, responding to user requests on Github just feels exactly like work.

That makes perfect sense. I'm no lawyer and am only discussing this out of idle curiosity, so I'm sure my comments were a naive take. The situation I'm envisioning is an audio recording from a 2-party consent state outlining the existence of evidence. Since that cannot be the legal basis of a warrant, an officer might wander through the neighborhood talking with associated citizens to try and independently establish the need for a warrant based on various witness statements. No evidence has been falsified or created out of thin air, and there's no hope of a conviction if the suspected building doesn't contain damning evidence.

However, I fully recognize how such a process might be abused and the need for a very firm legal hand to avoid accidentally becoming an authoritarian state.

As long as the material presented in court is acquired via legal methods, no laws are being broken, are they? Now, if officers use legal loopholes to take illegally-acquired evidence and somehow make it legal, that's a practice that should be rooted out and those involved prosecuted. But as long as officers can construct a legal case completely independent of the poisonous tree, what's the issue?

I'm guessing the evidence of actual harm from aspartame consumption is also orders of magnitude lower than the damage from sugar-induced diabetes.

I hadn't heard that, but a cursory Google search indicates that Kodiak is building one of these systems.

https://kodiak.ai/

Given the significant weight and danger of death for other drivers, it'll be years until legislators allow their "safety drivers" to be eliminated from the equation. This makes the AI system more akin to enhanced cruise control than robotic trucking.

I'm debating about renewing my truck license for this reason. If it's taking this long for automobiles to be approved and accepted, it'll be 30 years before trucking is automated.

The two decades during which human oversight of automated systems will be mandatory would be long enough for me to finish off my career getting paid to drive while I sit in a cab writing code, periodically checking over the status of my lead truck and the two or three slaved trucks following me.

I don't even know if you'd need two servers. Spin up a few Docker containers on the 512gb raid array and you could serve the forum and mods on the same machine. It's not like OpenTTD has the user demands of Reddit or Huggingface.

I use similar analogies to describe AI to friends and family. "When AI attempts to solve a maze, it just smashes into every wall over and over until it inevitably finds its way to the exit."

If it wasn't for high-powered graphics cards AI would still be a niche university research subject.

Putin has been in power for 20 years and the invasions of Georgia and Crimea gave all evidence necessary that he had imperialistic goals.

Considering the pride Russians proclaim regarding their role in defeating a WWII dictator, it's ironic they refuse to lift a finger to handle the one they raised and enabled at home.

Unfortunately for Russians, the fact they allowed their country to invade a peaceful neighbor in an unprovoked war of aggression and then repeatedly lied about almost every facet of that invasion means that the rest of the world will likely not give them the benefit of the doubt for another 50 years, even if they magically start telling the truth right now.

I think the point is that most of those requirements could be met with a $60/month hetzner server containing a 512 GB raid array. I think that comes with 100GB free backup and if you pay another 10 bucks or so you can upgrade to a terabyte. But it's been a while since I checked out those costs.

Having played (and lost countless hours) Transport Tycoon Deluxe in the 90s and then OpenTTD 10 or 15 years ago, I'm just tickled that this game adaptation is still going strong.

Chris Sawyer's ability to create addictive building games that remain fun to play long after their contemporaries have ended up in the dustbin of history is superhuman, in my humble opinion. Add to the fact that he did it all in Assembly, and it's hard not to place his achievements on a bit of a pedestal.

I think this comment highlights the two separate discussions going on in this thread. If you're building a customer-facing search engine, avoid reinventing the wheel by leveraging powerful tools like elastisearch.

On the flip side, if you're a data analyst or developer who has a large database with one or more text columns they want results from in a more flexible way than using "LIKE/ILIKE" SQL queries, it's probably easier and faster to create an FTS index/table in that database to get them 90% of the way there.

You're absolutely correct. Caching common search queries allows the site to allocate hardware for processing "expensive" queries, with the objective of having both complete at near the same time.

Without caching, the cost of operating the site would dramatically escalate.

I'm not trying to be argumentative. As long as people find a solution they're happy with, I think that's great. For me, I'm far less interested in handling typos, but I can see how it would be valuable in many applications. I'm usually less interested in tying in and learning another set of services if I can get 90% of the way there with one, but leaving the option of adding it later if additional requirements make it necessary.

If you're looking for a turn-key solution, I'd have to dig a little. I generally write a scraper in python that dumps into a database or flat file (depending on number of records I'm hunting).

Scraping is a separate subject, but once you write one you can generally reuse relevant portions for many others. If you can get adept at a scraping framework like Scrapy you can do it fairly quickly, but there aren't many tools that work out of the box for every site you'll encounter.

Once you've written the spider, it's generally able to be rerun for updates unless the site code is dramatically altered. It really comes down to how brittle the spider is coded (i.e. hunting for specific heading sizes or fonts or something) instead of grabbing the underlying JSON/XHR that doesn't usually change frequently.

1. https://scrapy.org

It depends on what the user requirements are. FTS works pretty well with both Postgres and SQLite, in my experience.

Here's a git repo someone can modify to do a cross comparison on a specific dataset, if they are interested. It doesn't seem to indicate the RMDBs are outclassed in a small-scale FTS implementation.

https://github.com/VADOSWARE/fts-benchmark

For sure, but the approach is quite viable. If 19 out of 20 searches by a user are almost instantaneous and single novel one requires a few seconds, they'll assume a hiccup in their internet connection and still view the site as "really fast". It's certainly useful for limiting demands on expensive hardware.

For such a light demand and fixed site requirements, a single-file sqlite dB is probably best. Modern Sqlite has full-text capabilities that are quite powerful and relatively easy to implement.

https://www.sqlite.org/fts5.html

Using measured comparisons and making informed choices is certainly the best way.

If you can start with Postgres to have a relational database with the benefit of Full Text Search (i.e. avoid Elastisearch) as well as JSON fields (i.e. avoid MongoDB) then you end up simplifying initial hardware/software requirements while retaining the ability to migrate to those solutions when user demand requires it.

So many developers seem to build with the idea that they'll become the next FAANG when actual (or reasonably forecasted) user load doesn't remotely require such a complex software stack.

Sqlite is extremely powerful for any locally-embedded databases, such as apps with an offline component. For example, searching text in notes, books, cached news, etc. It's very simple to implement inside Sqlite with much less overhead than including a Postgres/Mysql instance.

I've had good luck using FTS5 in Sqlite for small-scale use scenarios, such as examining scraped data, etc. It's remarkably powerful and quick.

https://www.sqlitetutorial.net/sqlite-full-text-search/

https://www.sqlite.org/fts5.html

I usually don't have an issue installing Linux on a computer. The issue generally pops up when the recipient wants a piece of software that is only available on windows (which is also true for people migration to Macs). It's one thing to learn a new OS, but having to learn new software generally becomes the sticking point.

Agreed, and I'm far more concerned about the impact of the walled gardens between mobile app companies than the old desktop wars. At the moment, this feels like the nostalgia of a comfortable old debate between two elderly men sitting on a park bench.