656 karma · joined April 22, 2013
You mentioned that implementing an upgrade is difficult because of the actual code change but also the testing and identifying what the changes are in the first place. We're starting trying to do a really good job of the last one - what are the changes in this next version, are they breaking, do they break for your codebase, and what should you do if they are? We want to move from there to automating as many of the code changes as we can, but my sense is the biggest value is in giving you the confidence that we at least captured all of the changes you need to be concerned about.
We have our own database of every version of every rubygems package alongside its runtime dependencies (like you see at https://rubygems.org/gems/pundit).
Then we parse your Gemfile and Gemfile.lock. We use the Gemfile to figure out gem group and pinned requirements (we run turn your Gemfile into a ruby AST since Gemfiles can be arbitrary ruby code; we use bundler's APIs to parse your Gemfile.lock). This gives us all of the dependencies your rely on.
Then we let you choose one or more package that you want to upgrade and the version you want to target (let's say Rails 7.0.4.3).
Now we have [your dependencies and their current versions], [target rails version], [all of the runtime dependency constraints of these gems]. We run this through a dependency resolution algorithm (pubgrub). If it resolves then you're good to upgrade to that version of Rails without changing anything.
If this fails to resolve, it's because one or more of your current dependencies has a runtime restriction on rails (or another indirect gem being pulled in by the new rails version). This is where the optimization part comes in. The problem becomes "what is the optimal set of versions of all your dependencies that would resolve with the next version of Rails". Currently we solve for this set trying to optimize for the fewest upgrades. As our dataset of breaking changes gets better we'll change that to optimizing for the "lowest effort".
Happy to elaborate.
We have a command line tool you can use to send your lockfile to our server so we can process it, but you have to use our web app to view the results.
> My biggest pain isn't with the plan. It's with the actual upgrade process. Things _always_ break. Every single time we do a major upgrade, there's this long tail of things we need to fix. Even worse is the undocumented changes that break. This gives me nightmares.
Planning is one way to make those major upgrades go wrong less often. Our idea is that we can break large upgrades down into incremental changes that are individually safe and mixed in to your regular dev cycles, so that when you go to do the large upgrade you're not making such a big change all at once. This means things like fixing every deprecation ahead of time and making sure all the other dependencies you use are compatible with the version you're upgrading to.
We're calling this "upgrade paths" in our tool. Something like a major rails version upgrade is terrifying because of all the moving parts. Often you may have a dozen other gems that need to be upgraded in order to upgrade Rails. You said "If I can incrementally upgrade one package on every PR until I'm up to date, that'd be amazing.". We want to make that the flow for major framework upgrades, so you're merging in incremental improvements in the blocking dependencies ahead of time.
Undocumented breakages are for sure on our roadmap. We have two types in mind today: - Changes that are missing from the changelog - Incompatibilities across gems (for instance there was a time recently where if you had both datadog and newrelic installed and used elsaticsearch upgrading the newrelic gem would break prod)
We have some ideas on how to figure this out automatically (like reading code diffs with GPT rather than just changelogs), but the best way is to see upgrades out in the wild. As we see more and more upgrade experiences from our customers we'll be able to catch these issues and build this dataset.
I'd love to hear other ideas for making major framework upgrades safer.
> * It ran the repo's test suite against progressively newer versions of dependencies. Showing when and where unit tests fail.
Would this be something like `git bisect` for upgrades? We run your CI to figure out where you can upgrade to without something breaking? We're trying to figure this out statically by reading changelogs and building up a database of breaking changes. Our roadmap is to make this database better over time by pulling in more and more undocumented changes (by seeing customer upgrade experiences and by sourcing information from places like github issues). In my experience it's more common (and much more dangerous) that things break when your dependency changes in a way that wouldn't be caught by CI.
Toward the middle of last year I re-connected with Andy (our third cofounder) who I went to school with and have wanted to work on a company with for a long time. We did a bunch of customer discovery / product work to figure out how to take what I learned doing this by hand and turn it into a software product. That was exciting enough that we decided to bring Andy on as a third co-founder and pivot our YC company.
Allison and my background is in building data businesses. Before Infield and Syndetic we worked at a startup in the beverage industry where we standardized inventory data for every alcohol product sold in the US. As we got into building Infield we didn't expect to use LLMs at all. We imagined a similar human-in-the-loop expert system to what we've built before.
I've been extremely impressed with recent language model's ability to handle unstructured changelog text. For example, consider the following snippet of a changelog:
Security:
- Address an issue with password validation
Breaking change:
- The `foo?` method now returns a boolean instead of int
Language models can carry the context through, so we are able to not just parse this apart into discrete changes (which I could figure out how to do with a regex) but also bring in context and categorize them. It can do this generically and really feels like something new.Is your frustration mostly from needing compile all this non-python stuff across environments?
Infield helps software teams keep their open source dependencies up to date. We’re automating away the toilsome work of reading changelogs, assessing risk, and upgrading packages so that software teams can focus on shipping features. We’re a small team of repeat founders passionate about making open source easier to use.
You’ll be helping found our engineering team with the opportunity to take ownership of large pieces of our technology stack. We're building in a wide range of areas - from large language models to complex data pipelines to modern, interactive web front ends - and want people who are excited about learning new technologies and passionate about open source.
Role: Full-stack founding engineer Stack: Ruby on Rails (with turbo for the frontend) and Postgres
You can email me at steve (at) infield.ai
Anyway thanks for the tip!
Using KDE or Sway, if I set my desktop scaling to 2x, all programs that run through XWayland have extremely blurry display (they've been rendered at 1x and then crudely scaled up). This includes firefox, chrome, and emacs. The browsers have ongoing work to support wayland natively, but neither is stable for me.
If I set my desktop scaling to 1x, I can then scale up individual xwayland programs with flags like `--force-device-scale-factor` if they support them. This gets me close to the behavior on X but on X I can just set a global 2x scale.
I totally understand the frustration of OSS maintainers when people demand they fix things with no intention of getting into the code. On the other hand, the marketing behind wayland encourages some unrealistic expectations. I went out and bought a new graphics card and it turns out I can't even run firefox! That was not what I expected to happen.
Surely it's progress for devices to be able to securely access name servers? I can't snoop on the network traffic going over https but somehow I can get a list of all names queried?
I gave up and just created an ext4 partition and things got a lot more stable.
I can only speak for myself, but as someone who needed a printer for the first time in a while due to the pandemic, I bought a used brother laser printer (HL-2170W) for $20 from someone local. It's not perfect, especially with network printing from a linux client over wifi, but I was able to get it all set up and refilled the toner with a very cheap generic from Amazon. It feels like a device I "own" rather than some black box I'm renting from HP.
I don't know where the excitement around buying the new ones comes from. When I looked it was hard to find them in stock and they didn't seem particularly inexpensive.
Getting about a million "Access to fetch at 'https://example.com/' from origin 'https://map.mta.info' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled."
1. X can't handle independent screen scaling of two different DPIs. I have 24" and 27" screens and on windows I can have the 24" one at 2x and the 27" one at 1.5x and it "just works". In X I have both at 1.5x.
2. Wayland supports independently scaling the two screens. Unfortunately programs that don't support wayland run through something called XWayland, which upscales them in a crude way that makes the fonts blurry. It's pretty much unusable because of this to have an XWayland app (this includes chrome, firefox, and emacs) running on a 4k screen in wayland, at least on KDE.
It's frustrating because it's SO CLOSE. There are WIP branches of both browsers running on wayland (but they're not stable for me) and once those land I think everything will finally work.
All this said I do still use KDE on X daily as my development environment, but every time I boot back into Windows I'm frustrated again by how much better the scaling is.