Nice interface, but I can't find the link to the github repo!
Update, found it but not on the site: https://github.com/NJPTeam/nabby
95 karma · joined June 4, 2020
Update, found it but not on the site: https://github.com/NJPTeam/nabby
I see that it looks like it will be self-hostable and that the repo is forthcoming but I had to scroll a lot to see that.
Looks interesting though!
Devcontainers are a good example of their strategy. VS Code's source code is open source, but the devcontainers extension is not [2] and alternative vscode-in-the-browser providers are not able to use the devcontainers extension as they're not allowed to use the official VS Code extension marketplace.
[1] https://ghuntley.com/fracture/ [2] https://twitter.com/castrojo/status/1671544329402302464?s=20
One area I struggle with when thinking about building container-based development environments is the best way to avoid mixing in your IDE specific dependencies with your project dependencies. I think some of the commercial tools do this, but I haven't gotten in set up well in my homelab.
I just came across two articles by a former GitPod employer who moved to Coder (these are the two main providers of open-source VS Code in the browser solutions). They're both really interesting.
The first is on how effectively Microsoft has used VS Code to fracture the market place to their advantage by strategically open-sourcing parts of VS Code while keeping many of its best features proprietary (Pylance, the python language server is a good example of this). https://ghuntley.com/fracture/
The second article is about why he thinks Coder's strategy is more promising than GitPod's prompting him to go work for them. It's not as detailed, but it touches on some of the parts of container-based development environments that I've found overly limiting. https://ghuntley.com/integrate/
Conda isn’t perfect but takes on a lot of problems that pip doesn’t deal with at all. Regular conda is really slow these days but you can use mamba instead or just configure conda to use the libmamba solver and it’s much nicer.
The folks at prefix.dev seem to be building some pretty cool drop in replacements for conda too.
nb_conda_kernels is pretty reliable but not actively maintained. Gator from the mamba folks is new and still a bit rough around the edges but looks like it will be pretty slick eventually.
It would be great if DuckDB handled this itself, but it seems like to be competitive with Athena on really massive datasets, you need to have a metadata layer that is used to figure out which parquet files in S3 DuckDB actually needs to query and then potentially run those in parallel. This seems to be the architecture of Puffin (which I haven't personally tried using yet).
[1] https://www.boilingdata.com/ [2] https://boilingdata.medium.com/lightning-fast-aggregations-b... [3] https://github.com/sutoiku/puffin
One possible thing to look into would be whether this dataset is partitioned too much. My understanding is that the recommended file size for individual parquet files is 512MB to 1GB, whereas here they are 50MB. It would be interesting to see the impact of the partitioning strategy on these benchmarks.
[4] https://parquet.apache.org/docs/file-format/configurations/ [5] https://www.dremio.com/blog/tuning-parquet/
Consider this hypothetical. You’ve got two equally wealthy kids who go to the same private school, take all the same classes, and take the same SAT prep class.
Kid 1 has a perfect GPA and a 90th percentile SAT score. Kid 2 has perfect SAT score and a 90th percentile GPA. If there’s only one spot, which kid should a college admissions office pick?
Tools like Obsidian[1] and Roam[2] seem to be all the rage these days (in a very small circle of nerds, I mean).
Not a marketer though, so what do I know. Good luck!
If you’re a bond trader at some bank and all the other bond traders are chatting on Bloomberg, you’re missing out. But they’re not going to chat with me even if I got myself a Bloomberg terminal.
Have you given much thought to either type annotations or decorators. One or both of them could potentially be useful for attaching schema information to the DataFrame inputs/outputs of functions and for reducing boilerplate when you might want to validate those inputs/outputs before or after calling the functions.
If you're an analyst who knows some VBA, that can be super useful but it would probably be a mistake to try to make your VBA-driven applications bullet proof. Nobody wants you to spend that much time on it, and the odds that something completely out of your control will change and break it anyway are quite high.