2,133 karma · joined June 13, 2008
https://www.linkedin.com/feed/update/urn:li:activity:7374472...
Disclaimer: I've enjoyed many delicious meals at Rand's table
The knife is amazing and exactly as shown in the video. Rand Fishkin has a nice short on LinkedIn trying out the knife too. I think he shows one his (sharp) kitchen knives slicing through a lemon, then the Ultrasonic. It's astounding.
Disclaimer: I am a (tiny) angel investor in Seattle Ultrasonics.
Anyway, I ended up with a hack like this, which works fine but didn't feel great.
def some_method(arg)
@_memo_wise[__method__].tap { _1.clear if _1.length > 100 }
...
end
memo_wise :some_methodTableTennis is a new gem for printing stylish tables in your terminal. We've used ad-hoc versions of this in our data projects for years, and I decided to bite the bullet and release it as a proper gem.
- auto-theme to pick light or dark based on your terminal background - auto-layout to fit your terminal window - auto-format floats and dates - auto-color numeric columns - titles, row numbers, zebra stripes...
By far the hardest part is detecting the terminal background color so we can pick light vs dark theme for the table. This requires putting the console into raw mode and sending some magic queries. These queries are widely supported but not universal. There are some great libraries for doing this in Go & Rust, but as far as I know nothing like it exists for Ruby. Check out the long comment at the bottom of this helper if you are curious:
https://github.com/gurgeous/table_tennis/blob/main/lib/table...
As always, feedback, feature requests and contributions are welcome.
export PATH := justfile_directory() + "/node_modules/.bin:" + env_var('PATH')
ci:
@just banner yarn install
yarn install
@just banner tsc
tsc --noEmit
@just banner lint
eslint src
prettier --check src
@just banner vitest
vitest --run
@just banner done!
banner *ARGS:
@printf '\e[42;37;1m[%s] %-72s \e[m\n' "$(date +%H:%M:%S)" "{{ARGS}}"
This example is a bit contrived, more typically we would have a rule like "just lint" and you might call it from "just ci".One of the best features is that just always runs from the project root directory. Little things like that add up after you've spent years wrestling with bash scripts.
For more advanced needs, I have a custom thing called greprep that let's you make changes using your favorite editor. Workflow is like this:
1. $ rg -n .... > /tmp/lines.txt
2. (edit lines.txt in vscode)
3. $ greprep /tmp/lines.txt to apply the changesFull list here - https://gist.github.com/gurgeous/bcb3e851087763efe4b2f4b992f...
The blog post is the analysis of the data set, the map is the visualization.
Disclosure: I am one of the authors
tv looks excellent. Fun name. I think if you added a couple of features it would ascend to my toolbox:
(1) scrolling (horizontal and vertical)
(2) better command line parsing. Running "tv" without stdin or arguments should produce an error/help. Running "tv xyz.csv" should read that file.
Good luck!
(edit: I am at a desktop now and I can say a bit more)
Here is the process in a nutshell:
1. Create a fast hashing algorithm to find rows that might be dups. It needs to be fast because you have lots of rows. This is where SimHash, MinHash, etc. come into play. I've had good luck using simhash(name) and persisting it. Unfortunately you need to measure the hamming distance between simhashes to calculate a similarity score. This can be slow depending on your approach.
2. Create a slower scoring algorithm that measures the similarity between two rows. Think about a weighted average of diffs, where you pick the weights based on your intuition about the fields. In your case you have handy discrete fields, so this won't be too hard. The hardest field is name. Start with something simple and improve it over time. Blank fields can be scored as 0.5, meaning "unknown". Hashing photos can help here too.
3. Use (1) to find things that might be dups, then score them with (2). Dump your potential dups to a CSV for human review. As another poster indicated, I've found human review to be essential. It's easy for a human to see that "Super Mario 2" and "Super Mario 3" are very different.
4. Parse your CSV to resolve the dups as you see fit.
Have fun!
Sample Ruby code:
8.times.map { (('a'..'z').to_a + (0..9).to_a).sample }.join
It's less collision resistant than UUID, of course.