I found it relevant and actually just the information I was looking for. Having a highly recommended model behind the tool makes it worth further investigation.
59 karma · joined July 6, 2021
M1 Max: FP16 hardware support, FP8 and Bfloat16 emulated in software (via dequantization)
H100: FP16 and FP8 hardware support
> which I ran both on a MacBook Pro M1 Max and a rented H100 SXM GPU
I actually found Facebook’s translations pretty good (better than Google Translate for things longer than a sentence). From my understanding of Khmer, Khmer is a bit more verbose and context dependent, hence LLMs in Khmer would be a big help understand those nuances.
In the inverse case (LLMs generating khmer from English) I heard from locals that it sounds formal and “robotic” which I found quite interesting.
- You’re right, data is very hard to come by. I’m curious, how do you plan to get around this? Outsourcing human labeling? We found it to be a very difficult task.
- The subcontractors and local construction companies we talked to were overwhelming excited about the idea.
- It’s entire people’s jobs to get this done and done correctly. They sit on site holding the pdfs in their hands, manually counting and calculating. You bet a lot of mistakes occur. They would absolutely love to have a digital assistant for this.
- Some of them (especially managers and owners) are quite technical and are using software such as BlueBeam and other CAD software to make these calculations. It’s quite manual currently, but gives great insight into a better solution. This led us to having the user manually select the symbol they wanted counted (which ML struggled to get right). Just getting the part counts (and highlighting them in the pdf) was a huge help!
- Impressive you got square footage calculations correct! In our experience, there was way too much variation between architects (and multistep dimension labeling) which made it hard (even for humans) to get right. How has your model generalized OOD thus far?
- Are you planning to integrate voice? Many of the subcontractors we worked with are very low tech. They usually talk with their clients in person, on the phone, or maybe text. But they don’t use email or their smart phones for much.
I will be following your work! I have friends who would love to use this once it passes the human threshold.
It’s an eye opening alternative explanation to the electrons flowing like a chain theory of this article.
They might be trying to create toxic back links to their domains and if those domains 301 to your domain, I believe this can negatively impact the SEO of your domain (from what I read). If so you can try to disavow them https://support.google.com/webmasters/answer/2648487?hl=en
I’d take this with a grain of salt (pun intended). There’s a lot of bugs that you cannot reproduce without certain permissions or a particular environment. Let alone the race conditions or user setup. In my experience, most bugs would not have been uncovered using this brute force approach. A few tests using your understanding of the code and critical thinking goes a lot further in my opinion.
There’s a few other expenses and some cons of living here but some research and YouTube videos will help you figure out if it’s right for you. And of course you can ask me :)
I’d also say £40k was way too low. I’d guess the founders have significantly higher upside. It would be worth asking for transparency in order to determine fair compensation.
Laser ($40): https://a.co/d/0wjNGBz
Diffraction grating ($12): https://a.co/d/6bpO8xm
Laser focusing lens: Not found
Fluorescence collection lens: Not found
Focusing lens: Not found
Collimating lenses: Not found
If you know of any implementations that can look at a spectrogram and say “hey there’s peaks at 150hz, 220hz and 300hz with standard deviations of 5hz, 7hz, and 10hz, decreasing in frequency over time thus this is a deep voice saying ‘ay’” and get it right every time I’d be really interested in seeing it (besides neural networks)
Personally I hypothesize that the reason it’s so hard is that the sources are intermixed sharing frequencies so isolating to certain frequencies doesn’t isolate a speaker. We’d need something like beam forming to know how much amplitude of each frequency to extract. I’d also hypothesize that humans, while able to focus on a directional source, also cannot “extract” clean signal either (imagine someone talking while a pan crashes on the floor - it completely drowns out what the person said)
1. STFT (get frequencies from the audio signal)
2. Log scale/ decibel scale (since we hear on the log scale)
3. Optionally convert to the Mel scale (filters to how humans hear)
Happy to answer any questions
keys = key_weights * x
query = query_weights * x
values = value_weights * what_to_look_at
For self attention, what_to_look_at = x
For regular attention, where_to_look_at could be a database, memory or anything else.
So in this example if we’re trying to predict the second “apple” the first “apples” is very helpful. If we’re predicting “juice” then we’d use one head of self-attention to look at the first “apples” and a second head to also look at the second “apple”
That’s my understanding at least
I started noticing thoughts that were so negative/depressive that I was actually aware of how irrational it was. For example things are actually quite good but my mind is in the dumps. So called Automatic Negative Thoughts (ants).
Since then been trying to reprogram my brain. Force those thoughts out of my head. Start singing randomly. It’s hard and I’m not always successful but I’ve definitely noticed a big improvement overall.
Hopefully this helps you too!
Time to start building a neural net to invalidate patent trolls.
https://github.com/djsamseng/cheat_sheet/blob/main/grep_for_...
#!/bin/bash
if [ $# -eq 0 ] then echo "Usage: ./grep_for_text.sh \"text to find\" /path/to/folder --include=*.{cpp,h}" exit fi
text=$1 location=$2
# Remove $1 and $2 to pass remaining arguments as $@ shift shift
result=$(grep -Ril "$text" "$location" \ $@ \ --exclude-dir=node_modules --exclude-dir=build --exclude-dir=env --exclude-dir=lib \ --exclude-dir=.data --exclude-dir=.git --exclude-dir=data --exclude-dir=include \ --exclude-dir=__pycache__ --exclude-dir=.cache --exclude-dir=docs \ --exclude-dir=share --exclude-dir=odas --exclude-dir=dependencies \ --exclude-dir=assets)
echo "$result"
What if we had a centralized certificate authority that verified a person? Imagine you walk up to the DMV and get a private key (password). When you go to a website you generate a public key and send it to the website you visit. That website uses the public key it received to send a message to the certificate authority to verify you (true/false). Now Instagram knows you are real, but are you faking? I claim to be "First Last" to Instagram. Instagram encrypts "First Last" using the public key and sends it to the certificate authority. If the certificate authority is able to decode "First Last" using the private key then it returns (true/false).
Could we extend this to solve user privacy? What if users said "Track me all you want as long as you don't know who I am". Websites can still serve targeted ads but users get the privacy that you are incognito. Instagram now knows your name but they also want to be able to identify you across the internet. So Instagram could also ask the certificate authority for a "personId" that identifies the person across the internet. But now you say, wait now Instagram knows my name and all my activity through my "personId". This is where the Engineers come in. We would have to make any code or action that connects "personId" to a human _illegal_. You write the code, you go to jail. This burden would only fall on websites that ask for someone's human identifiers (name, address, common geolocations, etc.). But that code isn't needed anyway! There is no reason to store "personId" and "First Last" together because you can always get a "personId" when the user gives the public key to the website. So if someone ever writes that code / uses that data query it's punishable by law.
So now we have 1. Every website knows it's users are real 2. Every website can know a user is who they say they are 3. Every website can track unique visitors and their internet activity (while not knowing who they actually are) 4. Every user is completely "anonymous". Yes the information could get out, but only temporarily because any code (even a news article or blog) that contains this connection is illegal.
- You’re checking out everyone else’s PRs and requesting changes for all the bugs you find. The few times you don’t a major outage occurs.
- Everyone’s asking you for help all the time. You actually find a way to “help” all of them but really you’re just solving their problems for them
- You say you love your job, but really it’s not like what it used to be. You’re no longer excited to be woken up at 2am to fix an outage.
- You never aimed to be here, doing this, working here. But here you are and this is now your baby.
Your baby has grown up. As long as you are here the stress will eat at you. But can I encourage you that a better life awaits you. Change will be hard, but what is waiting for you on the other side is so much better. But it takes humility, humility that “project x” will go on without you (definitely not as smoothly but others will step up to the plate). Humility to try something new where you are in the bottom 10% working your way up once again. It will feel like you’re giving up all the hard work you’ve put in, giving up what makes you king of the hill, but it’s worth it. It’s worth it because life isn’t about being king, but about the journey. Welcome to a life of freedom, of learning something new every day, of waking up each morning and diving into whatever is on your heart to discover.
Don’t quit social media. Don’t restrict yourself by setting rules you’ll break a few days later. Why? Because it doesn’t work.
Life can be and will be about what you are passionate about.
So sure go ahead and enjoy wasting time on the internet. But after 15-30 minutes when it’s no longer enjoyable but rather you are seeking that enjoyment you first had, get up and try something new. It might be awful (so one and done), but it also might be amazing. It might just become the activity you wake up excited to do every day that social media/web scrolling doesn’t even compare to.