Exploring LoRA – Part 1: The Idea Behind Parameter Efficient Fine-Tuning | Hacker News Reader

Exploring LoRA – Part 1: The Idea Behind Parameter Efficient Fine-Tuning

medium.com

166 pointsaquastorm15 comments

Author here. Happy to see this posted here. This is actually a series of blog posts:

1. Exploring LoRA — Part 1: The Idea Behind Parameter Efficient Fine-Tuning and LoRA: https://medium.com/inspiredbrilliance/exploring-lora-part-1-...

2. Exploring LoRA - Part 2: Analyzing LoRA through its Implementation on an MLP: https://medium.com/inspiredbrilliance/exploring-lora-part-2-...

3. Intrinsic Dimension Part 1: How Learning in Large Models Is Driven by a Few Parameters and Its Impact on Fine-Tuning https://medium.com/inspiredbrilliance/intrinsic-dimension-pa...

4. Intrinsic Dimension Part 2: Measuring the True Complexity of a Model via Random Subspace Training https://medium.com/inspiredbrilliance/intrinsic-dimension-pa...

Hope you enjoy reading the other posts too. Merry Christmas and Happy Holidays!

Thanks for sharing. This got me thinking, why is medium so used for such technical articles? Especially that lots of articles get blasted behind a paywall for me recently.

Making it less accessible, right? I was thinking exactly the same.

short answer: to make money

(Not to be confused with LoRa, (short for long range) which is a spread spectrum modulation technique derived from chirp spread spectrum (CSS) technology, powering technologies like LoRaWAN and Meshtastic)

really wish they had come up with another name. googling gets annoying

Contributors: They both use mixed capitalization. They have partially-overlapping audiences.

This gets me every time. I expect to see something interesting and it turns to be the other one. One is a fantastic thing and the other is mediocre, pick which way round at your discretion!

Pretty simple to spot LoRa vs LoRA.

Memory mnemonic: Capital A for "AI"

What exactly is the confusion? Does “parameter efficient fine-tuning” mean anything in context of the other Lora? If not, then it’s probably obvious which one this is about.

Actually it does: Lora the radio protocol has parameters to tune. Usually both sender and receiver needs to match these, so I read this like a method how these could be automatically tuned based on the distance and radio environment.

Super cool series of articles! :)

Generated notebooklm here: https://notebooklm.google.com/notebook/7094a513-af83-4c5b-a4...

What is this? Is this a google summarization service?

Reply on news.ycombinator.com