258 karma · joined March 24, 2016
Updating anywhere from 500-5M items over APIs that are rate limited.
Dumping datasets that have to be normalized and massaged into files/ and dropped off at third party servers anywhere from every 15 minutes to once a month. These files could contain anywhere from 500 lines to 5M lines.
Injesting datasets just as large as described above but massaged and saved into our caches and DB.
Thanks for your thoughts!
Plus it makes me feel great knowing I'm contributing to reuse and giving money to a company that helps employee less fortunate people.
http://huckcdn.lwlies.com/admin/wp-content/uploads/2017/08/1...
~/$MAJOR_TOPIC
|
|--- ./$MORE_SPECIFIC
|
|--- ./$MORE_SPECIFIC
|
|--- ./general-file.type
|
| ./general-file.type
|
|--- ./$MORE_SPECIFIC
|
|--- ./general-file.type
etcAs you find yourself collecting more general files under a directory that can be logically grouped, create a new directory and move them to it.
Also keep all your directories in the same naming convention (idk maybe I'm just OCD)
https://blog.ycombinator.com/startup-school-radio-episode-2-...