897 karma · joined May 31, 2007
I think in a growing site with a lot of custom per-user content (like a social network) the extra complexity of a cache layer and managing expiry is more pain than it's worth while iterating the product quickly. If you're mostly a content site, it's definitely the #1 thing you should be doing.
Realising that we're both at the same time, depending on the user or page, means that sometimes the cache stuff is the right thing to do, and sometimes not. I was leaning too far towards not, and happy now with the balance we've picked.
I find it interesting that you prefer the broader "aggregate of the internet" ratings, rather than ratings based on your friends. The longer we run the site, the more we find it's a bit polarising. Some people like the average of the internet, some people like what's essentially a systematic word of mouth setup.
Sometimes it's a little tricky for us to gauge what's popular with streaming, as we don't get many of the options down in Australia.
Glen is bringing a lot of what he learned doing transport modeling and advertising analysis to the project - but we don't have any reference material specific to what we're doing now.
Keeping on top of referral fraud is a bit tricky too, and gets worse the more complex you make the referral system.
edit I also remember that way back then when the code was redone there was no themeforest, just flashden and audiojungle. We were heavily focused on finding new customers more than directing people to specific files. So that might give you a bit of context.
I don't feel your original comment reflected a position on knowing where to (or not) compromise, you made a strong assertion to take one option off the table in all situations.
In my experience, developers who are make blanket rules up front about what can and can't be changed in the future development of a system don't end up making much of value.
That's just my perspective, and I apologise if I've misread your character.
Even if you're saying that only in performance there can be no compromises (but generally people who say things like no compromises don't bound those statements) it will cause large compromises in other areas, such a future feature development or long term maintainability and readability.
... that's what she said.
We now pay a lot more attention to underlying stack. Just because you've outsourced hosting (either cloud or managed physical servers), you really need to know every component yourself.
We used to store and process all of our uploads from our rails app on a GFS partition. GFS behaved like a normal disk most of the time, but we started having trouble processing concurrent uploads and couldn't replicate in dev.
It turned out so GFS could work at all, it had different locking than regular disks. Every time you created a new file it had to lock the containing folder. We solved it by splitting our upload folder in 1000 sequential buckets and wrote each upload to the next folder along... but it took us a long time to stop assuming it was a regular disk.
Because of the isolation, we don't have too many pestilence problems (gross oversimplification) and we want/need to keep it that way.
Visting America was a different story though: I'm getting fingerprinted, and people are jumping queues, and the security is crazy... and all I'm thinking is "Why did I bother?"
Waiting for americans to care enough about the discomfort of vistors (relative to motivation to be there) might take a while.
It adds a lot of noise to the commit history if you're trying to differentiate between "oh, this bug fix or feature was added into master/trunk/whatever" and "oh, that was just bob getting the lastest code from upstream"
Once you get in the habit of finding smart ass comment for every situation, it's really, really hard to stop. It wasn't such a big deal as a plain old dev, but I'm managing other devs now and it's too easy to be an accidental asshole.
It would have been much easier if I never started. Now I've just got to try and watch my mouth and break the habit.
The often suggested use for observers is for caching (which is a good example). In our big app, if the cache expiry doesn't happen the site will look a bit funny but the underlying transactional data is fine. Mixing those together wouldn't communicate the difference in how important they are.
If you start walking down a dark alley - you start feeling insecure. You've got a few options from that point.
You can back out of the alley and hey, no more insecurity. A little embarrassing but you're feeling OK, you just have to go the long way round the block.
You can push on through the alley in the dark - something bad could happen or you could get through fine. You don't really know until you've popped out the other side.
Or you could carry a god-damned torch. Obviously when carrying a torch, you don't have it turned on all the time, just when you wander into darker areas where you're less sure of the environment. It lets you walk at full pace without stubbing your toe or getting mugged or whatever.
In more concrete terms: I'm a bit of a poor-weather TDDer (same with pair programming) If i start working on something that could be algorithmically challenging, cuts across multiple layers of our app, touches something scary (like our accounting code, because i don't want to lose any of our $$$), or am just plain stumped - I write a test. It fails. I make it pass. I write the next. I make it pass. I realise my code looks like ass, I refactor. The tests still pass.
I can give an actual example from this week. Our app is a large RoR app that serves multiple sites from the one codebase based on an internal DSL we have. We're in the final week of a must launch redesign (date was set because of a bunch of marketing, etc) and I get an urgent and large last minute change dropped on my desk (i'm sure you all know that feeling). I'll admit my first move wasn't to go TDD, I just went for the quick and hopeful move of quickly adding some code that looks like it should work to our DSL.
It didn't. So my very next move, knowing that I have a deadline I must hit with a risky technical change, was to start writing tests. If i hadn't, I'd have to be verifying my code works by checking around 3 different pages after a server restart which is both slow and 4 layers away from the changes. So it was faster in real time.
I'm not a big fan of TDD zealots, TDD is just another tool in the box. But TDD haters really get my goat. You're willfully shutting yourself out of a tool that can make your life easier, and help deliver code faster.
I've been the technical reviewer on a lot of hires and found Rand's process fairly similar to mine. I think it's good that he views the document as having 2 purposes: the first being to get through all the recruiters/HR to get it into the tech reviewer's hands, the second being grabbing the reviewer's attention enough that you get an interview.
I haven't worn a tie since I worked in a supermarket, and I sure as hell wasn't a professional there.
To get the most bang for my buck (developer time wise) I would visit each site with firebug in inspect mode, hover the data I want to extract. From there I figure out how I would style that element, and because Hpricot supports CSS selectors I've straight away got a method for pulling that data out of the page.
Or maybe I'm just venting about my shitty week at work :)