I believe the ideal order is "Refuse, reduce, reuse, recycle, rot."
Refuse: Just say no to stuff. You don't have to worry about dealing with the waste from things that you don't take in the first place. This can be taken to not-helpful extremes, but in general, this is the best option. It's always depressing to see just how much trash (of the "actual trash" variety) one of our very rare take-out meals produces. If we all (4 of us) have something from a drivethrough, the amount of trash generated by that one meal is very close to a week's worth of kitchen trash for us with other meals. The paper is often too greasy to recycle, waxed paper cups are good for firestarters but little else, etc.
Reduce: All other things being equal, "less crap to dispose of" beats "more." I'll point to standard blister packaging as the sort of thing one ought avoid, if at all possible. Not only is it a very real health hazard during the opening process, it's almost entirely impossible to do anything with it after the fact. This is where bringing your own bags to the grocery store fit in, or for things like loose vegetables and such... just don't use a bag! You get some weird looks in the checkout lane when you dump half a dozen onions on the conveyer, but I remove the outer skins anyway when I work on them, and I don't have a plastic bag to deal with as a result.
Reuse: Anyone with small children (and lacking weird quirks about "You're playing with a box!") is probably an expert at this. I cannot tell you how many times a box something came in has turned into a good play toy for a week or two. Random kitchen boxes can be taped up and turned into blocks, corrugated cardboard is just awesome for everything, waste paper from address labels or such is useful for drawing, etc. Lots of options here. I've considered getting a setup to smash glass up and embed it in concrete I pour for various things, though I'm not sure this really gains me an awful lot in terms of mass.
Recycle: This is the standard first response, and depressingly often, "Recycle" actually translates to, "Oh, yeah, totally buy this thing, it can be recycled!" It's just an excuse to consume more. If you want to be quite upset by this, look at the state of plastic recycling in developed nations. It tends to consist heavily of "We loaded it into a shipping container, sent it to a third world country, and counted it as recycled, we don't care what they actually do with it." "What they actually do with it," often enough, is open burning, or just letting it wander down the river into the ocean. If you're concerned about the environment (as opposed to purely carbon emissions and GHGs), plastics should be a huge, huge problem. Unfortunately, literally everything comes in them.
Rot: Compost. If it's organic, or nearly so, compost it. Hot piles work best.