So first a couple caveats: (1) I'm sure AVR is a hell of a let better than PIC, (2) I come at this from a really weird place (exclusively emulators and compilers), and (3) I'm not talking about the AVR parts themselves, which might be more cost-effective for a given project.

That said:

* Harvard architecture (split I & D memory)

* 8 bit registers

* Not at all orthogonal, and particularly painful for pointer addressing

* The stack pointer is annoying to work with

* Address wrap at physical memory bounds, rather than the bounds of address space

I could pick a bunch more nits that would only really be relevant to someone writing an emulator (complicated instruction decode, IO addressing, &c) but those are my big complaints.

I find MSP430 much more pleasant to work in.