Well, actually, in C# struct instances ARE objects allocated on the stack, instead of the heap.
A parallel can be drawn with C++ in which the user of a class decides on declaration/initialization where to store the object (either on the stack, or on the heap). The difference with C++ is that in C# the author of the class decides where instances of it should be stored, so this decision happens when the class is declared, not when it's used.
Otherwise there are few differences between structs and classes and all differences stem from the differences in storage. For example struct instances must be passed by value and not reference, because by definition stack-allocated values are short-lived and playing with references to stack-allocated values is dangerous. Structs must also have an implicit constructor because stack-allocated values cannot be NULL (logically, you need a reference to represent NULL).
In my experience, all discussions about what is or isn't an object are counter-productive.
What really bugs me about Java is that you cannot build your own types that behave just like the built-in types. For instance you cannot override operators like "+" (which works for primitives or Strings), you cannot override [] (which works for arrays), you cannot declare other stack-allocated structures, arrays are reified and yet you cannot declare your own reified data-structures and so on. The presence of primitives doesn't bother me as much as lacking the means to build my own primitives.