Anyway, there are lots of things that offend people about PHP that are basically inexcusable: like the inconsistency in parameter order (needle, haystack or the other way around?) and the inconsistent use of underscores in API method calls.
But all of that is just noise. Use something like PHPStorm, Netbeans, Komodo or whatever and it'll tell you the right order or if the function doesn't exist (I know some people are still in love with their raw text editors; I'm not one of them). And you learn quickly enough the right names and orders.
One thing I love about PHP is the simplicity of it. Edit the file, save, refresh in the browser. I love this short feedback loop. It was a refreshing change from doing Web apps in Java (for example), which added a compile/build/package step in between.
After doing this I investigated the common frameworks (CodeIgniter, Kohana, Zend, etc) and frankly I was horrified.
These things are so hideously overcomplicated. It's like a zealous Java programmer tried to recreate "Super-Struts" in PHP in a devilish plan to destroy all of the advantages of PHP while leaving a lot of the negatives.
The number of times I got bitten in the ass or just wasted time trying to figure out how the "magic" in Zend worked was infuriating.
Take something as simple as a controller. This is stupidly easy to implement yourself that any sort of generic framework is just going to bring you pointless crap you don't care about to the stuff you do by a factor of probably 10:1.
The ORM side was even worse. Java is kinda built for ORMs. It has stateful Web containers. PHP isn't (and doesn't).
The one bright spot I found was templating. It is common to say "why use a templating language with PHP when PHP is a templating language?" and you have an entire valid position. I too have said this.
That being said, I found Smarty to be incredibly useful just from the point of view of caching parts of the page and creating my own plugins/extensions for tasks that were common to me. Smarty had other issues like I wasn't completley fond of it's caching model and the speed, prior to Smarty 3 at least (which I've never used), was also questionable.
But despite all that I liked it a lot.
So can you use PHP without a framework? Sure you can. Not only that, I'd highly suggest it if you know what you're doing.