Choice of language is pretty much a matter of comfort when you're first learning. The question is, do you want to be using Ruby/Rails or PHP/Wordpress/Codeigniter/etc. If you're just learning, you're going to want to code in whatever language you choose. Just for reference, you can create backends for websites in Clojure, Java, Python, Haskell, go, JavaScript(node.js), etc as well.

I'd say use Ruby/Rails if you want to create webapps and PHP/etc if you want to create Wordpress sites. Someone more experienced than me in these languages can comment further.

Heroku.com supports Ruby and from using it with Clojure and node.js apps, I highly suggest this route even if you enjoy setting up your own servers. You can also use providers like Amazon EC2/Joyent/Rackspace and set up your own box to be either a LAMP stack or Ruby/Rails.

There is a long list of Databases to choose from these days, including: MySQL, PostgreSQL, Riak, CouchDB, MongoDB, Cassandra, etc.

Truly the answer you're looking for is "Just start somewhere". The amount of tech you can choose between is dizzying and you'll eventually get to a point where you can make your own decisions if you just keep hacking.