I work for UniProt (a databases for biologists) and for us it is really a nice way of providing data to other users to integrate into their own systems. We also provide a sparql endpoint at beta.sparql.uniprot.org for other scientist to run analytical queries without having to deal with running a large datawarehouse of their own (beta.sparql.uniprot.org). i.e upload 5kb of SPARQL queries to us instead of downloading a 100Gb of data them to run SQL in their own systems.
These days there are a number of off the self systems that work well enough. Your main danger in the beginning is trying to deal with existing (large) public data before you are ready. The philosophical/reasoning side of the semantic web can also be very confusing but can be ignored in the beginning. Just use RDF as graph data format and SPARQL as a query language and you are good to go.
In the future I'd like to work at a place like this, so if you could tell me more I'd be grateful. (My email is in my profile.)
Thanks!
http://wiki.dbpedia.org/OnlineAccess?v=6yg
The biggest fly in the ointment right now, IMO, is that working with larger datasets, with queries that require complex inferences via OWL, are crazy computationally expensive. And while there is research underway on the subject of parallelizing this stuff so it can run on clusters, I wouldn't say that that is a solved problem yet. So you can need some pretty honking fat software if you want to do really complex stuff.