But for something on the backend, like a programming language or a database, how are these things user tested? Do concerns about usability fall by the wayside because most users will become experts in the system?
If user-testing is not used, how are users taken into account when deciding on the roadmap?
1. Programming language While this is probably more difficult than my cursory attempt to explain, it seems that testing programming language is more a problem of ensuring consistent, and expected input & output as well as performance characteristics of stdlib and core code across versions.
for example -- if you're writing stdlib for working with strings, you can write unit tests for most of the string functions, and do some simple timing analysis as well as output analysis to make sure that with every revision of the code that might make it into a next version maintains same-or-better test performance.
2. Databases I think that you can probably talk to the guys at rethinkdb or any other up-and-coming database about this -- there are certainly things you can test, of course like throughput (+/- heavy load), number of clicks to do a certain function that has been determined to be a core feature, etc. I think in the database world, you have a duality in the "user" -- yes, the UX must be good, but also, databases look to perform well and consistently, you must address the programmer and the normal user side of the developer that is using your new tool/database.