It’s basically a full version of Visual Studio with no restrictions, except that you can’t use it in an enterprise setting and for teams with more than five people (you can, however, use it for any other kind of commercial and non-commercial project).
- DirectX debugging tools, including GPGPU
- Parallel code debugging
- MFC and ATL for those that still need to support such type of applications
- 64 bit compilers
- DDK integration
- Sharepoint integration
That was true of most of them through 2010 (though the 2008/2010 Web Developer Express supported, IIRC, at least C# and VB.NET as backend languages.), but not the 2012/2013 versions.
The VS2012/2013 Express Editions are not language specific, but project type specific -- the 2012 editions were "for Web", "for Windows 8" (i.e., Metro/Modern UI), "for Windows Desktop", and "for Windows Phone". 2013 merged "for Windows 8" and "for Windows Phone" into "for Windows".