Heuristically, anytime you're doing I/O (network or filesystem) could be an opportunity to increase concurrency with async/await. By using asynchrnous I/O and trickling that asynchronocity all the way up to ASP.NET, you make much more efficient use of the ThreadPool and allow more requests to be served. That's because you are recouping cycles otherwise lost on blocking.

This isn't the only way to increase concurrency. Obviously you can grow your IIS web farm as well. But async/await (or some other .NET asynchronous programming pattern[1]) is a one-time .NET development cost as opposed to a recurring infrastructure/hosting cost.

[1] http://msdn.microsoft.com/en-us/library/jj152938.aspx