Good point, but such a prompt would be similarly annoying (i.e. annoying to a similar number of users). The way I've done this in the past has been to provide two links, one of which triggered the inclusion of HTTP headers at the server, to trigger the Save dialo). Assuming the download attribute is supported, it's even easier:

  <a href="file.txt">File</a> (<a href="file.txt" download>Download</a>)

This could impact visual design in a negative way. A download icon[1] could be used instead.

  <a href="file.txt">File</a> <a href="file.txt" download class="download-icon" title="Download this file">Download this file</a>

CSS would be used render an icon and to make the text of .download-icon conveyable only in non-graphical browsers.

[1] http://fortawesome.github.io/Font-Awesome/#icon/icon-downloa...