For dotnet exceptions are expensive where they are caught, at least if you're looking for the largest effect: it's the unwind.

As for smaller effects that matter in really hot code: try/catch also disallows a method from being inlined. Throw disallowed inlining when I wrote System.HashCode, but this was considered a bug and I'm not sure if it has been fixed - the workaround was to use utility throw methods.