One possible downside is that someone could Redirect A -> B and redirect B -> A, which risks tying up your resources following links, but browsers limit how many redirects will be followed, so it ought to be possible to limit redirects.
The problem is with services that consume other content. For example you might have a service which generates thumbnails of sites.
That service might GET https://attacker.example.org/301.html which itself might 301 back to file:///etc/passwd . If there is insufficient validation then a screenshot of the contents of /etc/passwd might be returned by the service.
All of that happens outside the context of browsers and sandboxing.
For more of that kind of thing, here's an interesting write up on some vulnerabilities found in Pocket. https://www.gnu.gl/blog/Posts/multiple-vulnerabilities-in-po...