I agree on these reservations, especially in terms of silent errors (which get compounded through minor ways in which backwards incompatibility can sneak in to the existing scripts) and dependencies.

As a half-solution, I ended up restricting myself to a very few libraries in this family (mainly dplyr, lubridate, stringr, broom) and to using packrat to consistently freeze the library versions for these.