I've found that the benefits in terms of developer experience, and being able to build good UI quickly, start accumulating almost immediately, even on tiny projects.
My initial reservations about the amount of boilerplate in setting up Relay, were in large part due to the server-side half of the equation. Since switching to building my backend in a different language (now using Elixir with absinthe, rather than node and graphql-js) I've found my productivity has skyrocketed.
At this point the overhead in terms of file-size (and the still pending support for server-rendering) are the only things giving me pause for thought.