Framing SICP as a book is a little misleading. I mean, it is a book, but mostly it's a collection of ~350 programming exercises. Exercise solving time will dominate reading time like 10:1. It's worth it! But plan accordingly.
I guess everyone has a different way of approaching this.