I think my biggest difficulty is that I'd like to be able to look at a duration of time and determine the events within that range.
Pulling raw iCal data over HTTP is possible but if you say, set up a recurring event several months back, you need to fetch that original event (even if it recurs every day) to include it in your calculations... I'd love a self-hosted version of this Nylas API [0] for example.