Are you suggesting that time extends infinitely into the past? Because if it doesn't, there will never be an infinite amount of time, so there won't be any point in time when this -1/12 becomes relevant to physics in this sense.

> I guess my point is more that for a mathematician it would probably be obvious that when you talk about the limit of a divergent series it could be any imaginary or intermediary value.

I'm a mathematician and I might agree (not entirely sure what you mean). But -1/12 is a concrete value so it doesn't apply here.