Beats me, English grammar isn't my strong suit.

I would write it (agree I you) since it is similar to a conjunction - the "with" is implied.