so i am writing this reply from brooklyn and do this regularly to visit my parents in central nj: the direct and perhaps cheeky answer to your question is "staten island," and the one time google maps duped me into thinking manhattan was a better route was one of the most miserable drives of my entire life
we street-park a car here most of the year - if i use the car to visit them the travel time is shortest (20m walk -> 1hr 5m drive)
but i feel very lucky to have a bunch of non-car options - if the weather is nice or i'm off-peak i can ride my bike to NYP in manhattan (8 miles), take my bike on the nj transit train to their stop, and ride the last three miles to them (~2h)
otherwise i can take the subway -> path -> nj transit (~2h 50m, but if schedules were more sensible it could be closer to 2h 20m)
and you might be like "the car is the obvious answer, look at how well you've proven my point!"
but that neglects the time requirement and responsibilities of keeping a car in NYC that i think should be factored into this - either 90m of your time once a week to move it for street parking, or a $65 ticket most of the time. if i'm returning to the city finding a spot can take 25-60 minutes and is often a 15-20m walk from my apartment.
there's a meaningful transaction cost each time the car is used AND in storing the car. and that's with free street parking once you've found a spot, which i don't think nyc should have in the first place!