I bet you're able to understand that time point without me supplying the location. Weirdly enough, this advantage is more pronounced in the age of globalization.
Right now, we do that via time zones. If we went to universal UTC we'd still need to be aware of and deal with those longitude differences. It would still be convenient to quantize those differences into multiples of 15 degrees...and so we still end up with zones.
We could call them "space zones" or "longitude zones" instead of "time zones", but since the main purpose is to let you figure out what time on your clock corresponds to events like sunrise in other locations, we might as well continue to call them "time zones".
If you’re both in the time zone, you need to know A) his work schedule. The idea that a person’s workday might be 1 AM - 9 AM might be difficult to comprehend, but only relative to our current understanding.
The benefit of time zones (that roughy correlate to local solar time) is that I can get a good idea about their day without any other info; (1am they're probably sleeping, 12pm they're probably at work but might take lunch). If I need to convert between time zones I use my handy lookup table that any computer/phone or back of the envelope math can do. If I want to have a meeting at 1pm in some new fancy Internet Time I have to do some gymnastics to figure out if my friends in India are even awake, let alone working. If I'm flying to Japan and my flight lands at 6am Internet time--should I find my hotel to sleep or go watch the sun rise?
Also: how would you feel about starting your shift at work at 4:00 AM? because in UTC that's already 9:00 AM :) (assuming you're on east coast. On west coast your working hours would be 1:00 AM - 9:00 AM)