The difference is that none of your coworkers got to sell a company to Yahoo! at an inflated dot com era price to fund a venture capital firm.
Sure, YC is a famous seed accelerator, but it’s also not unlike a number of other early stage investment firms that also provide mentoring and guidance to early stage founders. The success of these venture firms boils down to a math equation and a selection process, not any sort of unique secret sauce.
It is a venture of owning capital and growing it, not of any sort of unique skill set.