Yes - this is unfortunately everything. There are a number of categories to optimize (research area, lab size, funding, university, other lab members, etc) but the biggest factor is your relationship with your PI. I cannot imagine working as a PhD student in a lab where I don't deeply respect and get along with the PI. I was very lucky but selected for this when choosing PhD labs. A surefire way to have a terrible few years is to ignore others advice about PIs to avoid.