Good grades just validate what I see on experience, bad grades tell me something might be off - is it all smoke & mirrors.
And yes, it only matters for the first job. After that, I honestly hardly even look at what college you went to.
I'd suggest also breaking out your GPA for your major vs. overall. That tells me that you're really good at your major (probably related to the job you're applying for anyway) but that maybe you just are bad at liberal arts.