I believe there is a clear legal difference here.

These things are over the line (and illegal):

  Using images without permission

  Using code without permission

  Referencing images and code from another site without permission

Gray area:

  Designing a site from scratch to roughly copy another site's design