Absolutely, yes.

Think of it this way: If current runs through it, you need to cert it. Period.

Edit: I think the Pi might already have FCC so if you are NOT electrically altering the Pi in anyway, then you won't need to re-cert (as you are essentially installing software on it and for a consumer device FCC doesn't have ESD requirements).

For CE, depending on the directive, you'd need to do ESD (adding a box will alter that).

Disclaimer: Not compliance advice.