It's important not to blow off a few joules when you're speaking of radiation. The LD50 dose for a 75 kg human is only about 375 J (5 Sv) of gamma rays[1]. In this case, the 5 J delivered inside you (I didn't check your calculations on this, just using your number), and perhaps another 5 on approach and exit, only end up being about 10 J (0.12 Sv), which is not enough to kill, but it is still a rather large radiation dose. It's more than twice the annual limit for occupational safety in the US, and is near the lower limit of the dose that could cause acute radiation syndrome. If the hole passed directly through your digestive tract, concentrating the dose in the vulnerable tissues there, there's a decent chance you'd get mildly sick.

[1] Gamma rays have a weighting factor of 1 in the gray -> Sv calculation, and the black hole emits them isotropically, so they're roughly distributed across the body. So in this case, 1 Gy ~ 1 Sv.