Whatever that money would otherwise have been spent on is harming the recipient of that now foregone benefit. If you raise taxes to do it, taxpayers are harmed.
> Who is hurt by encouraging home ownership by black people?
If home ownership is a good thing, whoever would have otherwise bought those marginal houses is harmed.