>Free speech - yep, I can speak without forcing anyone else to do anything.
Great, except this means less than you think. Negative right (e.g. "freedom from X") free speech means that it's 100% legal for Comcast, Facebook, Twitter, Reddit, various Mastodon instances[0], that weird Trump-only social network, and whatever other platforms are out there to censor you because they're "not the government" and are merely refusing to publish something rather than banning it with the force of law. Free speech online is entirely a positive right (e.g. "freedom to have X"), because it all relies on private actors cooperating to publish your speech. In a negative-freedom world, the Internet is a censorship hellscape.
Positive freedom to speak would be something like net neutrality - i.e. common carrier rules. Nobody is obligated to provide Internet services, but if they do, they can't arbitrarily refuse service to certain people based purely on the user's speech. Right to shelter could work in a similar fashion - i.e. you don't have to build housing, but if you do, a certain percentage of it needs to be cheap housing. Or, instead of just mandating housing just exist, we just have the government spend a certain amount of tax outlays - that we already collect - on new home construction on the open market. We don't compel the workers to build housing with a gun, we compel them with money, which is a kind of compulsion that even the most free-market capitalist society readily accepts as legitimate.
Conversely, there are also negative-right conceptions of "housing as a human right", too. It could mean, for example, that zoning laws are unconstitutional. Right now there are plenty of government interventions in the housing market that are there to specifically prohibit certain kinds of housing arrangements - usually multi-family units - in certain areas. These arrangements impinge upon the market's ability to actually supply increased demand, and thus drive up the price of housing and price poorer people out of the market. Even if you entirely deny the existence of positive rights, think taxation is theft, and think Murray Rothbard had some good ideas... how is that not a violation of human rights?
[0] Mastodon is federated, but in practice most people use a large instance if they want to be on the Fediverse. It is possible for instance owners to refuse to federate with one another or block specific users from other instances.