I don't eat this stuff for my own dietary reasons, but even I know Soy and Quinoa and old fashioned rice -n- beans and hummus (aka mushed up chickpeas) and plain old peanut butter sandwiches are cheap complete protein sources.
If you want fat intake, which is good as long as your carb intake is low, disgusting and unhealthy as it is, factory produced vegetable oil can fry nearly nutritionally vacant potatoes. Dip them in corn syrup ketchup for extra vegetables or at least empty calories.
If you are diagnosed allergic to soy like my son, or think the stuff on the above list is "gross" life gets complicated and expensive rapidly if you subsist entirely on the weirdest of vegetarian supplements and prepared meals and prepared snacks.
Also I'm locked out "submitting too fast" but:
A non paleo diet of nothing but rice will probably kill you in the long term. Lets run with it anyway.
Depending on how badly you got screwed and how many middlemen stand in the way and the size of packaging, 50 cents per pound is (was?) not totally out of line for rice. So at $4/day you can afford 8 pounds of uncooked rice. After cooking it, it'll be a whole lot more than 8 pounds due to water adsorption. But the nutrient content of the uncooked rice (not much washes away) is about 3 cal/gram somewhat pessimistically aka about 1200 calories per pound of uncooked rice. And you'll be consuming 8 of those pounds... per day. so thats 9600 calories of cooked rice per day.
Given an intake of 9600 calories per day, I checked out some basal metabolic rate online calculators. I got about 9000 calories per day BMR for a 30 year old male 6 ft tall weighing a mere 1300 pounds. For out metric readers, 1300 pounds is almost exactly 1/2 the mass of my new commuter car. So pretty freaking huge.
On a very long scale, given a mere $4/day to live on (I spend $6/day long term average and eat like a king, so WTF about that) and only eat rice and somehow don't die of a deficiency or diabetes, your weight will eventually stabilize at a mere 1300 pounds given that monetary intake in the form of 100% rice.
So yeah, vegetarians probably are not going to starve to death on a mere $4/day unless they're really screwing up.
(my apologies, I checked Amazon Prime and its hard to get rice shipped to my door for less than $1/pound, although it seems like mere years ago I saw 50 pound sacks for $25 at the grocery store... At any rate, $4/day is enough to be obscenely morbidly obese)