I'd suggest starting with the tin whistle. They're very cheap, very easy (you can learn the basics in a day or two, a week at most to get your first tune down), and any Irish session will either have someone willing to teach or someone who knows where you can get a teacher. The Session[1] is an excellent resource for it.
As for which whistle to buy, get one in the key of D. A Walton's little black should be about $6, one of their slightly better brass whistles around $11, and a Generation nickel D is about $12. If you really want to splurge Susato makes a very nice high D for $50 (their Kildare S series), but it's quite loud and more for if you are playing before an audience. The most expensive whistles around are probably Burke's, at $230 for a high D. So even if you end up collecting Burke whistles obsessively and getting one in every key he makes you'll still be out less money than buying some high-end guitars!
The one disadvantage is that they're diatonic instruments. That means you can play in one key, and only one key, without cross-fingering the whistle.
For tuning, I recommend either an electronic tuner or the program Flutini[2], assuming you get a tunable whistle (the cheap ones aren't and it probably won't matter much).
[1] https://thesession.org/
[2] http://www.novasession.org/Flutini/