Safety is extremely important and cold water kills, a lot. American Whitewater does a great job keeping track and you can see their stats here: https://www.americanwhitewater.org/content/Accident/view/
No PFD is number 2 and I'm pretty sure a large portion of that flush drowning number can be partially attributed to cold.