Second, it's handy to have 1st world medical care available immediately just in case. If a bad landing caused severe trauma, burns, or what-have-you, or if an emergency appendectomy was needed then care would be right at hand.
Third, speed of recovery was important, which meant big ol' helicopters, which means aircraft carriers.
Fourth, a worst case scenario survivable landing could involve the capsule landing significantly out of the drop zone, causing injury to the crew (partial chute failure, very hard landing), and sinking forcing the crew to evacuate the capsule. Potentially all without radio contact. In that scenario you'd want to be able to mount a robust search and rescue operation, which is just the sort of thing that a carrier battle group excels at.
With more experience, less geopolitical pressure, and more capable spacecraft we can ratchet that response way down, but it wasn't entirely unjustified at the time.
Edit: Case in point, the 2nd ever manned flight in the US (sub-orbital even) Liberty Bell 7 had the capsule sink and the astronaut, Gus Grissom, swimming without much assistance in the open ocean. If recovery had taken much longer (even by minutes) it's likely he would have drowned. That set the tone for the later recovery operations.