I think you can kayak on the Hudson river. I could not do this (not enough time) but I bet it might give you good memories.
Biking on central park is also great; but walk the park later.
Statue of Libery: expect to spend at least half day. Book the tickets today and be ahead of time because they have airport-like security.
Read the news (or a book) in Bryant Park. Didn't know the place existed until we got there.
If you come from Europe and have enough time, get a bus in Port Authority and spend one day in Washington DC to visit the Smithsonian Air and Space museum. I guess it's a 4h bus trip, in a greyhound bus (like in the movies). Some of them have wifi. Totally worth it.
Do not do this:
- Make a visit ground zero on the first place. I happened to be near and it spoiled mood for the whole day.
- If you have tickets for music show (phantom of the opera, wicked, etc.)... Be near the theater before the show. Near means walking distance, not 1 metro station away because this is the exact time Murphy's law will send you a long distance train instead a local one.
- Do not budget less than 1:30-2h from Port Authority (42th street) to La Guardia. The Metro needs at least 1:30h. Taxis seem to be faster until there's some kind of problem.
- United Nations: didn't find it worth the visit altough there was quite... hm... interesing people protesting on the front.
Hope this helps :-)
Another tip: rent bicycles and cycle around Manhattan and Brooklyn on the Waterfront Greenway. Not for the faint of heart since it's not entirely finished, so you're merging with traffic at some points.