I always wear a baseball hat in Disney and rarely remove it. However, I definitely hold onto it for fast moving rides! 7DMT, Splash, any coaster, ToT. Unless you are prepared to lose a hat, either take it off or hold onto it on any questionable ride.