Totally depends on what type of people you feel more comfortable around. Which will lead me into regional and demographic stereotypes, and no good will come of that.
I preferred the Escape out of Manhattan over the Breakaway out of NOLA in all aspects (ship, port, fellow passengers). But it seems like I got lucky and my commute from LGA to the port was pretty painless, so that helped.