@Qualms:
If you think about the places they've been to since Chopper, the folk on those islands have either seen weirder things, were in masks, didn't care or were freaks themselves.
Not really. True, there´s been lot of freaks in One Piece already but Brook as walking skeleton sticks out because in comparison to every other freakish thing, he´s been revived from the dead while all the other freaks practically are normally living beings.
I don't think an island of mermen where they rarely see the outside world is going to creep them out that much, or the Marine headquarters, and let's not get started on the possibly weirder-than-Brook things in the New World.
Again, Brook is something you don´t see every day. The term Fishman Island itself already shows how much more Brook is a rare specimen in comparison to them. He is unique while there´s one island full of fishmen, which is even common knowledge in the OP world.
Besides, the sole fact about how much everybody freaked out while seeing Brook should prove his uniqueness status. True Luffy freaks out for everything and the fraidy cats are scared of many things, but guys like Zoro and Sanji were also "wtf" when they saw him.
But if that´s all to speculative then I guess going with practic examples does the trick. In chapter 459 Franky said the following:
"You know how people will react when they see you looking like that, don't you? You're a talking skeleton. You'll creep others out. You can't possibly make friends as such a hideous thing!! Even if you COULD escape this "Sea of Evil," you know what fate awaits you. And after decades of despair, unable to escape, you said you were "so lonely you could die." Well, if I were you… I would have just kicked the bucket already!!! Your life is so freakin' weird already… And then you act all gentlemanly, and try to hide the real mystery behind this…!! ...What is this "promise with your crew" that is so important... You would suffer all this to stay alive?!"
Which makes the statement so strong is that it comes from Franky, a character who himself is one of the most freakish things that ever appeared in One Piece. This statement proves for a clear difference in the freaky level between Brook and everything else.