It's a pop-in reaction shot, not a perspective thing.
edit: http://www.mangareader.net/103-2571-9/one-piece/chapter-464.html
Sorta like this or
http://www.mangareader.net/103-2571-10/one-piece/chapter-464.htm
With multiple people instead.
If it was like that, all the guys would be in the front.
@Kaba:
Wait, did Oda just create an optical illusion? :wassat:
No, I think he just made a mistake. However, if he didn't, people would be talking about how Sanji is partially covered up by Luffy's almighty hat.