Well, since Marine HQ is not far away from EL, and since Water7 also isn't far away, and Mugiwaras were only sailing for a day or something, the map can be something like this: (sry, fast work - low quality )
!
EL - Ennies Lobby
W7 - Water Seven
FT - Furorian Triangle
HQ - Marine Headquarters
ID - Impel Down
FI - Fishman island
The big red line on top is the RED LINE.
! So I really think that Fishman island, marine HQ and impel down are in a disconnected area, where you can only get throught gates of justice or going to fishman island. And the only secret passage to the new world can easily be inside Marine HQ (Assuming that all 3 can be connected together.)
And it's all a trap kinda thing