I just realized something.
Look at the map that Buggy's holding:
http://groups.msn.com/OnePieceMangav-3/volume3.msnw?action=ShowPhoto&PhotoID=516
A map like that is impossible on the Grand Line because it's impossible to tell exact location between islands without a compass. Furthermore, pinpointing the location of a sunken treasure is impossible without both a compass and a tool for measuring distance.
Which begs the question, why did Roger and crew leave the Grand Line? Buggy and Shanks look about 16, and they're both 38, so this flashback is close to the time when Roger reached Raftel and was executed.
Here's more evidence:
< <lookout: enemy="" ship="" to="" the="" north-northeast!!<br="">Also impossible on the Grand Line. The best you could do is using naval terms, such as Port bow, Starboard Bow, Port stern, Starboard stern, etc. Or clock directions in relation to the direction the ship is facing (1 O'clock, 12 O'clock, etc.).
So what were the Gold Pirates doing outside of the Grand Line? Obtaining information? It seems they already have capable crewmembers. Maybe Roger was meeting a contact, or perhaps stopping by somewhere to see someone…
But yeah...</lookout:>