There are a lot of differences between the anime and the manga.
Strangely, Nami is the first character appearing in the anime. It's Luffy in the manga.
If I remember correctly we don’t know how Luffy got his scar under his eye in the anime. It’s probably because of censorship.
In the anime, Usopp is part of the crew when Luffy meets Gaimon but not in the manga.
In the anime we have the story of Baggy and the story of Kobby but it was just a cover story in the manga.
Ace is with the crew when they are travelling in the desert of Arabasta but not in the manga.
There are two Davy Back Fight in the anime but only one in the manga.
Ao Kiji doesn’t appear after the Davy Back Fight, as a result it’s not him who helps Tonjit. He appears in a filler arc when he helps some shipwrecked people.
There are a lot of (unfortunately bad) fillers. The G-8 arc is by far the best. The vice-admiral of the G-8 base makes an appearance during the Marineford arc. The commander Drake and the G-8 base appear in the fourth movie. Even if you don’t like the anime,this arc really deserves a shot.