Because it's MORONIC. Sending an entire fleet of ships deep into the New World, which is pirate territory, is essentially asking all the Yonkou to team up to push them out of their territory. Wiser to lure one of them out into THEIR territory, to take down one of them and thus break the balance of power that way.
When you're talking about the Yonkou you're not just talking about the Yonkou themselves, but their crews and all their unaffiliated allies to boot. It's not so simple as invading pirate territory.
Ok. I see your point. Then back on the subject.
Garp told Luffy that Sachibukai wes formed to balance the powers of the world. The moment I read that, I immediately thought that if Sachibukai was not formed, the WG will be not be able to stand against the 4 Yonkous. That without the Sachibukai, the Yonkous will not only be able to continue existing or maintain the status qou (as in what is happening currently) but also put an end to to their archrival, the WG. In short, WG fears these Yonkous so much that they needed to make some strong pirates, the Sachibukai, change their side.
As for your point that I'm talking about the crew and not the individual Younko themselves, all I can say is that so far only WB is said to amass a large number of subordinates. Shanks has a small crew. But he still managed to go touring the East Blue even though the Marines is guarding the first half of the Grandline tightly. What I'm trying to say here is that Shanks alone was enough to hinder the Marines from capturing the entire crew. Otherwise, his whole crew would have a hell of a journey leaving and returning to the New World.
And the fact that their forces are called four Younkous (referring to the individual leaders themselves), not four New World Great Powers or four New World Axis of Evils, is already enough to say that these Yonkous can most of the time be a powerful one-man army. In my opinion, this makes them more powerful than the admirals even if you'll just compare their individual selves.