That's just the comical, "regular" nobility we've only seen so far. There could be tenryuubito that could be legitimately dangerous, and not just because they have huge amounts of money and political ties.
Key words. In whatever context they have actually been shown on panel so far, the World Nobles come across as spoiled, ugly, weak, pathetic. The one from the Fishman Island flashback was so clueless he nearly got himself killed. Those are the representatives of the "Noble lifestyle" Doflamingo cherished so.
I guess it's more about the power aspect, like he himself says.
The World Nobles are the only people who seemingly can do anything they want. Brings back that idea Luffy has about what it means to be Pirate King.
But the way it is presented here, Doflamingo didn't miss his status because of any plans to put its power to use. Its due to completely shallow reasons; "Why are we living in this dump?" "a guy disrespected me!" "Wheres the slaves?!"
He misses the luxury.
@Cyan:
The Celestial Dragons' political influence alone is dangerous. Doflamingo with his underworld connections and a proper Celestial Dragon's clout? He'd be fully untouchable.
Doflamingo was driven into crime specifically because he lost his CD clout. If he'd gotten everything he wanted and thus never vowed to destroy the world, he wouldn't have set up an underground network to destroy the world.
And even if he had, I doubt Kaido would care about whether Doffy was a full blown or a pseudo Noble.