I've long been of the opinion that the Elder Stars are actually pretty upstanding fellows….who hate themselves and what they must do every single day.
I'd be surprised if they weren't adventurers who got caught up in the history of the world and in order to protect people from a nefarious force/their own history, do what they must.
They're afraid of what might happen if someone tries to 'break the wheel' because they know the futility of it.
That's where Luffy comes in because he'll subvert even their expectations.
NOT an easy job for Oda to write, but that's what we trust him to do.
In a way, similar to Thanos of the last Avengers movie: wanting to "keep balance", and therefore annihilate part of the population, but not with joy or anger - something like that?
It becomes an interesting (and somewhat) classical twist - of opposition of not good vs bad, rather different visions of seeing/considering what is (the greater) good.
Greg, do you think that at Elbaf Big Mom will recover her memory and become non-hostile to Luffy as an indirect consequence? :-o