@Count:
Oda will make up whatever convenient plot circumstances are necessary for Big Mom to be vulnerable. If we are to learn about Pudding manipulating Big Mom's memories in, say, an Elbaf marriage flashback, there's all sorts of factors that could be at play to make it possible. Oda will do whatever he wants to make sure the plot flows how he wants to build-up stuff. Like having Luffy reel in a random poisonous fish and recklessly eat it so Reiju can be introduced and have an excuse to show off her poison sucking abilities. Or Pudding happening to be Bond villain monologuing when Sanji is visiting her room, as well as bragging to Luffy and Nami about killing Sanji at the wedding for no other reason besides sheer arrogance, because the plot deems it necessary for the Straw Hats to have ample knowledge to prepare a plan for the wedding. Or Kaido's introduction and meeting the Kid Alliance in general. Or Law's fruit being broken on paper, yet he still gets beaten by Doflamingo and Trebol because of inferior physical stats and without even switching their minds to confuse them. Or Usopp's face being scary enough to make Sugar faint just because.
He doesn't care about power levels, as he makes shortcuts for whatever makes the narrative more interesting as the priority. And Big Mom's a pretty arrogant and petty lady in general, so it's not hard for her family to manipulate. Which they've already been doing by hiding secrets about events that have happened in her own living quarters.
The idea doesn't even fit with where Oda seems to be going with BM and the Yonkou in general. Big Mom is portrayed as a cunning mafia-esque character. She should really be the one doing the manipulating. And part of the setup of the four Yonkou arc seems to be in portraying the Yonkou as near untouchable power houses. If all you need to control a Yonkou is a somewhat convenient devil fruit it throws the whole power balance of the one piece world into question.
About memory-manipulating Big Mom, it's really not THAT hard if you're no complete stranger to her. Just wait a little bit until the right moment and then do it. It may not be completely trivial, but also not too farfetched either.
The whole situation reminds me of Ace failing to kill the sleeping Whitebeard. It all seems doable until you realize that we're talking about a Yonkou.
I mean just look back at how much pain Reiju was in when her memories were being changed. Can you imagine BM just sitting there taking that? It doesn't make sense.