Yeah, but if Ryouma has Brook's swordsmanship in combination with his own legendary skills, he's going to be a pretty amazing combination. Like two super swordsmen combined into one.
Now if Brook's DF lends him other abilities, Ryouma won't have those of course, but his swordsmanship will be insane. If Zoro takes him on and wins, especially if he takes him with only two swords, then it should pretty much settle any swordsmanship discussions between Zoro and Brooke as far as the reader base is concerned.
Although I honestly feel that Brooke's "singing" sword techniques and Zoro's Asura stance will more than signifigantly differentiate their fighting styles. There's plenty of punching and kicking from the other one peice crew members, and they are a long ways from equal, but they get by. I see no reason we can't have two swordsmen if their skills have signifigant visual differentiation.