Fact portion of the post: Episode 460, around 8:15
Roger (Ace's father) says in a flashback to Garp (Luffy's grandfather) in regards to Ace's mother:
"The mother of the child, a woman related to you, will be executed for sure!"
Implication portion: Ace's mother would be too young to be Garp's sister, so at best, she would be his daughter.
If you're Luffy, your grandfather's ("dead-years-before-I-was-born") daughter could only be your aunt.
Your aunt's son is your cousin.
So, at closest, the relation implied by this statement in is that saying Ace and Luffy are cousins. (It could be more distant if the relation between Garp and Rouge are more distant.)