I didn't get that feeling from his flashback, he looked motivated and enthusiastic so I don't think he was just playing video-games instead of practicing or something like that.
It's possible he just wasn't a gifted enough student and needed more time to perfect his quirk or a different teacher/method of teaching, better suited to him.
Society has expectations for people to reach a certain level of skill in a field of their choice by the time they're a certain age, like by the time they finish high school or college, but that doesn't work for everybody. Of course those who achieve that skill level when they're young will probably be the ones leading their field later on but by definition not everyone can be among the best.