I don't think it has anything to do with the quality of athletes. What is a good athlete btw? Is it a fast runner? Is it a big, powerful, muscular guy? In England and France, the kind of athletes that would play american football or ice hockey in USA play rugby. Look at some of the main players of the big nations, they're not exactly fantastic physical specimens.
At 2m/6'6 and 70 kgs/150lbs England's Peter Crouch looks like he's been doin heroin for the last 10 years. Argentina's Lionel Messi is built like a hobbit and isn't particularly fast without the ball. But once he has the ball in feet… No one can touch him.
What USA lacks and that the majority of the big nations has, it's the passion for the sport; everywhere in Europe and South America kids play football in the streets after school, in their parents' garage, it's part of the culture. There's nothing much more to say really; when a country has no interest in a sport, there's simply no way to be successful.