It also bears considering that any given athlete (or any given team, as the case may be) cannot honestly achieve very much by boycotting unilaterally. That is to say, if the Commission itself were to boycott, that would obviously have huge, direct ramifications; beyond simply making a statement, it would have a very real impact. An athlete or team choosing to boycott, however, really wouldn't. It would be a nice show of support and solidarity, of course, which is always worth something, but beyond that it would be ultimately meaningless.
Of course, many people working together can make an impact where an individual could not possibly hope to, which would hold true here as well, if some suitably large group of athletes all chose to boycott. Still, that's why I talked about achieving an impact unilaterally; it is, for better or for worse, much harder to coordinate an effort across a population, especially one as diverse as 'Olympic athletes', than it is to take action purely on one's own.