I guess I'd wonder why we need to talk about a "right" to have sex in the first place. Generally libertarians presume a right to self determination which can only be legitimately infringed for certain kinds of reasons (e.g. harm to others). So the real question shouldn't be whether we have a right to have sex, but rather whether others would ever be justified in interfering if we chose to try to have sex.
As I discussed in this article, it's not completely unreasonable to think that we could be justified in preventing someone from doing something that they were choosing to do if we had very good reason to think that their choices would be catastrophic for them and we didn't have a reasonable chance to try to convince them without using force. For many young people, I think this is a somewhat fair description of things. I mean, paternalism is an unbecoming position between rational adults, but the word does come from the idea of treating adults like children. So if I had to use force to stop a 12 year old girl from getting into a sketchy guy's car after he seduced her with talk of how mature she is, and how he understood her so much better than her oppressive parents, I don't know that I'd think myself to be acting wrongly.
But as I also said, there comes a point where we have to let people make their own choices. If we say our piece, and someone still chooses to do something that we don't think is the right thing to do, we have to take into consideration the fact that others have the right to be free, and to make their own choices. I think it's a fine line with children, because it's hard to know whether they are actually making good choices, or are just being children. As G.A. Cohen pointed out, surely we would be hesitant to advocate freedom if we knew that it would systematically lead to people choosing things which would predictably have outcomes that would be catastrophic for them (for example, the freedom for a 5 year old to voluntarily put herself in a situation to be groped by an authority figure who she was too innocent to question). But as so many libertarians have pointed out, we would also surely be hesitant to advocate coercion if we knew that we might be hampering individuals' ability to live their lives in the way that they wanted to live them.
I guess my point is that talking about a "right to have sex" sort of misses the point. The question is whether we would ever be justified in interfering with young people's desire to have sex. And I think that while the answer is yes, we have to be careful about legitimizing unjust interfence in people's life choices.
Hopefully that helps somehow...
http://libertarian-left.blogspot.com/