It's a compliment, that can be applied for women, so no, no one will accuse you of sexual harassment.

Well, for starters, you wouldn't say "the balls", It would just be "You have balls" or "You've got balls". And, as the other poster has said, I would not use this for women, for obvious reasons. Lastly, consider when you are saying this. Is this a formal situation, or just a casual situation with friends? If you're with guy friends, it's fine, but otherwise I wouldn't use it at any other time.

I'll echo what others have said: it can be used with the article 'the' or without. "He had balls!" or "He had the balls to …." It's very informal. Don't try it on your boss! It works better if you're talking about a man, but it could be used about or to a female. I would not get upset if somebody said I had balls. It's just an expression, and I accept it as such. But I agree with David from the USA... don't risk it if your audience is female. Wait till you are an expert in judging situations and the culture you are in. You won't get done for sexual harassment, but it might just sound a bit out of place.

He had the balls to ask her out. (Use “the” if followed by an infinitive) He has balls. (Don’t use “the” if not) It’s a colloquial phrase that shouldn’t be used in a formal setting for either gender. Saying that a man “has balls” as a compliment could be considered derogatory towards women. Informally, it would be risky too if women are present.

I use it with woman, but only woman I know very well that have a sense of humor. They will give you a nice laugh, but otherwise, no...