Great question, Peter! This is not about grammar, but rather pragmatics. “Would you come to our party tonight?” is not really a question in the conditional — that's its form, but it is really an invitation. Therefore, the answer should not be a mere conditional statement, but an actual acceptance (or polite refusal). If you answered merely, “yes, I would”, you’ve misinterpreted the question. You would? Under what circumstances? That would be like if I asked you “do you know what time it is?” and you said “Yes, I do.” What’s the context, I mean, what kind of English course is this for?