Is this a guy you are interested in? :) I can't say for sure without knowing the whole history, but if he said (1) you're a hard person to match and then added (2) but you smell good, it could be that he felt bad for basically saying he is not interested and felt compelled to say something nice afterwards. But I really can't say for sure...

It really depends on the context, but I'm guessing that the guy means that it is difficult to match you up with someone (as in, for a date). There are other circumstances where this exact sentence could mean something completely different. For example, if a person requires an organ transplant but has a very specific genetic makeup, the doctor could say "he is a hard person to match (with a donor)." Or if a person is shopping for clothes, and she has a very specific skin tone, the salesperson might say "you're a hard person to match (with the color schemes of our available clothing)." So it really depends on the context in which this sentence was said.