I would have to agree that the initial example about the walking and cycling is probably not the way per se is supposed to be used. It seems out of place they way it was used.
I like to think of per se as referring to something as having an intrinsic property or nature. So for instance you could say "The man didn't talk much. He wasn't rude per se, he just seemed to be a naturally quiet person" This is the way I have personally seen per se used most often. You can see that per se in this case is used to clarify the perception you are trying to portray to someone while speaking in order that they do not get they wrong idea. To break it down even further, per se is used to point out that the man is not rude by nature but instead he is just a quiet or shy person. In this way, the person being communicated to does not assume the opposite of what you are actually trying to portray.