A "pair" tends to be used in English for clothing that covers parts of the body that we have two of and where each side looks like it could be separate, even if the clothing isn't actually separated into two things like shoes and gloves. Pants/trousers/shorts cover two legs or have two holes for legs to go through, so we use "pair" for them. Similarly glasses cover two eyes and have two lenses, so we use "pair" for them too. "Pair" is often used for other things which have two separate but connected parts, e.g. a pair of scissors (two blades), etc. We do not use "pair" for something like a shirt, because although it has two arms, its main function is to cover the torso. Also, we do not use "pair" for a skirt, because although it covers two legs, it doesn't look like it has two separate sides.

A pair of pants have two legs, so....