"Wouldn't kissed" is not possible; you can only use "wouldn't kiss" (base verb after a modal verb). "Wouldn't kiss" can be an imagined present or future. We add "have" for imagined past, so we use "wouldn't have kissed". This also tells us that the listener kissed Mary!

Because its imaginary situation . Kissing mary is a true fact which happened in the past. The speaker wants to talk about a wish/desire that didn't happen in the past. The time line for this wish is in the past therefore we need to use would +have+P.P verb