There is actually no real difference difference. Even the English grammar/style guides cannot agree on this subject. (I really do apologise for English.) You use whichever one sounds/feels/works better in the sentence. It really doesn't matter - it's more an issue of stylistic choice then grammatical. Strunk and White's Manual of Style as well as Miriam Webster's Dictionary of English Usage will both back you (and me) up on this point. Some grammar and style manuals don't even bother to address the topic. I hope this helps. If you need further clarity please let me know, but the bottom line is - there's no difference at all. EDIT: June Casagrande has written multiple books on grammar (she worked as an editor for years) and here she is talking about the subject: https://player.fm/series/124995/192375969

To compare to is to point out or imply resemblances between objects regarded as essentially of a different order; To compare with is mainly to point out differences between objects regarded as essentially of the same order. Thus, life has been compared to a pilgrimage, to a drama, to a battle; Congress may be compared with the British Parliament. Paris has been compared to ancient Athens; it may be compared with modern London.