It's a smile that is a bit scary. Perhaps if someone smiles, but they don't mean it because they are going to do something bad.

a sinister smile refers to a smile that is cunning, or sly, basically meaning an evil smile. So when someone gives a sinister smile it means they have something evil in mind, as in planned or they have already done something evil and might be waiting to see the results of what they have done.Think of a movie, when the evil guy plots something evil, the smile he would give then. If anything is unclear, please feel free to contact me.

http://images.mentalfloss.com/sites/default/files/styles/article_640x430/public/sinister-smiles-primary.png