The way that you have to think of it in the "wouldn't" example, is that it isn't something that is "on the table." The situation is hypothetical. Nobody asked you to live with him. You just know he would be a horrible roommate.
For the "don't" example, it is in the near future that you will have to live with this guy, and you don't want to.
"Won't" is not going to happen soon, but eventually will. When that time comes, you will not(won't) want to live with him.