The phrase "went to him" implies that you went one time; you checked, and you went - both actions were completed. The phrase "going to him" can imply an ongoing situation, such as going regularly to a doctor. The second construction can mean either you went once, and done - or more than once (an on-going situation).