Both sentences mean the same thing.

Both sentences are correct but have *different* meanings. The first one implies that making mistakes is necessary (for example, in order to improve), but that he is still afraid to do so. The second one means simply that he is afraid. Usually “afraid to” is followed by a positive verb, and usually is something under the person’s control: He is afraid to change jobs. In other words, changing jobs would be good for him but he is too afraid to do so.

Both are correct, but "He is afraid of making mistakes" is more natural. "To make mistakes" is an Infinitive, while "making mistakes" is a Gerund. The difference between the two forms is very subtle.