The problem is my speaking skill

I have struggled against my poor ability to improve my speaking skill for three years. At first, I imagined I would feel the greatest difficulty in improving my listening skill. But, in reality, it's not listening skill but speaking skill. Particularly, I'm bad at speaking in the conversation. Alternative and
instant speaking action is very hard for me. I'm short of time to think in English, talk to myself in English in daily life. I guess I have to spend more "awkward time" permitted to use only English everyday. Maybe I shouldn't avoid feeling frustration aroused when I'm forced to use only English. There might not be fun way to become fluent in English. It's tough to keep self-confidence.

say skills with an s instead of skill

and you don't really need the word aroused after frustration. 

But you're doing a great job!