I agree with Brad, I know the same concern, always thinking about grammar to make sure I'm not making mistakes. In my view practicing is the only the get the confident flow :)

You CAN speak freely, but you are afraid of making mistakes and being embarrassed. Don't worry about that! Just speak as quickly as you can, and don't worry about grammar, not using the "perfect" word, etc. Just let it flow like water, and you will soon be able to correct yourself after you relax.

а в чём проблема с разговорной речью?