a) It's rather normal that dealing with thirty or so letters at your own pace is easier than dealing with about gazillion sounds at their pace. So don't worry about it.
b) Seriously, do not worry about it. As you said yourself, you're now being nervous about being nervous, and that's a feedback loop that reinforces itself. Often worrying less helps to improve the results (I practically do not have any other reason for learning Spanish other than that I enjoy it, and I'm not even going to tell you how quickly I learn when it's just a fun thing that I don't have to do and could give up at any moment).
c) Failing is a part of learning, and it's an important part. Trying to learn without failing is trying to learn without learning. So just accept it. Even better, try to use it. From my experience, telling somebody how nervous I am about speaking in Spanish is actually a pretty good conversation topic. When I was in Spain and wanted to talk people, I had the same problem with shyness and anxiety, and I dealt with it by asking people for the way even though I had a smartphone with maps in my backpack, asking people in the street "¿Dónde puedo comprar una toalla?" because I wanted to go to the beach, or spending 4 euros in a sweets shop just for asking "¿Cuáles son los mejores?". I felt silly for doing that, but it worked. And despite learning a lot during the time, I still returned feeling frustrated about how I could have learnt more.
d) As others have said, practice. As I understand it, you being a native English speaker should make it rather easy to find somebody for language exchange, locally or here, and besides practicing that should also help you to see how others struggle as well, and that it's only as big deal as you make it inside your head.