Hello,
According to Macmillan dictionary, one of the definitions are:
1) liking and caring about someone very much, especially as a result of knowing them well or for a long time
example: I was very fond of my Uncle Jim
example: Are you fond of children?
So this can also be used with friends, but because it is used when you want to express a stronger emotion of like / caring, usually used with good friends.
Like you said, it can also be used for someone you like. This phrase is overall, a "endearing' than simply just "I like you".
You can also be "fond" of an artist, musician, etc's work. This would mean that you have an deep interest in and liking for the things they produce (their art, music, etc.)
The word "quite" acts to modify the word "fond" and put the strength of it on a level between "fond' and "very fond".
Fond can be used in multiple ways, so sorry if this is confusing. I think the main thing to note is that, like I said above, it is a more "endearing" term. (pure love and stronger connection, as opposed to a fling, or temporary infatuation).
Hope that helps!