For me, I find myself use "I'd prefer ..." with nouns And "I'd rather" with actions. But I don't know the actual differences.

Generally speaking the meaning's the same, but as Peachey pointed out the grammar's different: I would prefer +N [or] + 'to V'; I would rather + V I would prefer to come later I would prefer the red car I would rather come later Fun fact: you can use "I would rather" (but not "I would prefer") in an exaggerated way to express how strongly you feel, eg. "I would rather die than leave her!"; "I would rather lose all my money than give that dishonest shopkeeper one single dollar!" ;-)

Remember that "prefer" is a verb, and "rather" is an adverb. If you write "I'd rather..." then you must follow with a verb in the base form (this follows the "would [verb]" pattern).