yep~ here に means the reason and 寒さ is coldness, sometimes means the degree of coldness~　in general, we always use 体言（nouns）before に. and we only use 用言（verbs, adj...）in some specific expression. for example,　さびしいにはさびしいが、だいぶなれました。（Although lonely, i've almost accustomed to it.）

「寒さに震える」というと、物悲しい感じ（feeling sad) も演出します。 EX: 捨てられた子猫は、段ボールで寒さに震えていた。

My guess is that に in this case is a preposition particle, so it's probably 'shivering in the cold'.

Yes, you understood correctly. In the basic translation would be "to shiver with cold" Other examples with this に: 物音に驚く[ものおとにおどろく] 犬に怯える[いぬにおびえる] たくさんの人に会えたことに喜ぶ[よろこぶ] I hope this helps you.