Hello Ksenia, You can use either interchangeably in this context, but "pay back" requires an object (either indirect or direct). The correct phrasing would be: "I owe my brother some money and he says I have to pay HIM/IT back before the end of the month." "It" would refer to the money (direct object); "him" would refer to your brother (indirect object). I would also note that the tone of "pay up" is more demanding.

Yes if you change the syntax "Pay him back or pay back the amount i've borrowed" pay up= To give over the full monetary amount demanded. To pay on demand. pay back = To pay or return (what is owed as a debt). To repay (a person who is owed a debt). To give recompense to; reward: (How can we ever pay you back for what you've done?) To reciprocate; return: (Pay back a kindness.) To retaliate against or get revenge upon.