During this absence the Scottish court was dominated by James uncle, Robert Stewart, 1st Duke of Albany. The Duke resolutely refused to pay the ransom demanded by the English for the return of James.