Île Royale (Cape Breton) was ceded to the British under the terms of the Treaty of Paris and came under the colonial governance of mainland Nova Scotia. The following year, the Acadians who had been expelled were permitted to return. By 1783 and 1784, waves of Loyalist settlers began to arrive (more than 30,000). In 1784, Nova Scotia was divided into three separate colonies: Cape Breton Island, New Brunswick and mainland Nova Scotia. In 1820, the colony of Cape Breton was re-annexed back to mainland Nova Scotia. Cape Breton received a large influx of Highland Scots in the first half of the nineteenth century, with estimates of close to 50,000, as the result of the Highland Clearances.