Over the next several months progress was made. Roads were constructed and improved, frame houses built, and crops planted. Yet some of the colonists were unhappy with the hardships they encountered in the pioneer settlement - backbreaking labour, limited food supplies, cash shortages, and isolation from settled communities. Approximately one-third of the arrivals, mostly unmarried men and persons lacking agricultural skills, left the settlement for other parts of the province and the United States. Sickness swept over the colony, but only one death was reported. During the following winter and summer, some of the settlers found employment on the New Brunswick Railway.
Despite the loss of immigrants to the United States and elsewhere, the settlement grew steadily over the next several decades. In December 1874, the 111 Danish settlers were living in 26 houses. Horses and cows had been purchased, 14 barns had been raised, and 3,100 acres had been allocated. The following autumn another 26 Danes arrived without promise of government assistance. Settlers continued to trickle into the colony for several years, 127 in 1879, although some of them quickly left the area. A number of late arrivals had relatives living at New Denmark, which made the "settling in" process easier. By 1880, as a result of births, emigration, and migration, 355 people were living in New Denmark.