The depression of 1857 and the American Civil War dealt hard blows to shipbuilding and trading by sea. English steamboats made of iron drove wooden sailboats from the seas. Decline in shipping and shipbuilding led to a decline in the town's population. In 1880, the town had 2,213 people; in 1900, it had 1,808.

Opening of mines and quarries brought people to town. Blue Hill had three periods when copper was mined. The big rush was from 1879 to 1883. William Stewart, who may have discovered Nevada's Comstock Lode, came to town and found William Darling mining copper in a small way and talking about silver in a big way. Stewart declared, "Maine ought to take first rank as a producer of bullion." Boston and New York papers published his declaration. Developers come to town and local people caught the mining fever. It was a case of mine in every pasture, a strike on every knoll.

By 1879, six hundred people were digging, and by February 1880, a $25,000,000 value was placed on stock that had been issued. A mining exchange and the Gold and Copper Exchange (a hotel) opened in the village. Excited sightseers came to town to view the mines.

But soon the mining bubble burst and the mines closed. Most non-natives fled town, though a few stayed and became farmers. Mills and shaft houses were torn down for lumber, camps were moved to the village and set up along the Mill Brook. Colonel William Darling, the town's foremost mining producer, went mad. People became mildly ashamed of their past enthusiasm.

In 1917 and 1918, the American Smelting and Refining Company began mining as war led to a rise in the value of copper. Semi - processed ore was hauled to the steamboat wharf in horse - drawn carts where it was wheeled aboard sailing vessels. In the 1960's and 1970's Canadian firms reopened the mine for brief periods.