Hong Kong is unlikely to see a major exodus to the U.K. under its new visa program for millions of residents who hold British National (Overseas) passports, with the impact partially offset by new arrivals from mainland China, according to a top government adviser.

The actual number of people departing Hong Kong will be "far, far smaller” than current estimates that as many as 1 million people will leave for the U.K. over the next five years, Bernard Chan, a financier and the convener of Hong Kong leader Carrie Lam’s advisory Executive Council, said on Monday. While the territory has seen people flee during past crises, each time "either they return or there’s new immigrants coming in,” he said.

The difference between the exodus of people from Hong Kong in the 1990s and the one unfolding today, Chan said, is that now there is a large pool of talented mainlanders who can take their place. He said those leaving now are a mix of people who don’t trust China, who want a better education for their children or who "think they may not be as competitive” in a changing economy that increasingly values skilled mainlanders who can help international firms win business in China.