Imagine prompting ChatGPT to depict a picture of refugees in Canada and it generates an image of Muslim families. When asked about their ethnic backgrounds, the responses list Syria, Afghanistan, Iran, and Iraq but do not mention Ukraine.
This scenario is generated in a 2024 study published in Knowledge Mobilization for Settlement, where researchers asked ChatGPT to generate an image of a refugee family in Canada and prompted it to consider ethnic background, integration barriers, and educational levels. Shockingly, it repeatedly depicted refugees as Muslim families, in particular from the Middle East and including women wearing hijabs. It also cited language as a primary integration barrier for these families. The prompts were tested multiple times to capture the AI’s understanding of diverse populations. The results were surprising, revealing a significant lack of nuanced understanding of diversity in the representations of refugee families.
This study also reveals ChatGPT’s disparities in job recommendations for newcomers between the Global North and the Global South. When asked about job prospects for individuals from five countries in the Global North and five from the Global South with the same amount of experience and using the same prompt, ChatGPT suggested lower-tier positions, such as administrative assistant, for those from the Global South, while recommending higher-tier roles, such as software developer, for applicants from the global north. This discrepancy extended to an average salary disparity of $20,000.
Generative AI uses predictive models to generate responses by drawing on historical data. These cases demonstrate how systemic biases and discrimination are ingrained in AI training. These biases are further compounded by language barriers, highlighting another critical area where AI fails to serve newcomers effectively. The study demonstrated how ChatGPT offers substantially less assistance to non-English speakers. Researchers tested it by prompting about opening a bank account in English and French, the two official languages of Canada. ChatGPT provided more detailed and interactive responses to the English prompt than to the French one. ChatGPT’s inconsistent responses across languages show a gap in linguistic support to newcomers in a bilingual country like Canada. For new immigrants who don’t speak English or French, such limitations could lead to inequitable access to critical information, underscoring the need for the development of robust multilingual AI tools.
As newcomers transition through the complex journey of settlement, they often face challenges along the way, including navigating job searches, adapting to a new culture, and facing language barriers in their country of residence. According to Statistics Canada, the unemployment rate for new immigrants living in Canada for less than five years is 12.6 per cent, which is significantly higher than the total unemployment rate of 6.4 per cent as of July 2024. To navigate their career path, newcomers and the settlement sector may turn to AI for immediate assistance. These findings underscore why Canada’s immigration sector must adopt a human-centred approach to AI in order to ensure that technology supports the integration of newcomers without reinforcing stereotypes and biases.
Immigration, Refugee, Citizenship Canada (IRCC) is increasingly applying generative AI and data analytics tools such as Chinook for faster and more effective service delivery in administrative tasks, such as summarizing profiles, triaging applications, and assigning officers based on the sensitivity of cases. Even though they do not involve AI in the final decision-making, any plan to expand the use of AI in providing assistance to immigration settlement requires careful consideration.
“AI tools should be responsive to the specific needs of newcomers that require human oversight in the loop. Every newcomer has a unique story, especially refugee cases, [which] are highly sensitive. AI often misses the nuanced understanding of cultural sensitivity and empathy essential for supporting the integration journeys of diverse newcomers,” says Darcy McCallum, CEO of Social Enterprise for Canada.
Echoing Darcy, Isar Nejadgholi, senior research scientist at the National Research Council of Canada, said, “Because of vulnerabilities and intersectionalities of demographics in this population, it’s very important to understand the specific needs and challenges diverse newcomers face in integration.”
For effective use of AI in immigration, the IRCC should work closely with the settlement sector and provincial governments while serving as the intermediary to newcomers. “Developing AI tools requires an agile, iterative and multidisciplinary approach. Technologists, the settlement sector, policymakers, and AI researchers need to collaborate starting from early stages of AI tools design and development. This collaboration ensures that these tools are reliable, user-centric, culturally sensitive and ethically aligned that meet both technical and user expectations. It requires long-term planning,” Isar added.
Another 2024 study, titled “Human-centred AI applications for Canada’s immigration settlement sector,” found that AI solutions benefit from a prototyping-first approach, allowing for early, iterative testing and refinement.
“Prototypes help identify design flaws early, saving time and resources in the long run,” Isar noted. Technologists play a pivotal role in developing these prototypes, which require hands-on testing and continuous refinement before widespread deployment. “Additionally, to align ethical standards with user needs, centralized government oversight is required that would establish data-sharing protocols, privacy safeguards, and regulatory frameworks,” Isar noted.
Isar further pointed out that AI cannot replace humans in immigration support; rather, this sector requires more humans to apply AI effectively in integrating immigrants, who are one of the biggest contributors to Canada’s economy. “Students should be involved in AI research, as they bring fresh perspectives,” Isar recommends.
One organization, Immigrant Networks, uses AI algorithms to pair newcomers with professional mentors based on shared interests, saving staff time while ensuring appropriate support. “A newcomer’s success depends on language, communication, digital, and networking skills. With the mentorship from Immigrant Networks, 70 per cent of mentees secured jobs within six months,” said Immigrant Network’s founder and CEO Nick Noorani. This approach underscores the importance of employing more humans to train immigrants where AI might fall short of meeting individual needs.
Promoting AI literacy among newcomers and the settlement sector is also essential. “Critical thinking and fact-checking are especially important when interacting with AI in a non-native language,” explained Isar. By understanding AI’s limitations, the literacy training may support newcomers and allow settlement staff to tailor prompts to achieve accurate, useful results.
Canada’s prosperity relies on the success of its immigrant communities. As Nick Noorani states, “Canada is built on immigration. If immigrants fail, Canada will fail.” With ethical design and a human-centred approach, AI can complement Canada’s immigration efforts, ensuring that each newcomer’s potential contributes to a stronger and more inclusive society.
The author is a graduate of the Max Bell School of Public Policy at McGill University.