Facial recognition and the facial difference community

Social media

32% of participants who had tried to upload one or more photos or videos of their face to social media had experienced difficulties. This happened on platforms including Facebook, Instagram, Tiktok, Youtube, Tumblr and Snapchat. While the reason for the problem was not always specified or clear, some attempted uploads had been given content warnings by the platform. For example:

• ‘i got a prompt that face wasnt a match’
• ‘I have had various photos rejected from upload to social media or given a “content warning”’
• ‘Mentioned the skin condition is not for kids under 18’
• ‘Says no face is recognized or that no expression is recognized (for social media filters activated by facial emotion)’
• ‘The platform doesn’t recognise my face on my own device’

E-gates for passport checking at customs

The majority of people who had used e-gates at customs had experienced difficulties, making e-gates one of the least inclusive technologies[1] measured in this survey. Among those who had tried, e-gates had never worked for 12%, only worked occasionally for 23% and worked ‘sometimes’ for another 23%.  Only 27% said that they always worked for them. Some participants also expressed the acute anxiety and embarrassment which these failures can cause in such a public place:

• ‘I always get nervous going though customs or border control because I know more than likely the egate is not going to work for me. In turn I then know that this means I have to speak to a border control officer who again will more than likely make a comment about my face or ask me what has happened to my face. These interactions have never been pleasant and often make me feel like they are judging or assessing me as to whether I am a threat. I particularly get anxious when this takes place when visiting countries where English is not spoken as it can be difficult to communicate. When travelling with friends or family I ask them to go after me so they can help support if necessary rather than being on the other side of the gate.’
• ‘Can be very frustrating and embarrassing, and causes great delays especially in off peak times where officers rely on the technology to get travellers through. The technology doesn’t give any reason as to why it doesn’t work but I can only assume it’s because of my significant facial difference.’
• ‘I had facial recognition fail twice, once at [US airport]. A border control agent had to get a coworker to come over and share her opinion on whether my face matched my ID.’
• ‘Fortunately I can’t afford to holiday much and so do so when my passport is relatively new and less so near it’s expiry as otherwise my face may have aged and deteriorated significantly for facial recognition since the passport photo was taken and I’d struggle with it identifying me as me.’
• ‘One eye is more “closed” than the other due to permanent swelling, so issues have occurred with face detection technology in the past such as e-gates and passport photo booths where I am instructed to open my eyes (and not wink!)’

[1] Measured according to the cumulative percentages of people indicating that it ‘never’, ‘occasionlly’ or ‘sometimes’ worked for them.

Accessing secure bank account apps

When trying to use bank account apps (or similar secure mobile apps), 14% said the technology never worked for them, 4% said it works ‘occasionally’ and 29% said it only worked ‘sometimes’. Only 21% told us that the technology worked every time. This poor performance caused very significant inconvenience and stress for some participants:

• ‘Locked out of bank account. Bank re set it for me but had to contact their head office to disable the face recognition function.’
• ‘As long as the app is connected to Face ID, it generally works smoothly. However, there was one bank app that required setting up biometric authentication. The bank’s facial recognition system couldn’t successfully recognize my face, which prevented me from completing the Face ID setup and therefore from making transfers’
• ‘Had one app (unable to remember name) that required a photo to set up a bank account that I was unable to use as it would not approve that I was human.’
• ‘I’ve given up trying and dread the day when that’s the only way I can bank independently and other verification means cease. I can’t use fingerprint recognition either.’

Photo apps (such as Google photos or Apple photos)

• After e-gates at passport control, photo apps were another poor performer[1] in our survey for consistent inclusion of the facial difference community. Those who had used an app such as Google photos or Apple photos to organise photographs into categories often had significant problems with the technology. Just 25% indicated that these apps worked every time for their face. While only 4% said that it never worked for photos of their face and 11% said that it worked occasionally,a huge 43% indicated that it only worked ‘sometimes’:
• ‘My Photos app thinks I am about five different people. Granted- I am also trans and so differences such as my haircut are part of it.’
• ‘Harder to compile more pics of me when I’m smiling or showing expression rather than neutral’
• ‘I use the Photos app on my iPhone, which can categorize images by people. The accuracy of categorizing my own photos is even higher than for recognizing others—probably because my birthmark makes me stand out. I can’t decide if I should be flattered or amused by this!’
• ‘I tried Google photos but had to give up. I feel excluded by my visible difference and asymmetry worsening over time.’

[1] See note 1 above.

Face ID / Unlock to access mobile phone

Those who had tried using facial recognition to unlock their mobile phones had mixed results. 8% found that it never worked for them, and 14% told us that it only worked occasionally. In contrast, 43% said that it ‘usually’ works for them and 19% had experienced no problems with it. Examples:

• ‘I recently got google pixel 9 and I was setting it up and tried to set up face ID to access my phone. I couldn’t complete that setting. It kept asking me to tilt my head up/chin up. I couldn’t succeed. The face id doesn’t recognize my small and receding jaw. So, I just use my thumb id and/or code.’
• ‘I use camouflage makeup up to cover my birthmarks but if I was to have no make up on my phone doesn’t recognise me/face’
• ‘iPhone 12 mini – usually works but had problems at different stages of my Healing’
• ‘My facial difference is episodic, sometimes the technology works for me and sometimes it doesn’t. iPhone’
• ‘I have to wear sunglasses due to my eyes and sometimes it does not work with them on.’

Remote conferencing software such as Microsoft Teams, Google Meet and Zoom

This technology uses facial recognition for features such as background blurring and applying filters. Very few of our participants had used filters, so the data on this was inconclusive. However, the accuracy of background blurring performed relatively well, with 25% of participants who had used it ‘extremely satisfied’ with the way it worked for them, and 42% fairly satisfied. Some still encountered problems sometimes:

• ‘Due to the affected area on my face, I need to regularly apply gauze. I use flesh-colored tape, but during meetings, my image sometimes shows facial indentations (the area with the gauze tape can’t be recognized as part of my face). software used:Teams & Zoom & Google Meet’
• ‘Zoom, because the brightness hurts and burns skin’