Google brings AI-powered accessibility features for Android, Chrome, more

Google has announced a range of new accessibility updates set to roll out on the Android platform and Chrome web browser. This includes new AI-powered real-time Expressive Captions, an option for zooming into text without affecting the web page layout in Chrome, and more. Google is also bringing improvements to existing features such as TalkBack which can now answer questions related to images for which there is no alt text available.

Here are all the new features:

Gemini and TalkBack

Google is expanding Gemini integration within the TalkBack feature. While previously it was limited to offering descriptions of images that did not have alt text, the feature now also allows users to ask questions related to it. For example: If a friend texts you a photo of their new guitar, you can get a description and ask follow-up questions about the make and colour, or even what other things are there in the image.

Expressive Captions

The new Expressive Captions feature for Android uses AI to capture what someone says, and how they say it. For example, users can know when a sports commentary is calling out an “amaaazing shot” or when the video message is not “no” but “nooooo.” Additionally, users will also receive more labels for sounds, so they know when someone is whistling or clearing their throat.

The feature is rolling out in English in the US, UK, Canada, and Australia for devices which are running on Android 15 and above.

Improved speech recognition

Google’s project Euphonia which was launched in 2019, is expanding globally to make speech recognition more inclusive, especially for non-standard speech. Developers now have access to Google’s open-source tools through GitHub to build and train their models for diverse speech patterns.

Accessibility features on Chromebook

Chromebooks already offer several accessibility features such as supporting facial gestures for navigation, and a personalised Reading mode. Google is now expanding these features to College Board’s Bluebook testing app that allows students to take online tests.

Accessibility features in Chrome browser

Google’s Chrome web browser is getting two new features:

Screen reader for PDFs: Previously, Chrome browser did not allow readers to interact with a scanned PDF on their desktop, but from now on Chrome will automatically recognise these types of PDFs using Optical Character Recognition (OCR), and will allow users to highlight, copy, and search for text.

Page Zoom on Chrome: Google chrome on Android will now let users increase the size of the text without affecting the web-page layout. Users can customise how much they want to zoom in and then choose to apply the preference to all of the pages they visit. Users can access this feature by tapping the three-dot menu in the top-right corner of Chrome.

More

Apart from these new accessibility updates, Google has reportedly introduced a simple “docked bar” for its Gmail for Android. According to a report by 9To5Google, the quick reply interface which was introduced last year, is now being replaced by a tool bar docked to the bottom of the screen. This tool bar has Reply, Reply all, Forward, and emoji options.