Use sign-to-text to translate American Sign Language (ASL) into English text in real time

Sign-to-text is a feature built into Gboard that puts sign language on par with spoken languages. For many deaf and hard of hearing individuals whose primary language is American Sign Language (ASL), typing in English may not be their preferred means of communication. With sign-to-text, you can give Gboard permission to use your phone's camera to translate American Sign Language (ASL) into English text in real time.

You can use sign-to-text in any app that accepts Gboard input. Starting with American Sign Language (ASL), it lets you sign naturally to write messages, take notes, search the web, or prompt Gemini. Meanwhile, you can use Live Transcribe for spontaneous face-to-face chats like ordering a coffee or talking with colleagues. Learn more about Live Transcribe.

To use sign-to-text, you must:

• Have a Google Pixel 11 device.
• Be using Android 8 or later.
• Install the latest version of the Gboard app and set keyboard language to en-US or en-CA.

Important: Before you can use sign-to-text, you must grant Gboard permission to access your phone's camera. If Gboard doesn't have permission, you won't be able to use the feature.

1. In the Gboard toolbar, tap Sign-to-text .
   • Alternatively, tap Menu Sign-to-text .
     • To add sign-to-text to the toolbar:
       1. Tap Edit .
       2. Touch and hold, then drag Sign-to-text to the toolbar.
2. Review the details in the onboarding screen shown.
3. Tap Next.
   • You can review educational videos with clear instructions on how to position your front camera to capture your face, upper body, and hands effectively.

1. On your device, open any app where you can write, like Google Messages. Make sure Gboard is installed and the selected keyboard for the app.
2. In the Gboard toolbar, tap Sign-to-text .
   • This opens your camera and interactive preview window. You’ll be asked to grant camera permissions.
   • Alternatively, tap Menu  Sign-to-text .
3. Position your phone so the front camera has a clear view of your hands, upper body, and face.
4. Begin signing.
   • As you sign, Gboard actively captures your signs.
     • Landmarks will appear over your face, hands, and upper body in the camera.
     • Translated words will stream in real-time in the text field.
   • To indicate that you're finished signing, move your hands out of the camera's view.
5. To switch back to the keyboard, tap Sign-to-text .

Tips:

• The pose coordinates, your videos, and translations aren’t stored.
• As you sign, you can select suggested words from the toolbar.
• To minimize the keyboard, tap Minimize .

You can refine and edit your text easily through a combination of typing, signing, or Gboard’s suggestion strip. To update a word, you can:

• Tap Gboard's standard keyboard button to manually edit.
• Use the prediction strip to choose alternative word suggestions.
• Use writing tools to automatically format, punctuate, and polish your translated sentences. Learn more about writing tools.

With Gboard’s seamless interface, you can transition between signing and typing without losing your place.

Your privacy is our utmost priority. The sign-to-text feature protects your personal information.

With your camera, the sign-to-text feature observes body movements, like face, hands, and upper body, in real-time. It sends only a 2D representation of key points of the movements, not your video, to the server and translates them into written English.

These coordinates are processed using secure servers to perform real-time translation. Your inputs are never saved, shared, or used to train models.

If a system health warning occurs, Gboard immediately shows a brief notification and asks you to close other apps.