How to OCR Text Extraction in Flutter Using Google ML Kit

Optical Character Recognition (OCR) is a powerful tool that enables apps to recognize and extract text from images. In this article, I'll walk you through how to implement OCR in a Flutter application using Google ML Kit. This can be particularly useful for creating document scanners, digitizing handwritten notes, or even converting printed text into digital form

In this article, we will learn to recognize text from camera in Flutter. Text Recognition:
This feature enables the identification of text from images or real-time camera input. The ML Kit Text Recognition API is capable of recognizing text in any Latin-based character set. For further details on Text Recognition, please refer to Google ML Kit's documentation. Using ML Kit for Text Recognition in Flutter For text recognition in Flutter, we are going to use the following Flutter plugins: google_ml_kit and camera. google_ml_kit: This plugin is used for text recognition and many other features. For more details, please check the official pub documentation. Using ML Kit for Text Recognition in Flutter For text recognition in Flutter, we are going to use the following Flutter plugins: google_ml_kit and camera. google_ml_kit: This plugin is used for text recognition and many other features. For more details, please check the official pub documentation. camera: This plugin is used for accessing the device cameras, and image streams, which we are going to use later. For more information, check the documentation here. Before we get started, please read about the platform-specific requirements: For google_ml_kit: Platform-specific requirements and Known issues. For the camera plugin: Platform-specific requirements and Known issues. We start by adding these plugins to our project's pubspec.yaml file:
dependencies: flutter: sdk: flutter google_ml_kit: ^0.19.0 camera: ^0.11.0+2


After adding the plugins, import them into your camera_screen.dart file like:

import 'package:google_ml_kit/google_ml_kit.dart'; import 'package:camera/camera.dart'; Now, define your camera controller and initialize it inside the initState method. Then, initialize the TextDetector like this:

Create the CameraController CameraController? _camera; // Initializing the TextDetector final textDetector = GoogleMlKit.vision.textDetector(); String recognizedText = ""; @override void initState() { super.initState(); _initializeCamera(); // for camera initialization } void _initializeCamera() async { // Get the list of cameras available on the device final cameras = await availableCameras(); _camera = CameraController(cameras[0], ResolutionPreset.low); // Initialize the CameraController await _camera?.initialize(); // Start streaming images from the camera for image processing and text recognition _camera?.startImageStream((CameraImage image) => _processCameraImage(image)); } Once the camera is initialized, we start streaming images from the device's camera using the startImageStream method. We pass the _processCameraImage method, which is defined below, as the parameter to it:

void _processCameraImage(CameraImage image) async { // Getting InputImage from CameraImage InputImage inputImage = getInputImage(image); // Processing the image to recognize text final RecognizedText recognizedText = await textDetector.processImage(inputImage); // Using the recognized text String allRecognizedText = ""; for (TextBlock block in recognizedText.blocks) { allRecognizedText += block.text + " "; } // Update the recognized text in your application state setState(() { recognizedText = allRecognizedText; }); } InputImage getInputImage(CameraImage image) { // Function to convert CameraImage to InputImage final WriteBuffer allBytes = WriteBuffer(); for (Plane plane in image.planes) { allBytes.putUint8List(plane.bytes); } final bytes = allBytes.done().buffer.asUint8List(); final Size imageSize = Size(image.width.toDouble(), image.height.toDouble()); final InputImageRotation imageRotation = InputImageRotationValue.fromRawValue(cameraDescription.sensorOrientation) ?? InputImageRotation.rotation0deg; final InputImageFormat inputImageFormat = InputImageFormatValue.fromRawValue(image.format.raw) ?? InputImageFormat.nv21; final planeData = image.planes.map( (Plane plane) { return InputImagePlaneMetadata( bytesPerRow: plane.bytesPerRow, height: plane.height, width: plane.width, ); }, ).toList(); final inputImageData = InputImageData( size: imageSize, imageRotation: imageRotation, inputImageFormat: inputImageFormat, planeData: planeData, ); return InputImage.fromBytes(bytes: bytes, inputImageData: inputImageData); } As you can see in the above code, textDetector.processImage(inputImage) is the main method for recognizing text from the image. This method requires an InputImage type image as a parameter. So first, we need to get the InputImage for the recognition from the CameraImage, and we achieve this using the getInputImage(image) method defined below:

InputImage getInputImage(CameraImage cameraImage) { final WriteBuffer allBytes = WriteBuffer(); for (Plane plane in cameraImage.planes) { allBytes.putUint8List(plane.bytes); } final bytes = allBytes.done().buffer.asUint8List(); final Size imageSize = Size(cameraImage.width.toDouble(), cameraImage.height.toDouble()); final InputImageRotation imageRotation = InputImageRotationMethods.fromRawValue(_camera!.description.sensorOrientation) ?? InputImageRotation.rotation0deg; final InputImageFormat inputImageFormat = InputImageFormatMethods.fromRawValue(cameraImage.format.raw) ?? InputImageFormat.nv21; final planeData = cameraImage.planes.map( (Plane plane) { return InputImagePlaneMetadata( bytesPerRow: plane.bytesPerRow, height: plane.height, width: plane.width, ); }, ).toList(); final inputImageData = InputImageData( size: imageSize, imageRotation: imageRotation, inputImageFormat: inputImageFormat, planeData: planeData, ); return InputImage.fromBytes(bytes: bytes, inputImageData: inputImageData); }
Now, create your CameraPreview widget to generate a preview for the given camera controller. Pass the previously defined CameraController like this:
import 'package:flutter/material.dart'; import 'package:camera/camera.dart'; class CameraPreviewWidget extends StatelessWidget { final CameraController cameraController; CameraPreviewWidget({required this.cameraController}); @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar(title: Text('Camera Preview')), body: Center( child: cameraController.value.isInitialized ? CameraPreview(cameraController) : CircularProgressIndicator(), // Show a loading indicator until the camera is initialized ), ); } } // Use this widget in your main app or wherever needed void main() async { WidgetsFlutterBinding.ensureInitialized(); final cameras = await availableCameras(); final cameraController = CameraController(cameras[0], ResolutionPreset.high); runApp(MaterialApp( home: CameraPreviewWidget(cameraController: cameraController), )); } And put the above CameraPreview widget inside your build method like any other widget and that's it. Below full source code


import 'package:flutter/material.dart'; import 'package:camera/camera.dart'; import 'package:google_ml_kit/google_ml_kit.dart'; // Create the CameraController CameraController? _camera; // Initializing the TextDetector final textDetector = GoogleMlKit.vision.textDetector(); String recognizedText = ""; void main() async { WidgetsFlutterBinding.ensureInitialized(); final cameras = await availableCameras(); final cameraController = CameraController(cameras[0], ResolutionPreset.high); runApp(MaterialApp( home: CameraPreviewWidget(cameraController: cameraController), )); } class CameraPreviewWidget extends StatefulWidget { final CameraController cameraController; CameraPreviewWidget({required this.cameraController}); @override _CameraPreviewWidgetState createState() => _CameraPreviewWidgetState(); } class _CameraPreviewWidgetState extends State { @override void initState() { super.initState(); _initializeCamera(); } void _initializeCamera() async { // Get the list of cameras available on the device final cameras = await availableCameras(); _camera = CameraController(cameras[0], ResolutionPreset.low); // Initialize the CameraController await _camera?.initialize(); // Start streaming images from the camera for image processing and text recognition _camera?.startImageStream((CameraImage image) => _processCameraImage(image)); } void _processCameraImage(CameraImage image) async { // Getting InputImage from CameraImage InputImage inputImage = getInputImage(image); // Processing the image to recognize text final RecognizedText recognizedText = await textDetector.processImage(inputImage); // Using the recognized text String allRecognizedText = ""; for (TextBlock block in recognizedText.blocks) { allRecognizedText += block.text + " "; } // Update the recognized text in your application state setState(() { recognizedText = allRecognizedText; }); } InputImage getInputImage(CameraImage cameraImage) { final WriteBuffer allBytes = WriteBuffer(); for (Plane plane in cameraImage.planes) { allBytes.putUint8List(plane.bytes); } final bytes = allBytes.done().buffer.asUint8List(); final Size imageSize = Size(cameraImage.width.toDouble(), cameraImage.height.toDouble()); final InputImageRotation imageRotation = InputImageRotationMethods.fromRawValue(_camera!.description.sensorOrientation) ?? InputImageRotation.rotation0deg; final InputImageFormat inputImageFormat = InputImageFormatMethods.fromRawValue(cameraImage.format.raw) ?? InputImageFormat.nv21; final planeData = cameraImage.planes.map( (Plane plane) { return InputImagePlaneMetadata( bytesPerRow: plane.bytesPerRow, height: plane.height, width: plane.width, ); }, ).toList(); final inputImageData = InputImageData( size: imageSize, imageRotation: imageRotation, inputImageFormat: inputImageFormat, planeData: planeData, ); return InputImage.fromBytes(bytes: bytes, inputImageData: inputImageData); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar(title: Text('Camera Preview')), body: Center( child: _camera?.value.isInitialized == true ? CameraPreview(widget.cameraController) : CircularProgressIndicator(), // Show a loading indicator until the camera is initialized ), ); } }


Thanks for reading.  Happy Coding! 🎉