Implementing Live OCR in React Native Apps

Optical Character Recognition (OCR) is a powerful feature that enables apps to extract text from images in real-time. We've implemented this in multiple news and insurance apps, and here's how you can do it too.

Use Cases

Insurance Claims: Scan policy numbers and claim forms
News Apps: Extract text from images for accessibility
Document Scanning: Convert physical documents to digital text
Business Cards: Automatically capture contact information

Technology Stack

We'll use:

React Native Vision Camera for camera access
ML Kit Text Recognition for OCR
React Native Worklets for performance

Setup

npm install react-native-vision-camera
npm install @react-native-ml-kit/text-recognition

iOS Configuration

Add to Info.plist:

<key>NSCameraUsageDescription</key>
<string>We need camera access for text scanning</string>

Implementation

Basic Camera Setup

import { Camera, useCameraDevice } from 'react-native-vision-camera';
import { useTextRecognition } from '@react-native-ml-kit/text-recognition';

function OCRScanner() {
  const device = useCameraDevice('back');
  const { scanText } = useTextRecognition();

  if (!device) return <LoadingView />;

  return <Camera style={StyleSheet.absoluteFill} device={device} isActive={true} />;
}

Frame Processing

const frameProcessor = useFrameProcessor((frame) => {
  'worklet';

  const result = scanText(frame);

  // Process recognized text
  if (result.text) {
    runOnJS(handleTextDetection)(result.text);
  }
}, []);

Text Extraction Logic

function extractPolicyNumber(text) {
  // Example: Extract policy numbers
  const policyRegex = /POL-\d{8}/g;
  const matches = text.match(policyRegex);
  return matches ? matches[0] : null;
}

Performance Optimization

1. Throttle Frame Processing

let lastProcessTime = 0;
const PROCESS_INTERVAL = 500; // ms

const frameProcessor = useFrameProcessor((frame) => {
  'worklet';

  const now = Date.now();
  if (now - lastProcessTime < PROCESS_INTERVAL) {
    return;
  }
  lastProcessTime = now;

  // Process frame
}, []);

2. Region of Interest

Instead of scanning the entire frame, focus on a specific region:

const roi = {
  x: width * 0.1,
  y: height * 0.3,
  width: width * 0.8,
  height: height * 0.4,
};

// Crop frame to ROI before scanning

Real-World Example: Insurance Card Scanner

Here's a complete example from one of our insurance apps:

function InsuranceCardScanner() {
  const [cardData, setCardData] = useState(null);
  const device = useCameraDevice('back');

  const processInsuranceCard = (text) => {
    const data = {
      policyNumber: extractPolicyNumber(text),
      memberID: extractMemberID(text),
      groupNumber: extractGroupNumber(text),
    };

    if (data.policyNumber && data.memberID) {
      setCardData(data);
      vibrate(); // Provide haptic feedback
    }
  };

  // ... camera and frame processor implementation
}

Error Handling

Always handle permissions and errors gracefully:

const { hasPermission } = useCameraPermission();

if (!hasPermission) {
  return <PermissionScreen />;
}

Testing Tips

Test with various lighting conditions
Use real insurance cards, not screenshots
Test with different phone models
Validate extracted data format

Production Considerations

Privacy: Don't store sensitive images
Validation: Always validate extracted data
Fallback: Provide manual input option
Performance: Monitor FPS and memory usage

Conclusion

Live OCR in React Native is powerful and achievable with the right tools. We've successfully implemented this in apps processing thousands of insurance claims daily.

Next Steps

Add barcode scanning
Implement document edge detection
Add multi-language support
Integrate with backend validation