Project Objectives

Main Objective

To design and implement an integrated IoT Smart Glasses system with TinyML and Edge AI that provides real-time assistance for visually impaired individuals in Sri Lanka through facial recognition, navigation support, and Sinhala text reading capabilities.

Specific Objectives

  • Develop real-time facial recognition system for identifying known individuals with personalized voice feedback
  • Implement Sinhala speech-to-text and text-to-speech navigation aid with ultrasonic obstacle detection
  • Create Sinhala text recognition (OCR) with voice feedback for reading printed materials
  • Deploy lightweight AI models on resource-constrained devices (Raspberry Pi 5)
  • Ensure offline functionality without cloud dependency for privacy and accessibility
  • Provide affordable solution (~Rs. 70,000) compared to commercial alternatives

System Architecture

Mobile Application

Android/React Native app with voice commands, frame filtering, and audio feedback

Edge Server

Raspberry Pi 5 running Flask services for local AI processing

Smart Glasses

Pi Camera Module 2 mounted on lightweight frame with portable power

Flask Server Services & Functions

Face Recognition

FaceNet/MobileFaceNet models with TensorFlow Lite for identifying known individuals

Navigation System

HC-SR04 ultrasonic sensors for obstacle detection up to 1m range

OCR Service

Sinhala text recognition with Google ML Kit/Tesseract and TTS synthesis

Target Users

Primary Users

  • Visually impaired individuals in Sri Lanka (285M+ globally with visual impairment)
  • Blind users seeking independence in navigation and social interaction
  • Sinhala-speaking community requiring localized assistive technology
  • Users in rural areas with limited internet connectivity

Key Needs Addressed

  • Recognizing familiar people in social environments
  • Safe navigation with real-time obstacle detection
  • Reading printed Sinhala text independently
  • Affordable alternative to expensive commercial solutions (OrCam $3,500+)

Technical Specifications

Hardware Components

Processor

Raspberry Pi 5 (Broadcom BCM2712, 8GB RAM)

Camera

Pi Camera Module 2 (8MP Sony IMX219)

Power

27W USB-C Power Supply + Portable battery

Total Cost

~Rs. 70,000 ($200 USD)

Software & AI Models

  • TensorFlow Lite for edge AI optimization (float16 quantization)
  • Face recognition: 92% accuracy (controlled), 85% (real-world)
  • Sinhala OCR: 68% accuracy (controlled), 50% (real-world)
  • Speech recognition: 75-85% accuracy for Sinhala commands
  • Inference time: <1 second for face recognition
  • Offline operation: No internet required for core functionality

System Components

Facial Recognition Module

Real-time identification of known individuals using pretrained FaceNet model, with dynamic user enrollment via mobile app and voice commands. Provides whisper feedback for privacy.

Navigation System

Sinhala speech-to-text commands with ultrasonic obstacle detection (HC-SR04 sensor, 2cm-1m range). Real-time audio warnings and voice-guided assistance.

OCR & Document Recognition

Sinhala text recognition with 78% document classification accuracy (exam papers, newspapers, forms, notes, stories, word docs). TTS conversion for audio feedback.

Mobile Application

Android app for frame filtering, user interaction, and system configuration. Supports voice commands in Sinhala and English with accessibility features.

Edge Processing

Local Flask server on Raspberry Pi 5 running as systemd service. All AI processing happens on-device for privacy and offline functionality.

TinyML Optimization

Models compressed using TensorFlow Lite with quantization. Low latency (<1s), minimal power consumption, suitable for wearable devices.

Research Contributions

Technical Innovation

  • First integrated system combining face recognition, navigation, and OCR for Sinhala
  • Edge AI deployment on low-cost hardware (Raspberry Pi 5)
  • Offline functionality preserving user privacy
  • Real-time performance suitable for daily use

Social Impact

  • Affordable solution (10x cheaper than OrCam MyEye)
  • Localized for Sri Lankan visually impaired community
  • Enhanced independence and social confidence
  • Accessible without internet connectivity