AI chatbots & agents · Fintech, SaaS & apps
WhatsApp chatbot for low-literacy users with speech and vision AI
4/5
Usability score in real-user testing
A GPT-4-based WhatsApp chatbot designed for Pakistan's 60 million low-literacy adults. Speech-to-text, text-to-speech and image recognition were combined into a modular pipeline optimised for low-end phones, so users interact entirely by voice and photos. Real-user testing scored 4 out of 5 for usability.
- Industry
- Fintech, SaaS & apps
- Market
- Pakistan
- Services
- WhatsApp chatbot, LLM development, Speech-to-text & text-to-speech, Image recognition
- Pillar
- Automate
60MPotential users: Pakistan's low-literacy adults
Voice + imageNo typing required
4/5Usability score
The brief
Most chatbots assume the user can read and type. For tens of millions of adults in Pakistan that assumption excludes them entirely. The goal was an assistant that works through voice and images alone.
What we built
- A WhatsApp bot on GPT-4 with speech-to-text for incoming voice notes and text-to-speech for replies.
- Image recognition so users can photograph a document, product or problem instead of describing it.
- A modular pipeline tuned for low-resource devices and patchy connections.
- Real-user testing rounds that fed back into the conversation design.
Outcome
Users completed tasks entirely by voice and photo, and the system scored 4/5 for usability in testing. The project showed how multimodal AI can open digital services to people text-based tools leave out.
