INVENTIVE SUPPORTER FOR SIGHTLESS PEOPLE USING DEEP LEARNING
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
GORTERIA JOURNAL ISSN: 0017-2294
INVENTIVE SUPPORTER FOR SIGHTLESS
PEOPLE USING DEEP LEARNING
Senthil Kumar M1,Chidhambararajan
B2, Rajakumar M3,
RaghavanandhanN4, Sivaram S5
1
Associate Professor/CSE, SRM Valliammai
Engineering College,
2
Principal, SRM Valliammai Engineering College
345
Department of Computer Science and
Engineering, SRM Valliammai Engineering
College.
Abstract: Blindness is a lack of vision that cannot be corrected with glasses or contact lens. Visually impaired
people face lots of problems including navigation from one place to another. The aim of this system(project) is to
produce an Artificial Human Companion. In this system we introduced a eyeglass (SMART VISION) which
includes webcam, used to capture a live video (Frames) of the present location of the blind people and it is used
to identify the object. Here we plan to use the ultrasonic sensor, mems sensor and moisture sensor which are
fixed to a shoe (SMART SHOE) to identify the obstacles, pitfalls, path holes, slippery water, and water stagnant.
These two modules help us in detecting various obstacles and convey it as a voice over an earphone. As a part to
this system, we introduced a communicating model called SMART TOUCH which is used by Blind – Dumb
people to communicate with normal people. This overall outcome of this proposed system is to build a user –
friendly device which help these people (Blind and Dumb) to interact with the environment.
Keywords:Deep Learning, IoT, Android Studio, Smart Vision, Smart Shoe, Smart Touch
1. INTRODUCTION
As of these days, most commonly occurring defects affecting human is blindness and
voicelessness. These defects are occurred by birth or due to some uncertain conditions.
These defects can be cured when they are at the starting stage. As low vision can be tackled
using eyeglasses and continuous speech training can improve the people speech flow. But
there is no complete remedy and cure for these defects who are completely blind and
speechless. There are some existing works which helps those challenged people. But that
too can't fulfil their needs. So, we have proposed a system, which is going to be discussed
in this paper. Our system is the best and user-friendly artificial guardian for visually
impaired people. This system consists of three modules which helps these people to interact
much better with the community. Smart vision which helps the blind people to detect the
object and alert them which are at top level. Smart shoe which is sensor-based module
helps to detect the obstacles which are present at the ground level. For the dumb-blind
people we introduced a communicative module which is smart touch that uses braille dots
further which is converted to speech to communicate with other people. Finally, our system
with these three modules will be a complete shield for the blind-dumb people to feel like a
common people.
2. RELATED WORKS
O. K. Toffa and M. Mignotte are the authors discussed about a visual feature approach for
visually impaired people in image sonification. Through the earphones, visual data will be
transferred by an image sonification (conversion of data into sound) system [9]. Md.
Siddiqur Rahman Tanveer, M. M. A. Hashem and Md. Kowsar Hossain described about
the features of Assistant by android application for both visually challenged people and
blind tracker. In that proposed system, they used ultra – sonic sensor to detect the obstacles,
google map for navigation and with the help of Bluetooth module, the identified obstacles
are informed to the visually challenged people through voice [14].
VOLUME 34, ISSUE 4 - 2021 Page No: 122
GORTERIA JOURNAL ISSN: 0017-2294
Nallapaneni Manoj Kumar, Neeraj Kumar Singh are V. K. Peddiny are the authors
explained about the challenges, features, and applications of existing Smart Glass [5]. Rohit
Agarwal, Nikhil Ladha, and Mohit Agarwal are the authors, explained about Smart glass by
using ultra – sonic sensor. In their proposed system, the ultrasonic sensors were detected
the objects and then convey those obstacles as beep sound to visually impaired people [3].
Jyun – You Lin, Chi – Lin Chiang, Meng – Jin Wu, Chih – Chiung Yao, Ming – Chiao
Chen are the authors given a brief note on Smart glass using deep learning for visually
challenged people. In this proposed system, initially the objects are being detected and then
the detected objects are being classified. Then, by using TTS (Text – to – Speech)
command the classified objects are converted into voice for visually challenged people
[10].
Chao – Hsien Lee and Mao – Qun Gen are the authors described about the smart shoe for
visually impaired people by integrate with various sensors. This system produced
approximately 80% of accuracy [7]. Cuong Pham, Nguyen Ngoc Diep and Tu Minh
Phuong are the authors narrated about e – shoes for human activity identification. Here, the
authors approached the CNN (Convolutional Neural Networks) algorithms, and the
accuracy of this proposed system is achieved above 93% [3]. Vanitha kunta, Charitha
Tuniki and U. Sairam are the authors discussed about modern blind stick for the blind. In
this proposed system, they used Internet of Things technology and integrating various
sensors into the blind stick [11].Anuja Gote, Tejas Kulkarni, Shikha Jha and Sunny Gupta
are the authors narrated a literature about braille tab for visually challenged people to
communicate with other people. In this proposed system, dot matrix and 3D shaft
mechanism (printed) are used for communication [13]. Jevri Tri Ardiansah and Yaushisa
Okazaki are the authors proposed a system for visually impaired people i.e., conversion of
braille language into speech through the application [5]. Sameia Zaman, M. Abid Abrar, M.
Muntasir Hassan and A. N. M. Nafiul Islam are the authors explained about braille
language which will be useful for both visually challenged and deaf people by
Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) [8].
3. MATERIALS AND METHODS
We have created a simplified and better system for the usage of visually impaired people.
This system will be much useful than the existing and previous works done by others. Our
proposed system is a complete tool kit which consists of three models. These three models
will be working in various level of height for the betterment of visually impaired people.
Figure 1. represents the architecture of this system. The entire architecture of the system is
listed below,
Figure 1. Architecture of this proposed system
VOLUME 34, ISSUE 4 - 2021 Page No: 123
GORTERIA JOURNAL ISSN: 0017-2294
The various models are listed as,
Smart Vision
Smart Shoe
Smart Touch
3.1. Smart Vision
The first module SMART VISION which is used for object detection at top level.
Here, we get the object as image / video frames through high quality camera. The obtain
objects images are processed using MATLAB. At the same time, we have pre – processed
trained data sets for comparing these objects. The MATLAB uses CNN algorithm for object
classification and we also use an extra feature called transfer learning approach which uses
previously trained datasets for comparing with similar datasets like them. It also reduces the
training time and training datasets. Finally, the processed object is detected, and their characters
(name) are displayed using TTS (Text – To – Speech) library. The displayed object name is
converted into speech. The voice is conveyed through earphones which are used by visually
impaired people. Figure 2. – represents the block diagram of Smart Vision.
Figure 2. Block Diagram of Smart Vision
3.2. Smart Shoe
The module SMART SHOE which developed for special reason which is used to
detect the objects at the ground level. It is an additional guardian for the Blind people. The
Shoe which is incorporated with several sensors (Ultra sonic sensor, Moisture sensor, MEMS
sensor), battery, Arduino uno, Amplifier, Bluetooth module, and Power play unit. The Ultra
sonic sensors are used to detect the obstacles at the range of 500 cm. The Moisture Sensor used
to detect the water (Slippery water, Water Stagnant) at the range of 600 atmospheric pressure.
MEMS sensor used to detect the slanting height, path holes, pit fallsusing three-dimension co –
ordinates (3D). Those detected objects (obstacles) are amplified using amplifier and Bluetooth
module which is connected is used to convey the information to Smart phone application. The
detected obstacles are conveyed as a voice through smart phones (earphones). Use a power
supply module and a battery for the working of SMART SHOE. Figure 3. – represents the
block diagram of Smart Shoe.
VOLUME 34, ISSUE 4 - 2021 Page No: 124
GORTERIA JOURNAL ISSN: 0017-2294
Figure 3. Block Diagram of Smart Shoe
3.3. Smart Touch
Our final module is smart touch which is very much helpful for the blind-
dumb people to communicate with the normal people. This is an android application-
based system which gets the input by clicking the button which has braille six segment
dots. As they make an input of creating letter/word/sentence, it performs the operation
using braille pattern matching. After matching their character, the sentence formation is
done, and the displayed text is converted to speech through phone. We use a special
operation for spacing (swipe left) and speech (swipe right). The spacing is done
between words and after the formation of sentence swipe right option is performed for
speech. Figure 4. represents the block diagram of Smart Touch.
Figure 4. Block Diagram of Smart Touch
4. IMPLEMENTATION AND RESULTS
Experimental working of the above discussed paper is implemented(recorded) below.
4.1. Smart Vision
Here the camera is used to getting the image / video frame and obtained image
frames (objects) are processed using MATLAB functions. Those objects are segmented
VOLUME 34, ISSUE 4 - 2021 Page No: 125
GORTERIA JOURNAL ISSN: 0017-2294
layered classified recognized. Comparing with the pre – processed system the character
name of the object is matched and displayed. Then, the displayed object name is converted
into voice and convey through earphones. This module is used to detect the object which are
at the top level (eye – level). Figure 5. represents the prototype of Smart Vision.
Figure 5. Prototype of Smart Vision
4.2. Smart Shoe
In this module, our experimental setup of shoe consists of Ultra sonic sensor,
MEMS sensor, Moisture sensor, Amplifier, Bluetooth module, Power supply module and
Battery have been these sensors are used to defect the obstacles, path holes, slippery water.
These information are alerted using Bluetooth module to the Smart phone (Arduino
Bluetooth app) as voice. This module is very useful to detect the obstacles at the ground
level. Figure 6.& Figure 7. represents prototype and implementation of Smart Shoe.
Figure 6. Prototype of Smart Touch
VOLUME 34, ISSUE 4 - 2021 Page No: 126
GORTERIA JOURNAL ISSN: 0017-2294
The output which are detected using various sensors are listed below:
Figure 7. Implementation of Smart Touch
4.3. Smart Touch
This is a simple and very useful for Blind – Dumb people, which is an android
application. The first image shows the Braille six segment dots which is used for Braille
pattern (Character matching).
Using Braille dot language, we able to form words / sentence.
Figure 8. Six Segment Braille Dots
Figure 8. represents Six segment Braille dots. We are also implemented a advanced option
for easy usage which is swipe left and swipe right. Swipe left is used for spacing between
words and Swipe right is used to convey the sentence as voice. Figure 9. represents the
VOLUME 34, ISSUE 4 - 2021 Page No: 127
GORTERIA JOURNAL ISSN: 0017-2294
implementation of Smart Touch.
Tables
For Example:
Figure 9. Implementation of Smart Touch
This application brings communication much easier between Blind – Dumb and
normalpeople. Thus, our overall system will / can be more useful and helpful for visually
impaired people.
5. CONCLUSION
We have presented a better system for the benefits of the blind-dumb people. This
systemwhich has the major works in smart vision, for additional safety we introduced smart
shoe for the use of visually impaired people. Not being able to communicate with others
may cause depressed state for the dumb people so we have built an app for communication.
The output of our implementation reached a great height and produced a good accuracy
rate. As these three things will be a good artificial human companion for these people.
Further as future works, we try to build the app as a smart watch which will be more
convenient for these people.
REFERENCES
[1] V. Sharma, Y. Tomar, “SMART SHOE FOR WOMEN SAFETY”, IEEE, (2020), pp.
1 – 4.
[2] A. Rohit, L. Nikhil, A. Mohit, “LOW COST ULTRASONIC SMART GLASSES FOR
BLIND”, Vol. 07, Issue 17,(2017), pp. 210 – 214.
[3] P. Cuong, N. Nguyen, P. Tu Minh, “e – Shoes: Smart Shoes for Unobtrusive Human
Activity Recognition”, 9th International Conference on Knowledge and Systems
VOLUME 34, ISSUE 4 Engineering
- 2021 (KSE), Vol. 06, Issue 17,(2017), pp. 269 – 275. Page No: 128
GORTERIA JOURNAL ISSN: 0017-2294
[4] N. Manoj Kumar, S. Neeraj Kumar, K. Peddiny, “Wearable Smart Glass: Features,
Applications, Current Progress and Challenges”, IEEE,(2018), pp. 577 – 583.
[5] A. Jevri Tri, O. Yasuhisa, “The Design and Prototyping of Braille to Speech
Application as a Self – Learning Support Media for Visually Impaired Person”,
IEEE, Vol. 08, Issue 20, (2020), pp. 224 – 228.
[6] D. Tirthankar, B. Anupam, “A Speech Enabled Indian Language Text to Braille
Transliteration System”,(2020), pp. 201 – 212.
[7] C. Hsien Lee, M. Qun Gen, “Preliminary Study on Multi – Sensors Wearable Smart
Shoes using the Internet of Things Techniques”, IEEE 7th Global Conference on
Consumer Electronics (GCCE), Vol. 07, Issue 18,(2018), pp. 361 – 362.
[8] Z. Sameia, M. A. Abrar, M. Muntasir Hassan, A. N. M. Nafiul Islam, “A Recurrent
Neural Network Approach to Image Captioning in Braille for Blind – Deaf People”,
IEEE International Conference on Signal Processing, Information, Communication
& Systems (SPICSCON), Vol. 04, Issue 19,(2019), pp. 49 – 54.
[9] O. K. Toffa, M. Mignotte, “A Hierarchical Visual Feature – Based Approach For
Image Sonification”, IEEE TRANSACTIONS ON MULTIMEDIA,(2020), pp. 1 – 10.
[10] L. Jyun – You, C. Chi – Lin, W. Meng – Jin, Y. Chih – Chiung, C. Ming – Chiao,
“Smart Glasses Application System for Visually Impaired People Based on Deep
Learning”, Indo – Taiwan 2nd International Conference on Computing, Analytics
and Networks (Indo – Taiwan ICAN2020), Vol. 08, Issue 20,(2020), pp. 202 – 207.
[11] K. Vanitha, T. Charitha, U. Sairam, “Multi – Functional Blind Stick for Visually
Impaired People”, Fifth International Conference on Communication and
Electronics Systems (ICCES), Vol. 01, Issue 20,(2020), pp. 895 – 899.
[12] Sangeeta Kumari, A. Akshay, A. Pallavi, Y. Bhamare, Z. Naikwadi, “Enhanced
Brailley Display Use of OCR and Solenoid to Improve Text to Braille Conversion”,
International Conference for Emerging Technology (INCET), Vol. 08, Issue
20,(2020), pp. 1 – 5.
[13] G. Anuja, K. Tejas, Shikha Jha, G. Sunny, “A Review of Literature on Braille Tab
and the Underlying Technology”, Fifth International Conference on Devices,
Circuits and Systems (ICDCS),(2020), pp. 333 – 336.
[14] T. Siddiqur Rahman, A. Hashem, H. Kowsar, “Android Assistant EyeMate for Blind
and Blind Tracker”, IEEE,(2013), pp. 1 – 6.
VOLUME 34, ISSUE 4 - 2021 Page No: 129
You can also read
