Object Recognition Process Image

The science of object recognition  continues to evolve combining the science of artificial intelligence, computer vision and cloud based networks providing  greater recognition accuracy of objects and entities. Application of this science to apps have also become increasingly available on mobile devices for consumers with some targeted to specific needs and individuals with disabilities.  The OT’s with Apps post entitled “Object Recognition App Feature Match” presents a variety of such object recognition apps identified for individuals with learning or intellectual disabilities. While there are numerous object recognition apps currently available,  identifying the best app for AT clients serviced has been a need to determine potential application for individuals evaluated for their AT needs. Specifically, identifying potential apps for individuals with learning or intellectual disability was important as possible AT solutions. Recognition apps for individuals with visual impairment have been available for several years and have been a population offering recognition apps specific to their needs however may have features not considered optimal for individuals with learning or intellectual challenges. Available recognition apps as “Be My Eyes app”, “Braigo app”, “Be Spectacular app” or Aipoly app target specific users and innately present features that are barriers to others (i.e. require typing user names, password, require registration,  email accounts or subscription based services for operation).

While the Object Recognition App Feature Match document targets potential apps, trial of the apps appeared to be the best method to begin to determine functional application of these apps. Employing basic object recognition trial appeared to be the first step to identification of potential apps for individuals with learning or intellectual disability. Although just a start, application to clients needs would require further evaluation of their specific skills, specific task, environment and tools using evaluation frameworks as SETT or HAAT framework.  Understanding the tools and tasks as simply trialed with the object recognition apps and basic objects or products begins to provide information on which apps might be most accurate, have easy operation and product information helpful for individuals with intellectual or learning challenges.

Object recognition apps chosen to be used for this informal trial were identified with the following features:

• Ease of use
• Seeking high accuracy (however understanding that 100% accuracy may not be attainable. When risks of safety are involved, use of a recognition app is not recommended such as for medication, medical recommendations, etc.)
• No login required
• Speaks recognition information without turning on text to speech or screen reading options as VO (for sighted individuals)
• Information spoken is easily understood
• Presentation of features of the app are intuitive and can be easily memorized or used for operational and functional competency of use
• App has been undated recently, supporting current iOS systems to ensure operation
• No subscription fees are required for operation of the app.
• Apps with only bar code scanning were not selected or trialed due to their limited application

Object recognition apps that fit the majority of the above features were employed. Informal trials of the apps were performed using an iPad Mini 4 in a consistent location and time. Lighting remained consistent during all trials. The objects were singly presented for recognition for best object recognition with 1-2 trials allowed as needed for recognition. Implementing recognition required positioning the object or bar code in the camera field of the mobile device and pressing the camera button with Wi-Fi services available.

Results of object recognition accuracy with the following items is as follows:

Results of the informal trial showed the following apps for the objects trialed to be most accurate (100% of 7 items trialed):

• Third Eye app
• CamFind app
• TapTap See* app

Apps with good accuracy (86%) included:

• Seeing AI app
• IDentifi app

Comments: The objects chosen for trial were randomly chosen and trial completed within a home where some lighting and object manipulation (positioning) is capable. Objects used did not represent a cross section of goods or items individuals might need to recognize. Apps providing object recognition rather than bar code scanning were chosen in attempts at gaining information on labels from images only. Apps found most accurate included the Third Eye, CamFind and TapTap See app. Note that TapTap See and CamFind apps have the same developers, thus assuming they both use the same data base for recognition. Although TapTap See states they require VO to be turned on, the app did recognize and speak aloud object information without turning VO on.

While Seeing AI app resulted in slightly reduced recognition accuracy, it provides the most recognition options (short text, product, scene, documents, people, images) within the app. The multiple options in Seeing AI may be great for some individuals with basic reading skills and problem-solving skills, however they may present barriers to some individuals with intellectual challenges. The CamFind app, a recognition app not targeted specifically for individuals with disabilities, appeared to be a great app with universal design features. Created by CamFind (using CloudSight API), the developer also of TapTap See app, this app might be the easiest to apply given their long standing dedicated work with image recognition.

Conclusion: As for apps recommended for object recommendations, of course it depends. If seeking an app for an individual with a learning or intellectual disability who is able to problem solve and has some reading skill, I would consider Seeing AI app due to the multiple features it offers for object and text recognition. For individuals with intellectual disability, I would explore application of Third Eye app and CamFind or TapTap See app as long a Voice Over options do not cause a barrier. For individuals who are working just with products with bar codes and has functional competencies of application apps, I might suggest using Digit Eyes app (iOS, 9.99), which just identifies products with bar codes. As always choice of app or tools is dependent on information using evaluation information as gathered using a SETT, HAAT or other individual, skill, task, tool, environment evaluation method. Identifying safety issues with use of the recognition apps for individual with intellectual disabilities would also need to be an important consideration with the application.

As for devices, I would recommend a current iOS device offering fast processing, current iOS with best camera (highest MP offered) to support best picture quality.

Have you used object recognition apps with any of your clients? What has been your experience?

Carol, OT’s with Apps and Technology

References:

Leynse Harpold, Carol. “Object Recognition App Feature Match.”, 2017. Source: https://drive.google.com/file/d/1-_ajvAXsmYg4bHcYn0vphPvoFnxAuGsO/view?usp=sharing

Rouse, Margaret. “What Is Object Recognition? – Definition from WhatIs.com.” WhatIs.com, Feb. 2015, whatis.techtarget.com/definition/object-recognition.

Zabala, Joy. “SETT Documents.” Joyzabala, http://www.joyzabala.com/Documents.html.