Advertisement

Teleophthalmology: Evaluation of Phone-based Visual Acuity in a Pediatric Population

Published:August 10, 2020DOI:https://doi.org/10.1016/j.ajo.2020.08.007

      Highlights

      • Teleophthalmology is part of our practices due to coronavirus disease (COVID-19).
      • Visual acuity can be checked at home using paper charts or devices.
      • GoCheck Kids has a modest correlation of visual acuity compared to the chart screen.
      • GoCheck Kids has a fair correlation of visual acuity compared to HOTV-ATS.

      Purpose

      With the recent rise of teleophthalmology due to coronavirus disease, health care needs accurate and reliable methods of checking visual acuity remotely. The visual acuity as measured by the GoCheck Kids application was compared with that of the Amblyopia Treatment Study (ATS) and the authors' clinic protocol.

      Design

      This was a prospective, comparison of visual acuity assessment methods.

      Methods

      Established patients (3-18 years of age) in the practice of a single pediatric ophthalmologist were eligible. Visual acuity was measured 1) by GoCheck Kids mobile application, by the patient's family member; 2) by HOTV-ATS, by study personnel; and 3) by regular clinic protocol, by an ophthalmic technician. To assess agreement between measurement of acuity, intraclass correlations with 95% confidence intervals (CI) were computed.

      Results

      A total of 53 children participated. The mean differences between GoCheck Kids and HOTV-ATS acuities (0.094) were significantly different (P < .001). The intraclass correlation coefficient (ICC) was 0.55 (95% CI: 0.40-0.68). The mean differences between GoCheck Kids and chart acuities (0.010) were not significantly different (P = .319; ICC: 0.59; 95% CI: 0.45-0.71). The mean differences between HOTV-ATS and chart acuities (0.084) were significantly different (P < .001; ICC: 0.66; 95% CI: 0.53-0.76). The percentages of eyes with visual acuity measured by GoCheck Kids within 1 line of the HOTV-ATS and chart acuity were 65.3% and 86.7%, respectively.

      Conclusions

      GoCheck Kids as checked by a family member provided a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, although most were within 1 line.
      Telemedicine has been thrust into ophthalmology practices by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and resulting coronavirus disease-2019 (COVID-19). In March, 2020, the American Academy of Ophthalmology had recommended significant reduction of the number of clinic visits and recommended only seeing patients for urgent and emergent reasons.
      Important coronavirus updates for ophthalmologists. American Academy of Ophthalmology.
      These guidelines were reinforced by social distancing instructions and stay-at-home orders in many states like Virginia.
      Northam R. S. Governor Northam Issues Statewide Stay at Home Order. Office of the Governor.
      With this new landscape of medicine, accurate ways of measuring vision at our patient's homes are needed. There are multiple methods of checking visual acuity in children in clinic: Lea symbols, Kay pictures, HOTV, Snellen letters, Sloan letters, Early Treatment of Diabetic Retinopathy Study (ETDRS), and tumbling “E”; all with and without crowding bars.
      • Chen S.I.
      • Chandna A.
      • Norcia A.M.
      • Pettet M.
      • Stone D.
      The repeatability of best corrected acuity in normal and amblyopic children 4 to 12 years of age.
      • Holmes J.M.
      • Beck R.W.
      • Repka M.X.
      • et al.
      The amblyopia treatment study visual acuity testing protocol.
      • O'Boyle C.
      • Chen S.I.
      • Little J.A.
      Crowded letter and crowded picture logMAR acuity in children with amblyopia: a quantitative comparison.
      • Becker R.
      • Hübsch S.
      • Gräf M.H.
      • Kaufmann H.
      Examination of young children with Lea symbols.
      • Anstice N.S.
      • Jacobs R.J.
      • Simkin S.K.
      • Thomson M.
      • Thompson B.
      • Collins A.V.
      Do picture-based charts overestimate visual acuity? Comparison of Kay Pictures, Lea Symbols, HOTV and Keeler logMAR charts with Sloan letters in adults and children.
      • Huurneman B.
      • Boonstra F.N.
      Assessment of near visual acuity in 0-13 year olds with normal and low vision: a systematic review.
      • Inal A.
      • Ocak O.B.
      • Aygit E.D.
      • et al.
      Comparison of visual acuity measurements via three different methods in preschool children: Lea symbols, crowded Lea symbols, Snellen E chart.
      • Ruttum M.S.
      • Dahlgren M.
      Comparison of the HOTV and Lea symbols visual acuity tests in patients with amblyopia.
      Lea symbols, HOTV, and letter acuity have been recommended for use in children.
      • Huurneman B.
      • Boonstra F.N.
      Assessment of near visual acuity in 0-13 year olds with normal and low vision: a systematic review.
      ,
      • Moke P.S.
      • Turpin A.H.
      • Beck R.W.
      • et al.
      Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol.
      Visual Acuity is also mass-tested in vision screening programs with the goal of detecting children with visual impairment as recommended by the American Academy of Pediatrics and the American Association for Pediatric Ophthalmology and Strabismus.
      • Donahue S.P.
      • Baker C.N.
      on behalf or Committee on Practice and Ambulatory Medicine, American Academy of Pediatrics
      Procedures for the evaluation of the visual system by pediatricians.
      ,
      • Donahue S.P.
      • Arthur B.
      • Neely D.E.
      • et al.
      Guidelines for automated preschool vision screening: a 10-year, evidence-based update.
      In those screening programs, many different eye charts are used, and visual acuity is measured by different people: teachers, nurses, certified medical assistants, public health care workers, students, and other lay persons.
      • Schmidt P.
      • Maguire M.
      • Dobson V.
      • et al.
      Comparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision In Preschoolers Study.
      • Sabri K.
      • Thornley P.
      • Waltho D.
      • et al.
      Assessing accuracy of non-eye care professionals as trainee vision screeners for children.
      • Sabri K.
      • Easterbrook B.
      • Khosla N.
      • Davis C.
      • Farrokhyar F.
      Paediatric vision screening by non-healthcare volunteers: evidence based practices.
      • Hashemi H.
      • Yekta A.
      • Jafarzadehpur E.
      • et al.
      Sensitivity and specificity of preschool vision screening in Iran.
      • Sharma A.
      • Li L.
      • Song Y.
      • et al.
      Strategies to improve the accuracy of vision measurement by teachers in rural Chinese secondary schoolchildren: Xichang Pediatric Refractive Error Study (X-PRES) report no. 6.
      • Marmamula S.
      • Khanna R.C.
      • Mettla A.L.
      • et al.
      Agreement and diagnostic accuracy of vision screening in children by teachers, community eye-health workers and vision technicians.
      • Adhikari S.
      • Shrestha U.
      Validation of performance of certified medical assistants in preschool vision screening examination.
      • Saxena R.
      • Vashist P.
      • Tandon R.
      • Pandey R.M.
      • Bhardawaj A.
      • Menon V.
      Accuracy of visual assessment by school teachers in school eye screening program in Delhi.
      Mobile and computer applications have been developed to improve and increase the consistency of checking visual acuity in children's screening programs. Some of those applications have been investigated through peer-reviewed publications.
      • Moke P.S.
      • Turpin A.H.
      • Beck R.W.
      • et al.
      Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol.
      ,
      • Zhao L.
      • Stinnett S.S.
      • Prakalapakorn S.G.
      Visual acuity assessment and vision screening using a novel smartphone application.
      • Bastawrous A.
      • Rono H.K.
      • Livingstone I.A.
      • et al.
      Development and validation of a smartphone-based visual acuity test (peek acuity) for clinical practice and community-based fieldwork.
      • de Venecia B.
      • Bradfield Y.
      • Trane R.M.
      • Bareiro A.
      • Scalamogna M.
      Validation of peek acuity application in pediatric screening programs in Paraguay.
      • Yamada T.
      • Hatt S.R.
      • Leske D.A.
      • et al.
      A new computer-based pediatric vision-screening test.
      • Zhang Z.T.
      • Zhang S.C.
      • Huang X.G.
      • Liang L.Y.
      A pilot trial of the iPad tablet computer as a portable device for visual acuity testing.
      • Trivedi R.H.
      • Wilson M.E.
      • Peterseim M.M.
      • Cole K.B.
      • Teed R.G.
      A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children.
      The GoCheck Kids application is used for vision screening through photo screening and checking visual acuity. That application has been studied for photo screening and detecting amblyopia risk factors but not for checking visual acuity.
      • Peterseim M.M.W.
      • Rhodes R.S.
      • Patel R.N.
      • et al.
      Effectiveness of the GoCheck Kids vision screener in detecting amblyopia risk factors.
      • Law M.X.
      • Pimentel M.F.
      • Oldenburg C.E.
      • de Alba Campomanes A.G.
      Positive predictive value and screening performance of GoCheck Kids in a primary care university clinic.
      • Walker M.
      • Duvall A.
      • Daniels M.
      • et al.
      Effectiveness of the iPhone GoCheck Kids smartphone vision screener in detecting amblyopia risk factors.
      The present study was approved by the Institutional Review Board in December 2018 with the goal of exploring a family member checking a child's visual acuity.

      Methods

      This prospective study was approved by the Virginia Commonwealth University Institutional Review Board. Informed consent was obtained from parents of the subjects, and assent to study them was obtained from subjects >7 years of age. From April 2019 to February 2020, established patients in the practice of a single-pediatric ophthalmologist (E.S.) between the ages of 3 and 18 were eligible for the study. This population was selected because the goal of this study was to evaluate the visual acuity of children with known and treated refractive errors (if required).
      Visual acuity was measured in three ways: 1) by the GoCheck Kids mobile application (Scottsdale, Arizona); 2) by HOTV-Amblyopia Treatment Study protocol (HOTV-ATS) on M&S Technologies (Niles, Illinois); and 3) by regular clinic protocol. To mimic visual acuity tested at home, visual acuity was checked by the parent, guardian, or sibling (designated the “tester”) of the patient with GoCheck Kids. A medical student or the pediatric ophthalmologist checked the vision for HOTV-ATS, and an ophthalmic technician checked vision for the regular clinic protocol. The iPhone 7 plus (Apple, Cupertino, California) used for the study was provided to the study by GoCheck Kids. Visual acuity from the GoCheck Kids application was measured with the following protocol. A trained medical student or an ophthalmologist taught the parent or sibling of the patient how to use the application. GoCheck Kids is used to check threshold visual acuity by displaying a test letter in crowding bars (HOTV for children 6 years and younger and ETDRS letters for children 7 years and older) below 4 randomized letters in crowding bars (Figure 1). The child subject indicates which direction the tester should tilt the device until the test letter matches the letter in the randomized row. The child then notifies the tester of a correct placement of the letter, and the tester tilts the phone downward to lock in the answer (Video 1 [Supplemental Material available at www.ajo.com]). The protocol is based on software used in the video game EyeSpy.
      • Trivedi R.H.
      • Wilson M.E.
      • Peterseim M.M.
      • Cole K.B.
      • Teed R.G.
      A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children.
      The test is performed at 5 feet, using a previously measured piece of rope held between the child and the tester to ensure proper distance. The right eye was tested before the left eye, and the nontested eye was occluded. Testers were instructed not to look at the screen to blind them to the test. No feedback was given to the parents during the test to mimic testing environment at home. The GoCheck Kids application can report vision from 20/20 (0.0 logMAR) to 20/63 (0.5 logMAR). HOTV-ATS visual acuity was measured at 16 feet (the length of the examination room from chair to visual acuity system) using the M&S system running the Amblyopia Treatment Study protocol. HOTV with crowding bars were used for all participants. The right eye was tested before the left eye, while the nontested eye was occluded. Visual acuity was also measured at 16 feet with the M&S system, using the clinic's protocol, testing with the most challenging acuity that the child could reliably perform: Sloan > HOTV line > HOTV crowding bar (“chart acuity”). Also recorded was the relationship of the tester to the child, the child's sex and race, the glasses prescription, and the child's ophthalmic diagnoses. The presence of vision disease was defined as a diagnosis that affected the child's visual acuity (eg, a child with juvenile idiopathic arthritis with normal examination results was classified as not having a vision disease, and a child with amblyopia was classified as having a vision disease). Data for this study were analyzed using SAS/STAT version 9.4 software (SAS, Cary, North Carolina) for Windows (Microsoft, Redmond, Washington). Descriptive statistics were computed for demographic variables and for GoCheck Kids acuity and HOTV-ATS acuity and the paired difference between them in LogMAR units. To assess the significance between acuities, generalized estimating equations were used to test whether the mean differences were equal to zero. To assess agreement between measures of acuity, intraclass correlations with 95% confidence intervals (CI) were computed. In addition, the percentage of children with visual acuity as measured by GoCheck Kids that were within 1 and 2 lines of HOTV-ATS and of regular clinic protocol were calculated. Study data were managed using Research Electronic Data Capture (REDCap) hosted by Virginia Commonwealth University.
      • Harris P.A.
      • Taylor R.
      • Thielke R.
      • Payne J.
      • Gonzalez N.
      • Conde J.G.
      Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support.
      ,
      • Harris P.A.
      • Taylor R.
      • Minor B.L.
      • et al.
      The REDCap consortium: Building an international community of software platform partners.
      REDCap is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources.
      Figure thumbnail gr1
      Figure 1User interface for the GoCheck Kids mobile application for checking visual acuity. The application displays a test letter in crowding bars (HOTV for children 6 years and younger, and ETDRS letters for children 7 years and older) below four randomized letters in crowding bars. ETDRS = Early Treatment of Diabetic Retinopathy Study; HOTV = the four letters tested.

      Results

      Fifty-three children were enrolled in the study. The application was prematurely stopped when the vision measurements in 2 children were recorded as much worse than their true vision (96% testability). Observation from the study personnel suggested that the tester was tilting the phone downward while tilting the phone to the side, thus prematurely locking in an answer. These children were not used in the statistical evaluation. Demographics presented in Table 1. There were 15 children <7 years of age. Four eyes could not be measured by the GoCheck Kids application because they could not identify the largest (20/63; −0.5 logMAR) optotype on the device. At least 2 testers physically dropped the phone during testing. The mean logMAR (imperial) acuity assessed by GoCheck Kids was 0.106 (20/25.5); 0.012 (20/20.5) by HOTV-ATS; and 0.096 (20/22) by chart. The mean differences and CI intervals between GoCheck Kids and HOTV-ATS acuities (acuity differences: 0.094; 95% CI: 0.074-0.114) were significantly different (P < .001). The mean differences and 95% CI between GoCheck Kids and chart acuities (acuity difference: 0.010; 95% CI: −0.010 to 0.030) were not significantly different (P = .319). The mean difference and CIs between HOTV-ATS and chart acuities (0.084: 95% CI: 0.014-0.063) were significantly different (P < .001). The intraclass correlation between GoCheck Kids and HOTV-ATS acuities was 0.55 (95% CI: 0.40-0.68) and 0.59 (95% CI: 0.45-0.71) between HOTV-ATS and chart acuities, indicating fair agreement between each of the 2 sets of measurements of acuity. The intraclass correlation between GoCheck Kids and chart acuities was 0.66 (95% CI: 0.53-0.76), indicating a modest agreement between the measures. Bland-Altman plots are presented in Figure 2. The percentage of eyes with visual acuity as measured by GoCheck Kids that was within 1 line of the HOTV-ATS, and chart acuity was 65.3% and 86.7%, respectively. The percentage of eyes that was within visual acuity as measured by GoCheck Kids was within 2 lines of the HOTV-ATS, and both chart acuities were 96.9%. There were 14 children with a history of unilateral amblyopia. The differences between the acuity between the amblyopic eye and the nonamblyopic eye as measured by each method are in Table 2.
      Table 1Patient Demographics
      VariableStatistic
      Age, mon (%)51
      Mean ± SD120.6 (49)
      Min, Median, Maximum47, 118, 215
      Gender
       Males21 (41)
       Females30 (59)
      Race
       White26 (52)
       African American15 (30)
       Hispanic5 (10)
       Asian4 (8)
      Vision disease19 (37)
      Tester
       Parent49 (96)
       Sibling2 (4)
      Data values are n (%) or mean ± SD.
      Figure thumbnail gr2
      Figure 2Bland-Altman Plots for the 3 methods of checking visual acuity.
      Table 2Visual Acuity Measured by Each Method for Children with Amblyopia
      Age, yDifferences in Acuity Between Eyes
      Phone AcuityHOTV-ATSChart
      10.8000.2
      5.000.10.1
      5.80.10.20.2
      3.90.10.10.1
      5.50.100.1
      10.30.20.40.2
      7.00.200
      7.20.20.10.1
      10.70.30.40.1
      7.40.30.10
      5.40.30.10
      13.5Unable
      These children had visual acuities that were too low to be tested by the phone application.
      0.50.6
      16.8Unable
      These children had visual acuities that were too low to be tested by the phone application.
      0.50.4
      17.3Unable
      These children had visual acuities that were too low to be tested by the phone application.
      11
      a These children had visual acuities that were too low to be tested by the phone application.

      Discussion

      Telemedicine has arrived. COVID-19 has pushed us to adapt so we can continue to safely serve our patients. Before COVID-19, telemedicine was rapidly developing in the field of ophthalmology, mostly for diseases like diabetic retinopathy and glaucoma, by obtaining images and sending them to an ophthalmologist for review
      • Kirkizlar E.
      • Serban N.
      • Sisson J.A.
      • Swann J.L.
      • Barnes C.S.
      • Williams M.D.
      Evaluation of telemedicine for screening of diabetic retinopathy in the Veterans Health Administration.
      • Zimmer-Galler I.E.
      • Kimura A.E.
      • Gupta S.
      Diabetic retinopathy screening and the use of telemedicine.
      • Sanchez C.R.
      • Silva P.S.
      • Cavallerano J.D.
      • Aiello L.P.
      • Aiello L.M.
      Ocular telemedicine for diabetic retinopathy and the Joslin Vision Network.
      • Li B.
      • Powell A.M.
      • Hooper P.L.
      • Sheidow T.G.
      Prospective evaluation of teleophthalmology in screening and recurrence monitoring of neovascular age-related macular degeneration: a randomized clinical trial.
      • Bergua A.
      • Mardin C.Y.
      • Horn F.K.
      Tele-transmission of stereoscopic images of the optic nerve head in glaucoma via internet.
      or for interpretation by artificial intelligence
      • Padhy S.K.
      • Takkar B.
      • Chawla R.
      • Kumar A.
      Artificial intelligence in diabetic retinopathy: a natural step to the future.
      and in retinopathy of prematurity using deep neural networks.
      • Wang J.
      • Ju R.
      • Chen Y.
      • et al.
      Automated retinopathy of prematurity screening using deep neural networks.
      In the COVID-19 era of teleophthalmology, we do not have the benefit of these sophisticated methods of image acquisition and interpretation. We also need a way of accurately measuring visual acuity, especially in children, in order help diagnose and treat amblyopia. The ideal method of checking visual acuity at home would be easy to use, accurate, precise, inexpensive (or free), ensure monocular testing and be available to test every single patient and family. ABCD-Vision (Anchorage, Alaska) developed a paper-based HOTV crowding bar visual acuity test to be used for remote use in Alaska.
      • Tsao Wu M.
      • Armitage M.D.
      • Trujillo C.
      • et al.
      Portable acuity screening for any school: validation of patched HOTV with amblyopic patients and Bangerter normals.
      This 8.5- × 11-inch paper acuity chart can be emailed for at-home printing or mailed to patients who do not have access to a printer. Another method of checking vision at home is using a smartphone or computer. Some of these programs are based on peer-reviewed studies (Peek Acuity; Peek Vision, London, United Kingdom); EyeHandbook (Cloud Nine Development, Overland Park, Kansas); and Jaeb Visual Acuity Screener (Jaeb Center for Health Research, Tampa, Florida)
      • Moke P.S.
      • Turpin A.H.
      • Beck R.W.
      • et al.
      Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol.
      ,
      • Zhao L.
      • Stinnett S.S.
      • Prakalapakorn S.G.
      Visual acuity assessment and vision screening using a novel smartphone application.
      • Bastawrous A.
      • Rono H.K.
      • Livingstone I.A.
      • et al.
      Development and validation of a smartphone-based visual acuity test (peek acuity) for clinical practice and community-based fieldwork.
      • de Venecia B.
      • Bradfield Y.
      • Trane R.M.
      • Bareiro A.
      • Scalamogna M.
      Validation of peek acuity application in pediatric screening programs in Paraguay.
      • Yamada T.
      • Hatt S.R.
      • Leske D.A.
      • et al.
      A new computer-based pediatric vision-screening test.
      • Zhang Z.T.
      • Zhang S.C.
      • Huang X.G.
      • Liang L.Y.
      A pilot trial of the iPad tablet computer as a portable device for visual acuity testing.
      • Trivedi R.H.
      • Wilson M.E.
      • Peterseim M.M.
      • Cole K.B.
      • Teed R.G.
      A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children.
      ,
      • Tofigh S.
      • Shortridge E.
      • Elkeeb A.
      • Godley B.F.
      Effectiveness of a smartphone application for testing near visual acuity.
      ; and some are not (Farsight; Farsight.care; London, United Kingdom); Eye Chart Pro (Dok LLC; Apple, Cupertino, California); Visual Acuity Charts (Fonlo); and Snellen Acuity (João Meneses [available as an application]) (Table 3). There are several issues with using electronic applications to check visual acuity. The first issue is access. Some applications are only available on Android (Google, Mountain View, California), and others are exclusive to iOS (Apple). Other applications are available only on Windows-based computers (Microsoft, Redmond, Washington). Although smartphones are considered ubiquitous, consideration should also be taken into account for those without access to this technology due to their financial situations. This limits the ability to provide uniform directions to patients and have consistent measurements across all patients. However, if a patient uses the same method for checking visual acuity over 2 visits, the acuities are theoretically comparable. Second, we have to trust that the developer has modified the size of the optotypes for different screen sizes. Some applications provide configuration screens to ensure the correct size of the optotypes. Third, the choice of optotypes can be important. Tumbling “E” optotypes (used by Peek Acuity) can be difficult for very young children who have not developed the ability to express the orientation of the optotypes,
      • Donahue S.P.
      • Baker C.N.
      on behalf or Committee on Practice and Ambulatory Medicine, American Academy of Pediatrics
      Procedures for the evaluation of the visual system by pediatricians.
      although more recent studies report excellent testability.
      • Guimaraes S.
      • Fernandes T.
      • Costa P.
      • Silva E.
      Should tumbling E go out of date in amblyopia screening? Evidence from a population-based sample normative in children aged 3-4 years.
      Fourth, some orientation may be required for a family to check vision at home, for example, via in app directions, a video guide, or testing while on a teleconference with an ophthalmic technician. This unique study explores the use of GoCheck Kids to check visual acuity by the patient's relative with minimal training. There have been many studies evaluating the assessment of visual acuity by lay screeners in vision screening programs.
      • Schmidt P.
      • Maguire M.
      • Dobson V.
      • et al.
      Comparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision In Preschoolers Study.
      • Sabri K.
      • Thornley P.
      • Waltho D.
      • et al.
      Assessing accuracy of non-eye care professionals as trainee vision screeners for children.
      • Sabri K.
      • Easterbrook B.
      • Khosla N.
      • Davis C.
      • Farrokhyar F.
      Paediatric vision screening by non-healthcare volunteers: evidence based practices.
      • Hashemi H.
      • Yekta A.
      • Jafarzadehpur E.
      • et al.
      Sensitivity and specificity of preschool vision screening in Iran.
      • Sharma A.
      • Li L.
      • Song Y.
      • et al.
      Strategies to improve the accuracy of vision measurement by teachers in rural Chinese secondary schoolchildren: Xichang Pediatric Refractive Error Study (X-PRES) report no. 6.
      • Marmamula S.
      • Khanna R.C.
      • Mettla A.L.
      • et al.
      Agreement and diagnostic accuracy of vision screening in children by teachers, community eye-health workers and vision technicians.
      • Adhikari S.
      • Shrestha U.
      Validation of performance of certified medical assistants in preschool vision screening examination.
      • Saxena R.
      • Vashist P.
      • Tandon R.
      • Pandey R.M.
      • Bhardawaj A.
      • Menon V.
      Accuracy of visual assessment by school teachers in school eye screening program in Delhi.
      The goal of those studies is to refer children with decreased visual acuity below an age-based critical line for further examination by an eye care provider. Now, the narrative is shifting to explore testing visual acuity as measured by those methods as a visual acuity that an ophthalmologist will use for their evaluation and decision making process. This study shows that visual acuity using GoCheck Kids at 5 feet has fair agreement with HOTV-ATS. On average, the visual acuity as measured by GoCheck Kids is 1 line worse than HOTV-ATS and very close to the chart acuity. The visual acuity measured by HOTV-ATS averages one line better than the chart acuity. Therefore, it is unclear if both GoCheck Kids and chart acuity underestimate vision, or the HOTV-ATS overestimates vision. The average age of children in our study was 10 years old; these children may have understood that there were only 4 choices in the HOTV-ATS and guessed on the letters based on similar recognition rather than true identification, thus measuring vision as better than the other methods. In addition, prior studies have shown that HOTV-ATS overestimates vision compared to E-ETDRS by 0.68 lines for amblyopic eyes and 0.25 lines for fellow eyes.
      • Birch E.E.
      • Strauber S.F.
      • Beck R.W.
      • Holmes J.M.
      Comparison of the amblyopia treatment study HOTV and the electronic-early treatment of diabetic retinopathy study visual acuity protocols in amblyopic children aged 5 to 11 years.
      This is the largest weakness of this study. A similar study was performed for the Peek Acuity application.
      • Zhao L.
      • Stinnett S.S.
      • Prakalapakorn S.G.
      Visual acuity assessment and vision screening using a novel smartphone application.
      This study tested 111 children, 3-17 years of age and compared visual acuity from Peek Acuity with that from clinic protocol visual acuity and showed a good ICC for the first (0.88) and second (0.84) eyes tested; although the ICC dropped for second eyes (0.45) in children 3-5 years old. The authors did not test for that variable. When assessing and treating amblyopia, one might argue that the exact visual acuity does not matter as much as the difference of visual acuity between the 2 eyes. In children with amblyopia, there was no consistency in the difference between the two eyes among all the methods of checking visual acuity. A total of 1 in 14 children had the same difference of acuity among all methods. A strength of this study is the method that visual acuity was checked using the GoCheck Kids application. We purposefully had the family members of the child perform the measurement to simulate checking vision at home. The study personnel also limited communication and instruction to the tester once the test had started to further mimic home testing environment. Children were able to learn the visual acuity game quickly. Parents had mild difficulty learning the motions needed to manipulate the device, as indicated by the dropped phones and the premature stopping of the test. Further studies could be performed by having family members check vision at home with the application prior to presenting to the clinic for their examination. We recommend further studies use HOTV-ATS for children <7 years of age and E-ETDRS for children >7 years of age.
      Table 3Selected Options for Remotely Checking VA Using Mobile Applications or Computers
      VA MethodTesting DistanceDeviceStudiedAvailable
      GoCheck KidsHOTV

      ETDRS

      Crowded
      5 feetiOS AndroidMethod ✓
      • Trivedi R.H.
      • Wilson M.E.
      • Peterseim M.M.
      • Cole K.B.
      • Teed R.G.
      A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children.
      ,
      Underestimates HOTV-ATS by 0.09 LogMAR, comparable to Snellen by 0.01 LogMAR.


      FDA class I device
      Yes-free
      Peek AcuityTumbling E

      Boxed
      2 metersAndroid
      • Zhao L.
      • Stinnett S.S.
      • Prakalapakorn S.G.
      Visual acuity assessment and vision screening using a novel smartphone application.
      • Bastawrous A.
      • Rono H.K.
      • Livingstone I.A.
      • et al.
      Development and validation of a smartphone-based visual acuity test (peek acuity) for clinical practice and community-based fieldwork.
      • de Venecia B.
      • Bradfield Y.
      • Trane R.M.
      • Bareiro A.
      • Scalamogna M.
      Validation of peek acuity application in pediatric screening programs in Paraguay.
      ,
      Underestimates vision by 0.07-0.08 logMAR.23


      FDA class I device
      Yes-free
      Jaeb Visual Acuity ScreenerHOTV

      ETDRS

      Crowded
      5 feetWindows
      • Yamada T.
      • Hatt S.R.
      • Leske D.A.
      • et al.
      A new computer-based pediatric vision-screening test.
      Yes-free
      Eye HandBookModified Jaeger number

      X-O

      Tumbling E

      Landolt C
      14 inchesiOS

      Android
      Overestimates near vision by 0.11 logMAR (iPhone5).41
      Yes-free
      “Visual Acuity Charts”

      - iOS

      Snellen Chart, ETDRS Chart, Landolt C Chart, Tumbling E chart

      - Android
      Single optotype

      Snellen

      Tumbling E

      Landolt C

      ETDRS

      HOTV numbers
      2-6 metersiOS

      Android
      XYes, iOS: free until 5/13/2020, then $4.99

      Android: free: Snellen, Tumbling E

      $1.49: ETDRS, Landolt C
      EyeChart Pro
      EyeChart Pro has 2 applications for iPhone: the EyeChart-Vision Screening and the EyeChart HD-Vision Screening. Both apps work at 4 feet but do not recognize the size of iPhone 11 pro; the unit states it is using an iPhone 6. There are other mobile applications for checking visual acuity that are not included on this list.
      Snellen

      Sloan

      Tumbling E

      Landolt C
      4 feetiOS – iPad only
      • Zhang Z.T.
      • Zhang S.C.
      • Huang X.G.
      • Liang L.Y.
      A pilot trial of the iPad tablet computer as a portable device for visual acuity testing.
      Free basic; >$18.00 for increased features
      Farsight.careNumbers14 inchesWeb-basedXFree
      Kay iSight Test ProKay Letter

      Kay Picture line

      boxed line

      boxed
      10 feetiOSX- App

      ✓-Kay letters
      • O'Boyle C.
      • Chen S.I.
      • Little J.A.
      Crowded letter and crowded picture logMAR acuity in children with amblyopia: a quantitative comparison.


      Underestimate vision by 1 line
      Free for 6 months during COVID-19
      X = no; ✓ = yes; COVID-19 = coronavirus 2019; ETDRS = Early Treatment Diabetic Retinopathy Study; VA = visual acuity.
      a Underestimates HOTV-ATS by 0.09 LogMAR, comparable to Snellen by 0.01 LogMAR.
      b Underestimates vision by 0.07-0.08 logMAR.
      • Bastawrous A.
      • Rono H.K.
      • Livingstone I.A.
      • et al.
      Development and validation of a smartphone-based visual acuity test (peek acuity) for clinical practice and community-based fieldwork.
      c Overestimates near vision by 0.11 logMAR (iPhone5).
      • Tofigh S.
      • Shortridge E.
      • Elkeeb A.
      • Godley B.F.
      Effectiveness of a smartphone application for testing near visual acuity.
      d EyeChart Pro has 2 applications for iPhone: the EyeChart-Vision Screening and the EyeChart HD-Vision Screening. Both apps work at 4 feet but do not recognize the size of iPhone 11 pro; the unit states it is using an iPhone 6. There are other mobile applications for checking visual acuity that are not included on this list.
      A weakness of this study was that the children were not patched, using occlusion with patches over glasses, occlusion glasses (that have occluders that flip down over the nontested eye), or a handheld occluder. Children are more consistent with visual acuity when an occlusive patch was applied directly to the skin. In addition, as most children who would be treated for amblyopia may be <7 years of age, this study was limited the our small sample size in this age group. In its current form, GoCheck Kids is limited by the size of an iPhone Plus screen size. The lowest acuity that can be measured at 5 feet is 20/63 (0.5 logMAR). Some testing methods allow children to move closer to the chart and changing the numerator of the visual acuity, but this is not possible with the already short testing distance of 5 feet for GoCheck Kids. This significantly limits the use of this application when testing children with amblyopia, as many of them will have an acuity of <0.5 logMAR. When testing vision at home, it may be important to have a backup test if clinical suspicion arises that the acuity is not consistent with prior tests of vision or if the visual acuity is too low to be tested by the application. It is unknown how GoCheck Kids will perform for measuring visual acuity in children with uncorrected refractive errors. It may over-estimate distance acuity in children with myopia due to the 5-foot testing distance. A program that still allows for matching when needed and the ability to test at 10-feet would be beneficial for a pediatric testing device; a second device that is paired may be helpful for the child to hold when matching the letter at distance (though this would require the family to have two devices).
      In June 2020, GoCheck released an application with an updated user interface dedicated to checking visual acuity. It is available outside of the GoCheck Kids vision screening ecosystem. In conclusion, GoCheck Kids has a high rate of testability and provides a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, although most are within 1 line. More studies are needed to evaluate this method of checking visual acuity at home for teleophthalmology.

      CRediT authorship contribution statement

      Evan Silverstein: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing - original draft. Jonathan S. Williams: Methodology, Data curation, Writing - review & editing. Jeffrey R. Brown: Investigation, Writing - review & editing. Enjana Bylykbashi: Investigation, Writing - review & editing. Sandra S. Stinnett: Formal analysis, Writing - original draft.
      All authors have completed and submitted the ICMJE form for Disclosure of Potential Conflicts of Interest and none were reported.
      Funding/Support: Redcap use supported by the National Center for Research Resources grant UL1TR002649 (Research Electronic Data Capture).
      Financial disclosures: E.S. is the owner of See Vision, LLC.
      The authors thank GoCheck Kids who provided an iPhone7 Plus to the study with the GoCheck Kids application loaded and ready to use.

      Supplemental Data

      References

      1. Important coronavirus updates for ophthalmologists. American Academy of Ophthalmology.
        (Available at:)
      2. Northam R. S. Governor Northam Issues Statewide Stay at Home Order. Office of the Governor.
        (Available at:)
        • Chen S.I.
        • Chandna A.
        • Norcia A.M.
        • Pettet M.
        • Stone D.
        The repeatability of best corrected acuity in normal and amblyopic children 4 to 12 years of age.
        Invest Ophthalmol Vis Sci. 2006; 47: 614-619
        • Holmes J.M.
        • Beck R.W.
        • Repka M.X.
        • et al.
        The amblyopia treatment study visual acuity testing protocol.
        Arch Ophthalmol. 2001; 119: 1345-1353
        • O'Boyle C.
        • Chen S.I.
        • Little J.A.
        Crowded letter and crowded picture logMAR acuity in children with amblyopia: a quantitative comparison.
        Br J Ophthalmol. 2017; 101: 457-461
        • Becker R.
        • Hübsch S.
        • Gräf M.H.
        • Kaufmann H.
        Examination of young children with Lea symbols.
        Br J Ophthalmol. 2002; 86: 513-516
        • Anstice N.S.
        • Jacobs R.J.
        • Simkin S.K.
        • Thomson M.
        • Thompson B.
        • Collins A.V.
        Do picture-based charts overestimate visual acuity? Comparison of Kay Pictures, Lea Symbols, HOTV and Keeler logMAR charts with Sloan letters in adults and children.
        PLoS One. 2017; 12: e0170839
        • Huurneman B.
        • Boonstra F.N.
        Assessment of near visual acuity in 0-13 year olds with normal and low vision: a systematic review.
        BMC Ophthalmol. 2016; 16: 215
        • Inal A.
        • Ocak O.B.
        • Aygit E.D.
        • et al.
        Comparison of visual acuity measurements via three different methods in preschool children: Lea symbols, crowded Lea symbols, Snellen E chart.
        Int Ophthalmol. 2018; 38: 1385-1391
        • Ruttum M.S.
        • Dahlgren M.
        Comparison of the HOTV and Lea symbols visual acuity tests in patients with amblyopia.
        J Pediatr Ophthalmol Strabismus. 2006; 43: 157-160
        • Moke P.S.
        • Turpin A.H.
        • Beck R.W.
        • et al.
        Computerized method of visual acuity testing: adaptation of the amblyopia treatment study visual acuity testing protocol.
        Am J Ophthalmol. 2001; 132: 903-909
        • Donahue S.P.
        • Baker C.N.
        • on behalf or Committee on Practice and Ambulatory Medicine, American Academy of Pediatrics
        Procedures for the evaluation of the visual system by pediatricians.
        Pediatrics. 2016; 137https://doi.org/10.1542/peds.2015–3597
        • Donahue S.P.
        • Arthur B.
        • Neely D.E.
        • et al.
        Guidelines for automated preschool vision screening: a 10-year, evidence-based update.
        J AAPOS. 2013; 17: 4-8
        • Schmidt P.
        • Maguire M.
        • Dobson V.
        • et al.
        Comparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision In Preschoolers Study.
        Ophthalmology. 2004; 111: 637-650
        • Sabri K.
        • Thornley P.
        • Waltho D.
        • et al.
        Assessing accuracy of non-eye care professionals as trainee vision screeners for children.
        Can J Ophthalmol. 2016; 51: 25-29
        • Sabri K.
        • Easterbrook B.
        • Khosla N.
        • Davis C.
        • Farrokhyar F.
        Paediatric vision screening by non-healthcare volunteers: evidence based practices.
        BMC Med Educ. 2019; 19: 65
        • Hashemi H.
        • Yekta A.
        • Jafarzadehpur E.
        • et al.
        Sensitivity and specificity of preschool vision screening in Iran.
        Iran J Public Health. 2017; 46: 207-215
        • Sharma A.
        • Li L.
        • Song Y.
        • et al.
        Strategies to improve the accuracy of vision measurement by teachers in rural Chinese secondary schoolchildren: Xichang Pediatric Refractive Error Study (X-PRES) report no. 6.
        Arch Ophthalmol. 2008; 126: 1434-1440
        • Marmamula S.
        • Khanna R.C.
        • Mettla A.L.
        • et al.
        Agreement and diagnostic accuracy of vision screening in children by teachers, community eye-health workers and vision technicians.
        Clin Exp Optom. 2018; 101: 553-559
        • Adhikari S.
        • Shrestha U.
        Validation of performance of certified medical assistants in preschool vision screening examination.
        Nepal J Ophthalmol. 2011; 3: 128-133
        • Saxena R.
        • Vashist P.
        • Tandon R.
        • Pandey R.M.
        • Bhardawaj A.
        • Menon V.
        Accuracy of visual assessment by school teachers in school eye screening program in Delhi.
        Indian J Community Med. 2015; 40: 38-42
        • Zhao L.
        • Stinnett S.S.
        • Prakalapakorn S.G.
        Visual acuity assessment and vision screening using a novel smartphone application.
        J Pediatr. 2019; 213: 203-210
        • Bastawrous A.
        • Rono H.K.
        • Livingstone I.A.
        • et al.
        Development and validation of a smartphone-based visual acuity test (peek acuity) for clinical practice and community-based fieldwork.
        JAMA Ophthalmol. 2015; 133 ([Erratum appears in JAMA Ophthalmol 2015;133(9):1096]): 930-937
        • de Venecia B.
        • Bradfield Y.
        • Trane R.M.
        • Bareiro A.
        • Scalamogna M.
        Validation of peek acuity application in pediatric screening programs in Paraguay.
        Int J Ophthalmol. 2018; 11: 1384-1389
        • Yamada T.
        • Hatt S.R.
        • Leske D.A.
        • et al.
        A new computer-based pediatric vision-screening test.
        J AAPOS. 2015; 19: 157-162
        • Zhang Z.T.
        • Zhang S.C.
        • Huang X.G.
        • Liang L.Y.
        A pilot trial of the iPad tablet computer as a portable device for visual acuity testing.
        J Telemed Telecare. 2013; 19: 55-59
        • Trivedi R.H.
        • Wilson M.E.
        • Peterseim M.M.
        • Cole K.B.
        • Teed R.G.
        A pilot study evaluating the use of EyeSpy video game software to perform vision screening in school-aged children.
        J AAPOS. 2010; 14: 311-316
        • Peterseim M.M.W.
        • Rhodes R.S.
        • Patel R.N.
        • et al.
        Effectiveness of the GoCheck Kids vision screener in detecting amblyopia risk factors.
        Am J Ophthalmol. 2018; 187: 87-91
        • Law M.X.
        • Pimentel M.F.
        • Oldenburg C.E.
        • de Alba Campomanes A.G.
        Positive predictive value and screening performance of GoCheck Kids in a primary care university clinic.
        J AAPOS. 2020; 24 (1–17): 17
        • Walker M.
        • Duvall A.
        • Daniels M.
        • et al.
        Effectiveness of the iPhone GoCheck Kids smartphone vision screener in detecting amblyopia risk factors.
        J AAPOS. 2020; 24 (e1–16): 16
        • Harris P.A.
        • Taylor R.
        • Thielke R.
        • Payne J.
        • Gonzalez N.
        • Conde J.G.
        Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Harris P.A.
        • Taylor R.
        • Minor B.L.
        • et al.
        The REDCap consortium: Building an international community of software platform partners.
        J Biomed Inform. 2019; 95: 103208
        • Kirkizlar E.
        • Serban N.
        • Sisson J.A.
        • Swann J.L.
        • Barnes C.S.
        • Williams M.D.
        Evaluation of telemedicine for screening of diabetic retinopathy in the Veterans Health Administration.
        Ophthalmology. 2013; 120: 2604-2610
        • Zimmer-Galler I.E.
        • Kimura A.E.
        • Gupta S.
        Diabetic retinopathy screening and the use of telemedicine.
        Curr Opin Ophthalmol. 2015; 26: 167-172
        • Sanchez C.R.
        • Silva P.S.
        • Cavallerano J.D.
        • Aiello L.P.
        • Aiello L.M.
        Ocular telemedicine for diabetic retinopathy and the Joslin Vision Network.
        Semin Ophthalmol. 2010; 25: 218-224
        • Li B.
        • Powell A.M.
        • Hooper P.L.
        • Sheidow T.G.
        Prospective evaluation of teleophthalmology in screening and recurrence monitoring of neovascular age-related macular degeneration: a randomized clinical trial.
        JAMA Ophthalmol. 2015; 133: 276-282
        • Bergua A.
        • Mardin C.Y.
        • Horn F.K.
        Tele-transmission of stereoscopic images of the optic nerve head in glaucoma via internet.
        Telemed J E Health. 2009; 15: 439-444
        • Padhy S.K.
        • Takkar B.
        • Chawla R.
        • Kumar A.
        Artificial intelligence in diabetic retinopathy: a natural step to the future.
        Indian J Ophthalmol. 2019; 67: 1004-1009
        • Wang J.
        • Ju R.
        • Chen Y.
        • et al.
        Automated retinopathy of prematurity screening using deep neural networks.
        EBioMedicine. 2018; 35: 361-368
        • Tsao Wu M.
        • Armitage M.D.
        • Trujillo C.
        • et al.
        Portable acuity screening for any school: validation of patched HOTV with amblyopic patients and Bangerter normals.
        BMC Ophthalmol. 2017; 17: 232
        • Tofigh S.
        • Shortridge E.
        • Elkeeb A.
        • Godley B.F.
        Effectiveness of a smartphone application for testing near visual acuity.
        Eye (Lond). 2015; 29: 1464-1468
        • Guimaraes S.
        • Fernandes T.
        • Costa P.
        • Silva E.
        Should tumbling E go out of date in amblyopia screening? Evidence from a population-based sample normative in children aged 3-4 years.
        Br J Ophthalmol. 2018; 102: 761-766
        • Birch E.E.
        • Strauber S.F.
        • Beck R.W.
        • Holmes J.M.
        Comparison of the amblyopia treatment study HOTV and the electronic-early treatment of diabetic retinopathy study visual acuity protocols in amblyopic children aged 5 to 11 years.
        J AAPOS. 2009; 13: 75-78