Neural networks to identify glaucomatous visual field progression

Design 

This study was done as an observational retrospective cohort study. The study took place at The Study Center: Jules Stein Eye Institute, UCLA School of Medicine (Los Angeles, California, USA).

Methods 

Patients 

Data from two AGIS testing centers (Yale University and Georgetown University), collected from October 1988 to January 2000, were used to develop and test the neural network. Study eyes were required to have uncontrolled intraocular pressure (IOP). This means that despite maximally tolerated and effective available treatment, the last two IOP measurements, made on separate days, were both above a critical level, as determined by the AGIS protocol. For an eye with little or no visual field defect, the critical IOP level was 30 mm Hg. For an eye with a mild or moderate visual defect, the critical IOP level was 24 to 29 mm Hg. For an eye with documented disk rim deterioration or specified worsening of visual field, the critical IOP level was 18 mm Hg or more. The first visual field test established eligibility into AGIS. Patients then had a second test within 60 days, which served as a reference measurement for comparison with later tests. A detailed description of study criteria is available.12

Standardized visual field testing was performed with the HFA (Carl Zeiss Ophthalmic Systems, Inc., Dublin, California, USA), equipped with STATPAC-2 software. Examiners were trained and certified to perform the tests. The central 24-2 threshold program with the size III white stimulus full-threshold strategy was used in all testing. The pupil diameter was at least 2 mm before testing. Otherwise, the pupil was dilated using neosynephrine, possibly supplemented by a cycloplegic agent.

Reliability is the consistency or reproducibility of response. There were five criteria for determining intratest reliability: total number of stimuli presented, fixation loss, false-positive responses, false-negative responses, and amount of short-term fluctuation. A visual field test was repeated if the patient had a reliability rating of more than two, according to the AGIS criteria.13

The HFA recorded data from 55 locations in the visual field, of which 52 were used in visual defect scoring. These included 23 locations in each hemifield and 6 locations in the nasal field. The locations above and below the blind spot and the location in the exact center of vision were not used in the calculation of the AGIS score, although they were included in the inputs to the neural network.

For AGIS, a visual field defect had to be caused by glaucoma, as determined from clinical examination. Scoring was based on the number, pattern, and depth of depression, compared with age-adjusted normals, as found in the Humphrey STATPAC-2 total deviation plot. The amount of depression, in decibels (dB), that makes a test location considered defective, depends upon its location. These amounts are found in no more than 5% of age-matched healthy subjects. The scoring for this study was performed by one of the authors (A.L.).

The AGIS visual field score ranges from 0 (no defect) to 20 (all sites severely depressed). Points are awarded for the presence of nasal defect (a cluster of three or more depressed locations in the nasal field), nasal step (one or more depressed locations in the nasal field in the absence of depression in any of the three locations on the opposite side of the horizontal midline), and hemifield defect (cluster of three or more depressed sites in a hemifield). In our study, eyes with baseline scores of 17 or more were excluded. Progression of glaucoma (decrease of visual field) was defined by the AGIS protocol as an increase in the AGIS score of four or more, confirmed by two sequential subsequent 6-month tests.14, 15

We used a back-propagation neural network in this study. In this design, the processing elements in each layer are connected to all processing elements of the preceding and following layers but not to processing elements of the same layer (Figure 1). During the training phase, the input data are presented to the lowest layer of the neural network, with each input value associated with a single processing element. The processing element multiplies the input by a weight associated with its synapses or connections with the next layer. If the result exceeds a threshold specified by a predefined transfer function, then the processing element transmits its result to all the elements of the second layer. Each processing element receives data from the first layer and processes it in a similar manner as before. Eventually, the data reaches the output layer and produces an answer, which is compared with the correct answer. An error value is calculated and passed backwards through the network. The weights of the synapses are adjusted according to a predefined learning function. This process is repeated with every presentation of input data. Each example is repeated several times in random sequence during training and the error converges to a minimal value. During the testing phase, new inputs are presented to the neural network, without the correct answer, and weight adjustment occurs. The outputs are recorded and compared with the correct answers.

View Large Image Download to PowerPoint

FIGURE 1. Schematic representation of a back-propagation neural network. Each circle represents a processing element.

The neural network was implemented with the NeuralWorks Professional II/Plus (NeuralWare Inc., Pittsburgh, Pennsylvania, USA) software package. The network consisted of an input layer, three hidden layers, and an output layer. The numbers of processing elements in each layer were 110, 37, 15, 5, and 1, respectively. The default settings of momentum (0.4), transition point (10000 trials), learning rule (Ext DBD), and transfer function (hyperbolic tangent) were used since they were useful in our previous work.9, 11

Each input data trial consisted of 55 numbers corresponding to the 55 visual field relative thresholds measured at the initial examination, followed by 55 additional numbers corresponding to the visual field relative thresholds for a later examination for the same eye; thus, there were 110 inputs for each trial. These 110 inputs corresponded to the 110 processing elements in the input layer of the neural network. There were 1,686 different trials when considering all 106 eyes from 80 patients. Another neural network used data from 80 eyes of 80 patients, using one randomly selected eye from those patients with both eyes enrolled in AGIS. There were 1,261 trials when considering only 80 eyes. Separate randomizations and neural networks were used for each data set. Each trial contained a parameter for the diagnosis, which was arbitrarily defined as progression = 1 or no progression = 0. Each data set was randomized and divided into thirds. Two thirds of the randomized data were used for training, and the remaining one third was used for testing the network’s capability of giving the correct output diagnosis. Separating the training and testing sets minimized the potential of memorizing patterns seen in training and maximized the true potential of the neural network. The closer the output was to 0, the more certain the network was of a nonprogressing field; the closer the output was to 1, the more certain it was of a progressing field. The process of dividing the data was repeated two more times and each time a different combination of groups for training and testing was used.

One additional randomization of a separate neural network for each data set was performed, using a broader definition of progression. In this case, progression required an increase in the AGIS score of 4 or more with no confirmation needed.

Results 

Data from 80 patients enrolled in the AGIS were used in the current study. The mean age at the time of the initial reference examination was 67.4 ± 7.3 years. The mean follow-up period was 7.2 ± 2.3 years, and the mean duration between adjacent examinations was 0.46 ± 0.39 years (Table 1). Twenty-six patients had both eyes enrolled in AGIS. Analyses for data from these patients were performed in two ways: (1) with one eye randomly selected and (2) with both eyes included.

TABLE 1. Baseline Characteristics of Patients

	No. of patients	80
	No. of eyes	106
	Mean age of patients (yrs)	67.4 ± 7.3
	Mean follow-up period (yrs)	7.2 ± 2.3
	Mean duration between examinations (yrs)	0.46 ± 0.39

The following results are derived from visual field data from 80 eyes of 80 patients. A best threshold for the neural network’s results was determined by the best receiver operator characteristic (ROC) curve analysis.16 With the definition of progression as a change in AGIS score of at least four, confirmed by two subsequent tests, the best threshold was found at 0.24. The authors determined the best threshold as the cutoff value between 0 and 1 for defining progression and nonprogression. The value of 0.24 gave the best data with regard to sensitivity and specificity. Thus, if a result was higher than this threshold, it was considered by the neural network to be a progressing visual field; if a result was below the threshold, it was considered to be a stable one. The sensitivity of the network was calculated as the number of true-positive results divided by the number of true-positives plus the number of false-negative results; the specificity was calculated as the number of true-negative results divided by the number of true-negatives plus the number of false-positives; the diagnostic precision was calculated as the number of correct diagnoses divided by the total number of test trials. The sensitivity of the network to detect progression was 0.86; the specificity was 0.89; the diagnostic precision was 0.88 (Table 2). The area under the ROC curve was 0.92. Using the alternate definition of progression (a change in AGIS score of four or more with no subsequent confirmations needed), the best threshold was also found to be at 0.24. The sensitivity was 0.87, the specificity was 0.89, and the diagnostic precision was 0.88. The area under the ROC curve for the broader definition was 0.94. Both of these ROC curves are shown in Figure 2. No significant difference between the ROC curves was found (P = .84) by a nonparametric test.17 The neural network had results similar to what was categorized as stable and progressing with the AGIS scoring system (FIGURE 3, FIGURE 4).

TABLE 2. Measurements of Diagnostic Accuracy for two Definitions of Glaucomatous Progression at two Threshold Levels

		Progression With Confirmation		Progression Without Confirmation
		Threshold = 0.5	Best Threshold (= 0.23)	Threshold = 0.5	Best Threshold (= 0.21)
	Sensitivity	83%	92%	75%	86%
	Specificity	96%	92%	94%	88%
	Diagnostic precision	94%	92%	89%	88%
	Area under ROC curve	—	0.96	—	0.93

ROC = receiver operator characteristics.

View Large Image Download to PowerPoint

FIGURE 2. Receiver operator characteristic (ROC) curves for 80 eyes from 80 patients for two definitions of progression.

View Large Image Download to PowerPoint

FIGURE 3. Percentages of visual field follow-up examinations categorized as stable vs progressing by the Advanced Glaucoma Intervention Study (AGIS) criteria and the neural networks using two definitions of progression.

View Large Image Download to PowerPoint

FIGURE 4. Venn diagram of visual field follow-up examinations categorized as progressing by the Advanced Glaucoma Intervention Study (AGIS) criteria and the neural networks using two definitions of progression.

When data from 106 eyes of 80 patients enrolled in the AGIS were used, the results were slightly different. Using the original definition of progression, the best threshold was found at 0.25, the sensitivity was 0.88, the specificity was 0.92, and the diagnostic precision was 0.91 (Table 2). The area under the ROC curve was 0.94. Using the alternate definition of progression that required no confirmations, the best threshold was found at 0.23, the sensitivity was 0.82, the sensitivity was 0.88, and the diagnostic precision was 0.86. The area of the ROC curve in this case was 0.90. The ROC curves are shown in Figure 5. Again, no significant differences were found between the two ROC curves (P = .70).17

View Large Image Download to PowerPoint

FIGURE 5. Receiver operator characteristic curves for 106 eyes from 80 patients for two definitions of progression.

The input weights after each training session were recorded for the purpose of determining whether certain processing elements in the first layer, corresponding to specific visual field locations, were given more weight than others. None of the locations was statistically different from others, and no consistent patterns were seen.

Discussion 

Physician decision making is often surrounded by a degree of uncertainty, which arises in part from noise in diagnostic tests. A physician must be able to process several pieces of information at once and attempt to distinguish the signal (true diagnosis) from the noise (random fluctuation). This uncertainty is especially large when following a glaucoma patient. In the visual field aspect of glaucoma, there are at least two reasons for the uncertainty: the lack of objectivity in evaluating visual fields and random test-to-test fluctuations. The AGIS investigators sought to decrease the uncertainty by developing an objective system of ranking the severity of glaucoma, which also objectified a system of determining progression of visual field loss. The current study investigated the capability of a neural network to discriminate between stable and progressive visual fields, as defined by the AGIS scoring system, given only visual field threshold information.

The neural network performance was promising and comparable to that of the AGIS method of determining progression (FIGURE 3, FIGURE 4). Sensitivities and specificities were high with the data from 80 eyes, as well as all 106 eyes (Table 2). The results indicate that the neural network could correctly identify a high percentage of the progressing and stable visual fields. The results suggest that the neural network was fairly successful in distinguishing true decreases in visual field from noise that did not reflect true progression, as defined by the AGIS investigators.

Even though the AGIS score is based on patterns in the visual field and the neural network presumably is able to detect these patterns, the neural network seemed to treat all of the visual field locations roughly equally. In the input layer, no processing element or group of elements corresponding to a certain visual field region was given more weight. One possible explanation may be the fact that there are many ways for a visual field to be classified as progressing. For example, many combinations of nasal defect, nasal step, and hemifield defects can yield an AGIS score of five. Many more combinations are thus possible for getting a difference in AGIS score of four or more. Defects from all over the visual field can contribute, thus, washing out the effect of any one location or cluster of locations.

Running the neural network with the alternate definition of progression (an increase in AGIS score of four or more, regardless of the results of subsequent tests) still yielded high accuracy comparable to the results using the original definition. It suggests that verifying a clinically significant change in AGIS score may not be necessary for labeling an eye as having a progressing visual field. This result is difficult to interpret, because the neural network inputs consisted only of data from two tests: the initial and a subsequent one. The neural network was not given any information about the temporal relationships of the tests. However, the result does suggest that the AGIS criteria may be somewhat too conservative in their definition of progression. Relaxing the definition may still reveal most of the progressing visual fields but may increase the risk of falsely identifying noise as progression. The major limitation to this conclusion is that there is no real standard or definition of the progression of glaucoma with visual field data.

There is little previous research with neural networks and visual field progression. In our study, the neural network was able to identify visual field progression according to the AGIS system without being given the AGIS scores and criteria. It successfully patterned the AGIS system of scoring, which was derived from the AGIS investigators’ knowledge and clinical experience. It may prove to be valuable to investigate visual fields with lower AGIS progression scores to discern subtle progression missed by AGIS. Further research will also be conducted with neural networks to investigate other methods of quantifying visual field progression. While currently there is little direct clinical application of these new results, they enhance the body of knowledge concerning finding ways to identify and quantify glaucomatous visual field progression.