During exercise, pulse oximetry is problematic due to motion artifact and altered digital perfusion. New pulse oximeter technology addresses these issues and may offer improved performance. We simultaneously compared Nellcor N-395 (Oxismart XLTM) pulse oximeters with an RS-10 forehead sensor (RS-10), a D-25 digit sensor (D-25), and the Ivy 2000 (Masimo SETTM)/LNOP-Adt digit sensor (Ivy) to arterial blood oxygen saturation (SaO2) by cooximetry. Nine normal subjects, six athletes, and four patients with chronic disease exercised to maximum oxygen consumption (V˙o 2 max) under various conditions [normoxia, hypoxia inspired oxygen fraction (Fi O2) = 0.125; hyperoxia, Fi O2 = 1.0]. Regression analysis for normoxia and hypoxic data was performed (n = 161 observations, SaO2 = 73–99.9%), and bias (B) and precision (P) were calculated. RS10 offered greater validity than the other two devices tested (y = 1.009x − 0.52, R 2 = 0.90, B±P = 0.3 ± 2.5). Finger sensors had low precision and a significant negative bias (D-25: y = 1.004x − 2.327, R 2 = 0.52, B±P = −2.0 ± 7.3; Ivy: y = 1.237x − 24.2, R 2 = 0.78, B±P = −2.0 ± 5.2). Eliminating measurements in which heart rate differed by >10 beats/min from the electrocardiogram value improved precision minimally and did not affect bias substantially (B±P = 0.5 ± 2.0, −1.8 ± 8.4, and −1.25±4.33 for RS-10, D-25, and Ivy, respectively). Signal detection algorithms and pulse oximeter were identical between RS-10 and D-25; thus the improved performance of the forehead sensor is likely because of sensor location. RS-10 should be considered for exercise testing in which pulse oximetry is desirable.
- oxygen saturation
- normal subjects
during exercise testing, it is often desirable to monitor arterial oxygen saturation of hemoglobin (SaO2), especially in the context of pulmonary or cardiovascular disease or when subjects are breathing hypoxic gas mixtures as part of research protocols. Pulse oximeters, because they are noninvasive and obviate the need for arterial catheterization, are often used in this context. In addition, many researchers investigating pulmonary gas exchange during exercise utilize pulse oximetry either to prescreen research subjects before more invasive studies or in place of direct measurement of SaO2 using cooximetry (3, 7, 8, 11).
Pulse oximeters use a light source and photodiode light detector to measure the amount of light passing through an arteriolar bed. SaO2 can be estimated noninvasively because the light-absorbing characteristics of hemoglobin differ between oxyhemoglobin and deoxyhemoglobin. Although well accepted for use in resting subjects, using pulse oximetry during exercise for accurate measurement of SaO2 has been problematic for several reasons. First, depending on the sensor site, sensors are subjected to varying degrees of motion resulting in signal corruption and thus inaccurate estimations of saturation (9). Furthermore, sensors placed on the digits are even more susceptible to this problem during cycle exercise because gripping the handlebars results in weakening or even complete loss of signals (17, 19). Recently developed pulse oximeters offer potential advantages because they utilize advanced signal-processing methodologies in an attempt to provide continuous and accurate measurements of oxygen saturation when signals are weak (e.g., low perfusion) or corrupted by motion artifact (1, 2, 15).
The purpose of this study was to evaluate new pulse oximeter technologies during exercise in a variety of study populations. We tested two devices: the Ivy 2000 (Masimo, Irvine, CA) with the LNOP-Adt finger sensor and two Nellcor N-395 (Oxismart XL, Mallinckrodt, St. Louis, MO) pulse oximeters equipped with either a RS-10 forehead sensor or a D-25 digit sensor. These devices were used in normal subjects, athletes, and patients with either chronic obstructive pulmonary disease or chronic heart failure. We hypothesized that the combination of motion-tolerant pulse oximeter and RS-10 forehead sensor, because of the previously mentioned problems with motion artifact and digital perfusion during exercise, would provide the most valid noninvasive measure of SaO2. We chose to compare athletes and patients with normal subjects because monitoring SaO2 in these two groups is likely to be problematic for the athletes because of poor signal detection caused by the diversion of large amounts of blood to working muscles during high-intensity exercise and for the patients because of a potential circulatory compromise caused by chronic disease.
This study was approved by the Human Subjects Committee of the University of California, San Diego. Nineteen subjects were recruited by advertisement and, after giving written informed consent, agreed to further study. Subjects belonged to one of the following three groups.Group 1 was healthy active nonsmoking adults (n = 9), group 2 was healthy nonsmoking competitive cyclists (n = 6), and group 3(n = 4) was patients with either chronic heart failure or chronic obstructive pulmonary disease.
A screening history and physical examination was performed, and female subjects were screened for pregnancy. Maximal oxygen consumption (V˙o 2 max) was determined on an electronically braked cycle ergometer (Excaliber, Quinton Instruments, Gronigen, Netherlands). After a 5-min warm up at 25–100 W, groups 1 and 2 rode a progressive exercise test (25–30 W/min) until they were unable to continue.Group 3 protocol was similar except that the work rate increment was 10–20 W/min. Heart rate was monitored by cardiac monitor (Lifepak 6, Physio-control, Redmond, WA). Subjects breathed through a nonrebreathing valve (2700, Hans-Rudolph, Kansas City, MO). Expired gas was sampled continuously from a heated mixing chamber, and oxygen and carbon dioxide concentrations were measured (mass spectrometer-1100, Perkin-Elmer, Pomona, CA). Expired gas flow was measured by using a pneumotach (no. 3 Fleisch) and differential pressure transducer (Validyne, DP45-14, Northridge, CA), and electrical signals from the mass spectrometer and pneumotach were logged at 100 Hz by using a 12-bit analog-to-digital converter. Ventilation (V˙e) oxygen consumption (V˙o 2) and carbon dioxide production (V˙co 2) were calculated by using a commercially available software package (Consentius Technologies, Salt Lake, UT). V˙o 2 max was calculated as the average of the four highest consecutive 15-s measurements ofV˙o 2.
All subjects fulfilled at least two of the following four criteria forV˙o 2 max: 1) heart rate ≥ age predicted maximum; 2) respiratory exchange ratio > 1.10; 3) no further increase or a decrease inV˙o 2 with increasing workload; and4) no further increase in heart rate despite an increase in workload. Group 1 also underwent a similar protocol, breathing 12% oxygen. The order of the two exercise tests was balanced in group 1, and subjects rested for ∼1 h between tests.
Results of these preliminary tests were used to select workloads that represented 30, 60, 90, and 100% ofV˙o 2 max in groups 1 and2, and 25, 50, 75, 90, and 100% ofV˙o 2 max in group 3.
Under continuous electrocardiogram (ECG) monitoring (Lifepak 6), a 20-gauge arterial cannula was placed in the radial artery of the nondominant hand by using a sterile technique. We followed the instruction as supplied by the manufacturer, attaching the Ivy 2000 with Masimo's LNOP-Adt sensor and the Nellcor N-395 with the D-25 digit sensor (N-395/D-25) to the digits of opposite hands. A light-opaque shield was lightly taped over the digit to exclude interfering ambient light. Digit placement (digits 2,3, and 4) and hand selection (right or left) were randomized between subjects. The Nellcor N-395 with a RS-10 forehead sensor (N-395/RS-10) was positioned over the left eyebrow on the forehead and secured with a terrycloth sweatband.
Data collection protocol.
Each subject was seated on the cycle ergometer and breathed through the mouthpiece for 10 min before the start of resting measurements. Data collection consisted of duplicate 3-ml samples of arterial blood, which were collected at rest and in the last 2 min of each exercise level (30, 60, 90, and, in some subjects, 100% ofV˙o 2 max in groups 1 and2 and 25, 50, 75, and, in some subjects, 90–100% ofV˙o 2 max in group 3).Group 1 performed the exercise protocol in normoxia and while breathing hypoxic gas (inspired oxygen fraction = 0.12),group 2 exercised in normoxia only, and group 3exercised in both normoxia and 100% oxygen. SaO2 by pulse oximetry was obtained by interfacing the three devices with a portable computer and recording data over a 30-s period simultaneously from all three devices and synchronously with arterial blood sampling. Data in which poor signal detection was evident (heart rate deviated 10 or more beats/min from that measured by ECG) were identified so that analyses could be conducted with and without these data points.
Arterial samples were maintained on ice until analyzed for hemoglobin concentration and SaO2 using an IL 682 cooximeter (Instrumentation Laboratories, Lexington, MA). Blood gas analyses were completed within 30 min of data collection. SaO2 was measured as where FO2Hb is the oxyhemoglobin fraction, FCOHb is the carboxyhemoglobin fraction, and FmetHb is the methemoglobin fraction.
We compared measured SaO2 by cooximetry to each of the pulse oximeters by using linear regression (Statview 5.0, SAS, Cary, NC). Prediction limits of ±95% for the prediction of SaO2 by pulse oximeter as a function of measured SaO2 were generated for each device. The ±95% prediction limits can be thought of as the ±2 SD limits of the least squares regression line at any point along the regression. To examine the effect of poor signal detection, heart rate by ECG was compared with heart rate by pulse oximetry, and separate regression analyses were performed after eliminating all SaO2 data in which the simultaneously obtained heart rate deviated by ≥10 beats/min from that measured by ECG. In addition, data obtained while breathing 100% oxygen are on the flat part of the oxyhemoglobin equilibrium curve. These data were not included as part of the regression analyses and were analyzed separately. Bias [or mean error; calculated as (SaO2 pulse-oximetry − SaO2cooximetry)/n−1] and precision (SD of SaO2 pulse-oximetry − SaO2cooximetry; the smaller the SD, the greater the precision) were calculated for each device. As for the regression analyses, these parameters were calculated for all data points and also after eliminating data points in which there were poor pulse rate signal detections. Significance was accepted at P < 0.05, two-tailed. Data are presented as means ± SD.
Subject descriptive data are given in Table1. As expected, athletes had a greaterV˙o 2 max than healthy normal subjects (group 1; P < 0.005) or patients (group 2; P < 0.0001). Patients were significantly older and heavier than either the athletes (P < 0.001) or the healthy normal subjects (P < 0.001).
SaO2 measured by pulse oximetry during exercise in normoxia and hypoxia is compared with results obtained by direct arterial blood measurements in Fig. 1 and Table 2. Figure 1 shows the correlation between SaO2 measured by cooximetry and by pulse oximetry for each device for all subjects. All three devices showed significant correlations between cooximetry and pulse oximetry values. However, there were considerable differences between devices. The N-395/RS-10 forehead SaO2 (Fig.2 A) was very highly correlated with SaO2 measured by using cooximetry (R 2 = 0.90, P < 0.0001), and values obtained were centered around the line of identity (slope = 1.009, intercept = −0.52). Poor pulse-rate or arterial signal detection was evident in 13 (8.1%) measurements of SaO2 and, when these data were eliminated,R 2 increased to 0.94, although slope and intercept of the relationship did not change appreciably (slope = 0.998, intercept = 0.65).
Correlation for the Ivy 2000 finger sensor was not as close (Fig.2 B; R 2 = 0.78, P< 0.0001), and there was a tendency for the device to underestimate SaO2, particularly under hypoxic conditions, (slope = 1.23, intercept = −24.2). Many of the measurements suffered from poor pulse-rate signal detection (76 measurements; 48.4%), and eliminating these measurements increasedR 2 to 0.87. However, even with this modification, slope of the relationship was unchanged (1.20) and intercept was still markedly negative (−19.9). Thus adequate pulse-rate signal detection does not prevent the problem of underestimating SaO2 in this device.
The N-395/D-25 finger sensor utilized the same pulse oximeter tested for the N-395/RS-10 but provided data that were much less closely correlated with SaO2 measured by cooximetry (Fig.2 C; R 2 = 0.52, P< 0.0001). Although the N-395/D-25 finger sensor underestimated SaO2 at all levels, measurements were not greater during hypoxia than during normoxia (slope = 1.004, intercept =−2.32). Poor pulse-rate signal detection was present in 43 (26.9%) measurements, and removal of these data from the regression improved the strength of the relationship substantially (R 2 = 0.87) but did not significantly alter slope and intercept of the relationship (1.04 and −3.30, respectively).
Bias and precision of the SaO2 measurement from the three devices compared with the cooximeter are given in Table 2 and Fig. 2. Averaged over all subjects, the N-395/RS-10 forehead device had significantly lower bias and greater precision than the two finger probes (precision = 2.5 for N-395/RS-10 vs. 5.2 and 7.3 for IVY 2000 and N-395/D-25, respectively). Eliminating data points with poor pulse-rate signal detection had a minimal effect on these values (precision = 2.0 for N-395/RS-10 vs. 4.3 and 8.4 for IVY 2000 and N-395/D-25, respectively). The N-395/D-25 and N-395/RS-10 performed similarly in all three subject groups, although there were minor differences between athletes and patients for the N-395/RS-10. However, the IVY 2000 was significantly worse in group 3 compared with athletes and normal subjects.
Bias and precision SaO2 data from patients exercising in 100% oxygen are presented in Table 3. In this data set, SaO2 approaches 100% [all partial pressure of oxygen (PaO2) values were >500 Torr]. This approach essentially eliminates any variation in the independent variable (i.e., SaO2 measured by cooximetry). Therefore, any deviation of SaO2 measured by the three devices is related to issues such as perfusion and/or signal detection. Under these conditions, the N-395/RS-10 forehead sensor had significantly less bias and greater precision than the other two devices (P < 0.05).
Bias and precision of the heart rate data, which can be used as an index of the adequacy of pulse-rate signal detection for the different devices, are presented in Table 4. Averaged across all subject groups, there were significant differences between devices. The N-395/RS-10 had significantly less bias and greater precision of heart rate measurements compared with the other two devices (P < 0.001), and the N-395/D-25 demonstrated significantly less bias compared with the Ivy 2000 (P < 0.001). There were no significant differences between subject groups for the N-395/RS-10. Both the N-395/D-25 and Ivy 2000 significantly underestimated the heart rate compared with the ECG measure of heart rate. There were no significant differences in bias between subject groups for the N-395/D-25, but the Ivy 2000 had significantly greater bias in the athletes than in the other two subject groups.
The N-395/RS-10 offered greater validity of SaO2and heart rate measurements under all conditions (normoxia, hypoxia, hyperoxia) and in all subject groups (normal subjects, athletes, patients) than the other two devices tested. This is reflected by a higher correlation between SaO2 measured by this device and that measured by cooximetry, a smaller ±95% prediction limit, less bias, greater precision, and fewer data points with poor pulse-rate signal detection. This is likely due to the position of the sensor, because the identical pulse oximeter equipped with a digital sensor (N-395/D-25) performed substantially worse than both the forehead sensor and the IVY 2000 pulse oximeter equipped with a finger sensor. In general, the N-395/RS-10 performed similarly in all three subject groups, as did the N-395/D-25, albeit with more bias and less precision than the N-395/RS-10. However, the Ivy 2000 performed significantly worse in the measurement of SaO2 in the patients we studied than it did in the normal subjects or athletes.
There have been many validation studies of pulse oximetry during exercise over the last 20 years, with widely varying conclusions offered on the part of the authors. For example, Powers et al. (10) tested three devices (two finger sensors and one ear sensor) and found standard error of estimates (SEE; numerically similar to precision) ranging from 1.43 to 1.97%, similar to values we found with the N-395/RS-10 forehead sensor. These authors concluded that the accuracy of pulse oximetry was sufficient for use during exercise testing. However, Symth et al. (14) found a SEE of 4.08 and 5.39% in the two devices they evaluated during exercise in normoxia and hypoxia (one pulse oximeter equipped with an ear probe and the HP 47201A ear oximeter, which analyzes light from eight wavelengths).
Exercise induces potential problems related to the accuracy of pulse oximetry. For example, motion artifact can interfere with arterial signal detection, and new generation pulse oximeters such as the Masimo Ivy 2000 and the Nellcor Oxismart incorporate advanced signal processing to deal with nonstandard signal detection (1, 2,15). In addition, poor perfusion states, such as those that occur with cool skin temperature, may induce significant bias (18). However, as can be appreciated from Figs. 1 and 2, significant measurement errors occurred even when adequate pulse-rate signal detection was evident. Conversely, especially with the N-395/RS-10, many data points with poor pulse-rate signal detection still provided reliable data. Thus this criterion alone cannot be used to judge the quality of the data obtained by using these devices. An estimate of the extent that a particular device underestimates SaO2 may be obtained by the administration of several breaths of hyperoxic gas mixtures (inspired oxygen fraction = 0.5–1.0) in the final few seconds of an exercise test. This will elevate SaO2 to ∼100%, and any systematic bias by the device will become apparent. As can be seen in Table 3, the two devices using finger probes substantially underestimated SaO2 in patients breathing 100% oxygen by an average of ∼5%, which was similar to their performance compared with cooximetry during normoxia.
We wish to emphasize that our conclusions are strongly influenced by the setting in which the device was used. Clearly, pulse oximeters offer several advantages in the clinical setting. These devices are noninvasive, easy to use, and do not require significant analysis time or maintenance of other equipment to obtain data. In addition, the nature of the bias of the devices we tested is such that, when significant errors are present, the tendency is toward underestimating the true SaO2. Consequently, pulse oximetry offers a conservative estimate of SaO2 during clinical exercise testing and thus is likely suitable for safety monitoring.
In addition to concerns related to accuracy during exercise, which we have briefly outlined, there are other issues to consider, such as sensor location. Finger sensors may be easier to place and to maintain in position in some patients. The forehead sensor offers a potential advantage in that it avoids the digits and the severe effects caused by gripping and motion. Also, the forehead site may be better than the earlobe, as signals are usually greater and the sensor can be easily secured and held in place with the addition of a headband. However, the signal obtained from forehead sensors is susceptible to contamination from venous blood and consequently will be biased toward low readings if central venous pressure is raised (12, 16). This was not observed in the present study, even in patients with heart failure. However, it may be more of a problem with certain types of exercise, such as rowing, in which a Valsalva maneuver is performed at the start of the stroke. However, this may in part be prevented by the use of a compressive headband, such as the one used in the present study (4). The forehead sensor location may also interfere with secure placement of some types of headsets that support respiratory mouth pieces and vice versa. Finally, in one of our subjects in one instance, poor arterial signal acquisition occurred when the subject altered his facial expression during maximal exercise. This was accompanied by a loss of the pulse-rate signal and a markedly erroneous data point. As can been seen from Fig. 1, a similarly erroneous data point also occurred with no obvious loss of pulse-rate signal, and it is not possible, post facto, to determine the source of this error. Potential users of these devices should also should be aware that significant differences in the time to detect the onset of hypoxia have been reported with different sensor locations (6). Because we tested healthy subjects and patients who were ambulatory and able to complete an exercise test, we cannot extend our findings to clinical situations other than exercise testing. It is possible that some devices using an ear sensor may offer similar performance to the forehead sensor, as these might be less susceptible to motion artifact and poor perfusion than finger sensors. However, because we did not test any ear sensors, we cannot comment on their performance.
When precise measurement of oxygen transport is important, such as in answering research questions, measurement of SaO2alone is probably not adequate, particularly during normoxic exercise, in which relatively small changes in SaO2 are associated with large differences in PaO2. For example, although the mean bias for the N-395/RS-10 for all patient groups was 0.3% SaO2, the precision was ±2.5% SaO2. With the use of a standard oxyhemoglobin equilibrium curve at a pH of 7.4, a PaO2 of 40 Torr and a temperature of 37°C, ±2.5% SaO2 encompasses a variation of almost 40 Torr in PaO2, assuming a normal resting PaO2 of 90 Torr. (13). Additionally, during exercise in which SaO2 is also affected by temperature and pH, typical changes observed in maximal exercise, such as a reduction in pH from 7.4 to 7.2 and an increase in the temperature from 37 to 39.5°C, result in a ∼4% decrement in SaO2 in the absence of any change in PaO2.
Recently, Dempsey and Wagner (5) have offered a standardized definition of mild exercise-induced arterial hypoxemia as a fall in SaO2 to below 95% from a normal resting value of 98%. This value is very close to the precision of the most accurate device tested (N-395/RS-10) and for the two finger-probe devices, both of which had an average bias of −2.0%. Many data points could be expected to fit this definition on the basis of nonrandom measurement error alone. Ideally, if circumstances dictated the use of pulse oximetry, it would be desirable to validate the pulse oximetry device against direct measures on arterial blood in a representative sample of the study population.
In conclusion, the N-395/RS-10 forehead sensor offered greater validity of SaO2 measurements under all conditions and in all subject groups than the other two digital oximeters tested. Bias was negligible, however, and precision was ±2.5%. Both finger sensors showed significant negative bias and underestimated SaO2 during exercise. They also showed low precision (>5%). Eliminating data points with poor pulse-rate signal acquisition, as evidenced by an error in heart rate measurement, did not substantially improve the performance of these devices.
We would like to thank our subjects for their enthusiastic participation and Harrieth Wagner, Nick Busan, and Jeff Struthers for technical assistance.
This study was supported by National Center for Research Resources Grant MO1 RR-00827, National Heart, Lung, and Blood Institute Grant HL-17731, and Mallinckrodt.
Address for reprint requests and other correspondence: S. R. Hopkins, Univ. of California, San Diego, Dept. of Medicine-0623, 9500 Gilman Dr., La Jolla, CA 92093-0623 (E-mail:).
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
- Copyright © 2002 the American Physiological Society