Estimating the Parameters of Degradation Models when Error Terms are Autocorrelated Jehad

The need of autocorrelation models for degradation data comes from the facts that the degradation measurements are often correlated, since such measurements are taken over time. Time series can exhibit autocorrelation caused by modeling error or cyclic changes in ambient conditions in the measurement errors or in degradation process itself. Generally, autocorrelation becomes stronger when the times between measurements are relatively short and becomes less noticeable when the times between process are longer. In this paper, we assume that the error terms are autocorrelated and have an autoregressive of order one, AR(1). This case is a more general case of the assumption that the error terms are identically and independently normally distributed. Since when the error terms are uncorrelated over the time, the estimate of the parameter of AR(1) is approximately zero. If the parameter of AR(1) is unknown, one can estimate it from the data set. Using two real data sets, the model parameters are estimated and compared with the case when the error terms are independent and identically distributed. Such computations are available by using procedures AUTOREG and model in SAS. Computations show that an AR(1) can be used as a useful tool to remove the autocorrelation between the residuals. Zusammenfassung: Der Bedarf von Autokorrelationsmodellen für Abbaudaten kommt von der Tatsache, dass Abbaumessungen oft korreliert sind, da solche Messungen über die Zeit hinweg genommen sind. Zeitreihen können eine Autokorrelation aufweisen, verursacht durch Modellfehler oder zyklische Änderungen in den Umgebungsbedingungen in den Messfehlern oder im Abbauverfahren selbst. Im Allgemeinen wird die Autokorrelation stärker, wenn die Zeiten zwischen den Messungen verhältnismäßig kurz sind, und sie wird weniger deutlich, wenn die Zeiten während des Ablaufs länger sind. In dieser Arbeit nehmen wir an, dass die Fehlerterme autokorreliert sind und einem autoregressiven Prozess der Ordnung 1, AR(1), unterliegen. Dieser Fall ist ein allgemeinerer Fall der Annahme, dass die Fehlerterme identisch und unabhängig normalverteilt sind. Sind die Fehlerterme unkorreliert über die Zeit, dann ist die Schätzung des Parameters im AR(1)-Modell ungefähr Null. Fall der Parameter im AR(1)-Modell unbekannt ist, kann man ihn aus den Daten schätzen. Wir verwenden zwei reale Datensätze und schätzen die Modellparameter und vergleichen sie dann mit dem Fall wenn die Fehlerterme unabhängig und identisch verteilt sind. Solche Berechnungen stehen zur Verfügung durch Benutzen der Prozeduren AUTOREG und Modell in SAS. 192 Austrian Journal of Statistics, Vol. 40 (2011), No. 3, 191–200 Die Berechnungen zeigen, dass ein AR(1)-Modell verwendet werden kann, als nützliches Werkzeug, die Autokorrelation zwischen den Residuen zu entfernen.

If the parameter of AR( 1) is unknown, one can estimate it from the data set.Using two real data sets, the model parameters are estimated and compared with the case when the error terms are independent and identically distributed.Such computations are available by using procedures AUTOREG and model in SAS.Computations show that an AR(1) can be used as a useful tool to remove the autocorrelation between the residuals.

Introduction
With today's high technology many products are designed to work without failures for many years.Thus it is very difficult to analyze the time data with traditional reliability analysis.A recent approach assess products reliability using degradation measurements of product performance has to pre-specify a level of a degradation and obtain measurements at different times.Thus the time-to-failure is defined as the time when the degradation of a unit reaches a critical level.
In this literature, Lu and Meeker (1993) considered a non-linear mixed effect model and used two-stage method to estimate the model parameters under the assumption that the errors term are independent and identically distributed and the autocorrelation is negligible.They applied their model to fatigue crack growth data.Meeker and Hamda (1995) gave a statistical tools and concepts that are used in designing a high reliability product using degradation data.Meeker and Escobar (1998) described accelerated degradation models that are related to physical failure mechanisms.Lu, Meeker, and Escobar (1996) compared degradation analysis and traditional failure time analysis.They showed that degradation analysis provides more precision than the traditional failure time analysis.Wu and Shao (1999) established the asymptotic properties of the ordinary and weighted least squares estimators under the non-linear mixedeffect degradation model.Oliveria and Colosimo (2004) compared between three estimation methods to estimate the time-to-failure distribution, namely analytical, approximate, and numerical methods.
To assume independent and identically distributed random errors in a degradation model is not a good idea in general.Since the observed degradation on a specimen over time is a time series and a time series data can exhibit autocorrelation caused by modeling errors or by cyclic changes in ambient conditions in the measurement errors or in degradation process itself.
In this paper, we will assume that the errors in degradation model are autocorrelated and has AR(1).The model parameters are estimated and compared with the case when the errors term are independent and identically distributed.
This paper is organized as follows: In Section 2, the general degradation model assuming the error terms has AR(1) is presented.In Section 3, applications to real data sets and results are discussed.Our conclusions and recommendations are given in Section 4.

Degradation Model with AR(1) Error Terms
The general degradation model commonly used in reliability analysis has the following form: where y ij are the observed degradation measurements of the ith unit and jth measurement, t ij is the time of the jth measurement for the ith unit, (β i0 , β i1 , . . ., β ip ) is a vector of the (p + 1) unknown parameters for the ith unit, some of these parameters are random from unit to unit and some are fixed, D(t ij , β i0 , β i1 , . . ., β ip ) is the actual path of the ith, n is the number of tested units, m i is the total number of inspections of unit i, and ε ij are random errors assumed to have an AR(1), i.e.
where λ ij are random and assumed to be independent and identically (iid) normally distributed with zero mean and variance σ 2 and independently from the random effect parameters.
Remark 1: Model (1) has not a zero mean and can be rewritten as In application, you can subtract the mean from the process.
Remark 2: From equation (2), we can see that, if ϕ i = 0, then we have the iid case.
Remark 3: In this paper, we assumed the error terms are AR(1); however, higher orders AR(p) can be assumed.
For the ith unit, assume model ( 1) is a linear or it can be written in a linear form.Rewrite (1) in the following matrix form: where and To obtain the parameters estimate we modify the first stage of the two stage method suggested by Lu and Meeker (1993).
It can be shown that (Brockwell and Davis, 2003) cov where Since V i is a positive definite matrix, there exists a non-singular matrix T i such that T i T t i = V i .From left, multiplying model (4) by T −1 i , we have where where I (n) is the identity matrix of size n × n and t is the transpose operator.Therefore, So the generalized least squares estimate of β i are given by βGi = ( with cov The ordinary least squares estimate, βsi of β i is given by βsi with cov In general, T i is not unique but the estimate of β i is invariant of the choice of T i .It can be shown that (Rawlings, Pantula, and Dickey, 2001), J. Al-Jararha et al. 195 Remark 4: If ϕ i = 0 and from (13), the generalized least squares estimate of β i is the same as the ordinary least squares estimate, βsi .Fuller (1996), reported (pages 477, and 519) that, the variance of any linear contrast, λ t βGi is less than or equal to the variance of the corresponding linear contrast λ t βsi ; further, the ordinary least squares estimate will be an over or under estimate of the true variance.
If ϕ i is unknown, then it can be estimated from the data set; in this case, we can replace ϕ i by φi in the covariance matrix, V i , in the above discussions.Fuller (1996), Theorem 9.7.1, gives an asymptotic results supporting such situations.The computations can be implemented by using the procedure AUTOREG of SAS Institute.
When model ( 1) is a non-linear model, Gallant (1987) and Seber and Wild (1989) described methods for estimating the model parameters with autocorrelated errors.We will not describe such theories since such computations can be implemented by using procedure model of SAS Institute.

Real Data Applications and Results
Two real data sets will show that using AR(1) for an error terms can be used to remove the autocorrelation between the residuals even when φi are slightly differ from zero.One can note that, in the above discussions, if ϕ i = 0 then the two estimators are obtained from equations ( 11) and ( 12) are equivalent.

Laser Data
Consider the Laser data from Meeker and Escobar (1998), Table C.17, page 642.From Figure 13.14, page 338 of Meeker and Escobar (1998), it can be assumed that the data can be fitted by the simple linear model The data will be analyzed when the error terms are iid (0, σ 2 ) and when the error terms are AR(1).Table 1 summarizes the results.One can notice that when we are assuming the error terms are AR(1), is actually we are adding one more parameter to the model.When the error terms are iid (0, σ 2 ) for model ( 14), and from Table(1), it is very clear that for the most paths, except path 5 and maybe path 1, the residuals are positively correlated, since the p-value of Durbin-Watson test for testing positive autocorrelation are significant.Such correlations can be removed, for most cases, when the error terms are AR(1).However, the few cases were AR(1) can't remove the correlations between the residuals, in fact, we need an AR(2) or of higher order to overcome such situations.For example, for path 10, we need an AR(2).From Figure 1 we can notice that, from the plot of predicted and actual values of y, all observations are within the 95% confidence bound.Further, the ACF and PACF plots for the residuals are supporting our assumption, i.e. the error terms are AR(1).
In addition to our believes that the degradation data are correlated, one can assume the error terms are AR(1), when the iid (0, σ 2 ) assumption is not a valid condition and without changing the model.

Fatigue Crack Growth Data
From Hudak, Saxena, Bucci, and Malcolm (1978), Lu and Meeker (1993) considered fatigue-crack-growth data as an a motivation example.They fitted the non-linear model when ε ij are iid (0, σ 2 ).From their analysis, the values of r 1 , the 1st order autocorrelation of the residuals, are high for some paths like path 7, 12 and 19, and such autocorrelations are significant from our analysis.Model (15) can be fitted by using procedure model of SAS Institute, when the error terms are either iid (0, σ 2 ) or AR(1).Table 2 gives both cases along with the p-value of Durbin-Watson test.Table 2 shows that the correlation between the residuals for all paths are removed at α = 0.05 when the error terms are AR(1).However, if we look to path (11) and at α = 0.10, for example, we need an AR(2) assumption, the p-values are given in parenthesis.Further, the plot of predicted and actual values of logcrak, Figure 2 shows that all observations are within the 95% confidence bound.Further, the ACF and PACF plots for the residuals are supporting our assumption, i.e. the error terms are AR(1).

Conclusions and Recommendations
In this paper, we assume that the errors in the degradation model are autcorrelated and has AR(1).This assumption covers the case where the error terms are iid (0, σ 2 ).The model parameters are estimated and compared when the error terms has AR(1) and when they are iid (0, σ 2 ).Computations show that an AR(1) can be used as a useful tool to remove the autcorrelation between the residuals.

Figure 1 :
Figure 1: Predicted actual values, ACF and PACF for Path(1) for Laser Data (left) and for Fatigue Crack Growth Data (right).

Table 1 :
Laser data.Comparison between the ordinary least squares estimate and generalized least squares estimate when the error terms are AR(1).Based on the Durbin-Watson test, p +ve is the p-value for testing positive autocorrelation and p −ve is the p-value for testing negative autocorrelation.

Table 2 :
Fatigue crack growth data.Comparison between the ordinary least squares estimate and generalized least squares estimate when the error terms are AR(1).