Artigo erro variância identificação.pdf

8/11/2019 Artigo erro varincia identificao.pdf

1/14

Automatica 45 (2009) 25122525

Contents lists available atScienceDirect

Automatica

journal homepage:www.elsevier.com/locate/automatica

Variance-error quantification for identified poles and zeros

Jonas Mrtensson ,Hkan HjalmarssonACCESS Linnaeus Center, School of Electrical Engineering, KTH Royal Institute of Technology, S-100 44 Stockholm, Sweden

a r t i c l e i n f o

Article history:

Received 11 July 2007

Received in revised form

7 December 2008

Accepted 9 July 2009Available online 10 September 2009

Keywords:

Accuracy of identification

Asymptotic variance expressions

a b s t r a c t

This paper deals with quantification of noise induced errors in identified discrete-time models of causal

linear time-invariant systems, where the model error is described by the asymptotic (in data length)variance of the estimated poles and zeros. The main conclusion is that there is a fundamental difference

in the accuracy of the estimates depending on whether the zeros and poles lie inside or outside the unitcircle. As the model order goes to infinity, the asymptotic variance approaches a finite limit for estimates

of zeros and poles having magnitude larger than one, but for zeros and poles strictly inside the unit circletheasymptotic variance grows exponentially with themodelorder.We analyze howthe variance of polesand zeros is affected by model order, model structure and input excitation. We treat general black-box

model structures including ARMAX and BoxJenkins models.

2009 Elsevier Ltd. All rights reserved.

1. Introduction

Model accuracy is an important issue in system identificationapplications. The experimental conditions, such as excitation

signals, feedback mechanisms, disturbances and measurement

noise,naturally affect theaccuracyof the model,but also the choiceof model structure and model order can have a strong influence.The model error can generally be divided into two parts: bias-errorand variance-error, where the bias-error is dueto un-modeled

dynamics and the variance-error is caused by disturbances whichare modeled as stochastic processes (Ljung,1999;Sderstrm &Stoica, 1989). The focus in this paper is on the variance-error,which will be thedominating part if the model structure is flexibleenough to describe the true underlying dynamics. We will thus

assume that the true system belongs to the model set.The problem of quantifying the expected model error has

received substantial research interest over the past decades. Inparticular, the variance of frequency function estimates has beenstudied extensively. In the mid-eighties Ljung(1985) presented

a variance expression which showed that, for high model orders,the asymptotic variance1 of the frequency function estimate does

The material in this paper was partially presented at the 44th IEEE Conference

on Decision and Control and European Control Conference ECC 2005, 12th15th

December2005 in Seville, Spain, andat the 16thIFAC WorldCongress on Automatic

Control,3rd8thJuly 2005 in Prague,Czech Republic. Thispaperwas recommended

forpublicationin revisedform by AssociateEditorBrett Ninnessunder thedirection

of Editor Torsten Sderstrm. Corresponding author. Tel.: +46 8 7907434; fax: +46 8 7907329.E-mail addresses: [email protected](J. Mrtensson),

[email protected](H. Hjalmarsson).1 Henceforth, asymptotic variancedenotesthe varianceof an estimatedquantity,

normalized by the sample size, as the sample size tends to infinity.

not depend on the model structure but only on the model order.Furthermore, it was shown that the asymptotic variance at aparticular frequency only depends on the ratio of the input andnoise spectra at that particular frequency.

A refined asymptotic variance expression (still asymptotic inmodel order) for frequency function estimates with improvedaccuracy (for many model structures) was proposed in Ninness,Hjalmarsson, and Gustafsson (1999a,b)and expressions that areexact for finite model orders were derived in Ninness andHjalmarsson (2004)and Xieand Ljung (2001, 2004). More recently,a variance expression for finite sample sizes and model ordershas been presented in Hjalmarsson and Ninness (2006). Theoriginal result in Ljung (1985) can also be used for closed loopidentification and this was extended to some alternative closedloop identification methods in Gevers, Ljung, and Van den Hof(2001). The asymptotic variance for parameter estimates in aBoxJenkins model was analyzed inForssell and Ljung(1999)fora whole range of different identification methods. Closed loopasymptotic variance expressions that are exact for finite modelorders are also presented inNinness and Hjalmarsson (2005a,b).

There are numerous other properties of the system, besidesthe frequency function, that could be of interest to estimate.Recently, a geometric interpretation of the asymptotic variance hasbeen developed (Hjalmarsson & Mrtensson, 2007)and, when theunderlying system is causal, linear and time-invariant (LTI), thesetechniques have been used to derive more transparent expressionsfor the asymptotic variance for estimates of system quantitiesthat can be represented by smooth functions of the estimatedparameters (Mrtensson & Hjalmarsson, 2007). In this paper thefocus is on poles and zeros of causal LTI systems. Particularlyinteresting from a control perspective are unstable poles and non-minimum phase zeros, i.e. poles and zeros outside the unit circle,since they pose fundamental performance limitation on the closedloop system(Skogestad & Postlethwaite, 1996).

0005-1098/$ see front matter 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.automatica.2009.08.001
http://www.elsevier.com/locate/automaticahttp://www.elsevier.com/locate/automaticamailto:[email protected]:[email protected]://dx.doi.org/10.1016/j.automatica.2009.08.001http://dx.doi.org/10.1016/j.automatica.2009.08.001mailto:[email protected]:[email protected]://www.elsevier.com/locate/automaticahttp://www.elsevier.com/locate/automatica


2/14

J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525 2513

A series of results regarding the asymptotic variance ofpoles and zeros has been presented over the past years. Non-

minimum phase zeros were treated for different model structuresin Hjalmarsson and Lindqvist (2002), Lindqvist (2001) andMrtensson and Hjalmarsson(2003) where it was found that theasymptotic variance approaches a finitelimit as the model ordergoes to infinity. For ARX models these results were extended

to closed loop identification in Mrtensson and Hjalmarsson(2005a), where also similar results were presented for poleswith magnitude larger than one. Asymptotic variance expressionsthat are exact for finite model orders were derived for zeroswith arbitrary location in Mrtensson and Hjalmarsson(2005b).

It should be noted that the asymptotic variance can be a poormeasure of the variance for finite sample size, see Vuerinckx,Pintelon, Schoukens, and Rolain (2001) where also a more accuratemethod to construct finite sample size confidence bounds forestimated zeros and poles is presented.

In this paper,the results on the asymptotic variance of poles and

zeros referred to above are combined and generalized to a unifiedtheory on the asymptotic variance of poles and zeros of estimatedcausal LTI systems. We put emphasis on presenting, analyzingand interpreting the results, but the derivations (proofs) of the

variance expressions are also included to make the document self-contained. The results hold for prediction error identification.

Outline

This paper is organized in the following way. In Section2 theproblem set-up, the standing assumptions, that will hold through-out the paper, and some notation are introduced. The main tech-

nical results are presented in Section 3. The asymptotic variance ofestimated zeros and poles located in the open unit disc|z| < 1 isdiscussed in Section 4 which is followed by Section 5 covering poleswith magnitude larger than one. Section6covers zeros with mag-nitude at least one and in Section7 some conclusions are drawnbased on the analysis of the variance expressions and some simu-

lations arepresented to investigate theaccuracyof theexpressions.Section 8 contains a brief summary of the findings and Appendix Acontains results that are needed for the proofs of the paper.

Notation

The multiplicity of a root zo of a polynomial A(z)is defined asthe integerk for which the limit limzzo A(z)/(zzo)k = c= 0exists. Two polynomials are said to be coprime if they have nocommon roots. A rational transfer function Q = Qn/Qd, whereQn and Qdare coprime polynomials in z

1, is said to be minimumphase ifQn(z)= 0 or Qd(z)= 0 implies|z| < 1. A zerozo ofQ issaid to be non-minimum phase (NMP) if|zo| > 1.Qis said to bestable ifQd(z)= 0 implies|z| < 1. A rootzp to Qd(z)= 0 with|zp|> 1 is said to be an unstable pole ofQ.

For a transfer function Z(q)with a zero at zo, we will use the

notationZ(q)=Z(q)/(1 zoq1). Itwill be clear from the contextwhich zerozo that is used.

For a rational transfer functionG,Gdenotes a transfer functionwhich satisfies|G(ej)| = |G(ej)| for all for which G(ej) isdefined and which has all its zeros and poles in the closed unitdisc|z| 1. That such a function exists follows from the spectralfactorization theorem.

We shall consider vector-valued complex functions as row

vectors and the inner product of two such functions f(z),g(z):C C1m is defined asf,g = 1

2

f(e

j)g(ej) d whereg denotes the complex conjugate transpose ofg. When fandgare matrix-valued functions, we will still use the notation

f,g to denote the integral 1

2 f(ej)g(ej) d whenever thedimensions off andgare compatible. The space L2consists of all

Fig. 1. Block diagram of SISO LTI system with output feedback.

functionsf: C C1m such thatf,f


3/14

2514 J. Mrtensson, H. Hjalmarsson / Automatica 45 (2009) 25122525

noise with unit variance, and where we assume that the feedbackcontroller K stabilizes the system. When this system is modeled as

yt= G(q, )ut+H(q, )et,with G and Hgiven by (1), and the parameter vector is estimatedusing prediction error identification with a least-squares costfunction, the asymptotic covariance matrix for the parameter

estimate

N (Nis the sample size, i.e. the number of input/output

measurements) is given by

P=o1,where

= 12

(ej)(ej) d , (4)

where in turn

(q)=

Go

Ho

So

AoRna (q)

So

Ao

o na (q)

GoHo

So

BoRnb (q)

KGoSo

Bo

o nb (q)

0 1Co

o nc(q)

0 1

Do

o nd (q) Go

Ho

So

FoRnf (q)

KGoSo

Fo

o nf (q)

, (5)

where n(q)= [q1, . . . , qn]T, and whereSo

1

1+GoKis the sensitivity function.

It can be shown also, using Gauss approximation formula, thatthe asymptotic variance of an estimate of a root zo of one of thepolynomialsAoFoof the true system is given by

AsVarzo P, (6)

where

dz( )

d

=o

.

For exact conditions for the above asymptotic results to hold werefer to Chapters 89 inLjung(1999).

Since(6)implies the approximation

E|z(N) zo|2 AsVarzo/N (7)for the finite sample size variance of an estimate z(N) of a root ofone of the polynomialsAoFo, AsVarz

o provides information aboutthe variance of root estimates for large sample sizes. The objectiveof this paper is to analyze the right-hand side of(6) and we willmake the following assumptions which will hold throughout the

paper.

Standing Assumptions 2.1. The polynomials Co, Do and Fo areminimum phase. The rational transfer function R is minimumphase. WithK= Kn/Kd , whereKnandKdare coprime polynomialsinz1, it holds that

Acl=AFKd+BKnhas all its roots strictly inside the unit circle. The matrix , definedin(4),is non-singular and the noise variance is positive,o > 0.Finally, all results are valid only for roots of multiplicity 1.

The conditions above are standard in a system identificationcontext, cf.Ljung(1999). The minimum phase conditions onCo,DoandFo guarantee that the one-step ahead predictor of the output

is stable. Thus poles outside the open unit disc|z| < 1 are locatedin Ao. In prediction error identification, only the spectrum of the

Table 1

Details forTheorem 3.1.

Polynomial L(z) i

AozHo(z)Go (z)So (z)R(z) 1

BozHo (z)Go (z)R(z) 1

CozHo (z)

Ho (z)

2

Do zHo (z)Ho (z) 2Fo

zHo(z)Go (z)So (z)R(z) 1 See Remark 3.2.

reference signal matters andR should be interpreted as a spectralfactor of this spectrum and hence the stability and minimumphase conditions on this quantity are not particularly restrictive,save that periodic signals are not covered. Furthermore, the datashould be generated under stationary conditions which require thetransfer functions fromr ande to u and y to be stable. Since, asnoted above, Ao contains all poles on or outside the unit circle,this is equivalent to the closed loop pole polynomial Acl beingminimum phase. The non-singularity ofis a joint condition onidentifiability at

o and persistence of excitation. A sufficient andnecessary condition for this assumption to hold is that the rowsof are linearly independent. This in turn requires both pairs{Co, Do}and{Bo, Fo}to be coprime. We refer toGevers, Bazanella,Bombois and Miskovic(2009)for interesting recent sufficient andnecessary conditions for the asymptotic covariance matrix to benon-singular.

3. General results

In this section we present the main technical results of thispaper. The first result, Theorem 3.1, will give us a generalexpression for the asymptotic variance of a pole or zero identifiedin open or closed loop, and that expression will be a starting pointfor deriving more explicit expressions.

3.1. Characterization in terms of basis functions

Theorem 3.1 (A General Result). The asymptotic variance of a root zo

of any of the polynomials Ao, Bo, Co , Door Fois given by

AsVarzo = o|L(zo)|2K i(zo,zo), (8)whereK i is the ith diagonal element of

K(,z)

nk=1

Bk ()Bk(z), (9)

where{Bk}nk=1 is any orthonormal basis for the rowspace of (defined by(5)).Table1provides L and i for the different cases A oFo.For roots of Ao, Boand Fo, the result(8) holds only if R=0.

Furthermore, when some parameters are known, the above result

still holds, save that the rows of that correspond to the knownparameters should be removed when computing th e subspace used togenerate K.

Proof. The proof is given inAppendix B.

Remark 3.2. In the cases marked by inTables 1and 2we havethatL(zo)= 0 (forAthis only holds in open loop operation). Thisdoes of course not mean that the asymptotic variance is zero. Thereason is that K i(z,z) has a pole at the same location that cancelsthe zero inL. To see this we use that K i(z,z) can be written

K i(z,z)=i (z) 12

(ej)(ej) d 1 i(z), (10)


4/14


where iis the ith column of, see Appendix B. Take now, e.g., thecaseCo where L(z

o)= 0 since Co(zo)= 0. In that case there is afactor 1/Co(z

o) in 2(zoc) that cancels the factor Co(z

o) in L(zo).

We illustrate the use ofTheorem 3.1with an example.

Example 3.3. Consider the FIR system

yt

=(1

ej q1)(1

ej q1)ut

1

+et, (11)

where > 0, where the input is white noise with unit variance,i.e.R= 1 and whereetis white noise with varianceo= 1. Wewill useTheorem 3.1to analyze the asymptotic variance of zeroestimates associated with a FIR model of order n.

In order to compute the asymptotic variance for the zero zo =ej we first need to determine L in (8).From the second row inTable 1we obtain L= zHo(z)/Go(z)R(z)for zeros ofBo. For a FIRmodel Ho 1 and furthermore Go(z)= Go(z)/(1zoz1)=

z1(1 ejz1). Thus

L(ej )= 2ej2

1ej2 .From Table 1 it also follows that i = 1 in (8). We thus haveto determine the (1, 1)-element of K and for this we need

an orthonormal basis for the predictor gradient . This can bedone numerically by, e.g. GramSchmidt orthogonalization, andin Section6.4.2we will see that such a basis can sometimes beobtained without computations. In this example we notice that= n 0 so an orthonormal basis is given by zk 0,k=1, . . . , n. Thus we obtain

AsVarzo = o|L(zo)|2K1(zo,zo)

= 4

2(1cos(2))n

k=1|ej2 |2k

= 2

2(1cos(2))1 2n1 2 . (12)

Eq.(12)shows that the asymptotic variance grows exponentially

with the model order n when 1, i.e. forNMP-zeros.Fig. 2shows the zero estimates for 100 different noiserealization when 1000 inputoutput samples are used to estimate(11).ComparingFig. 2(a)(b) shows that for = 1/2 the samplevariance increases indeed quite significantly when the number ofparameters is increased fromn= 3 ton= 5. On the other hand,Fig. 2(c)(d) show that the increase is hardly visible when =3/2for the same change in model order.

Eq.(12)shows also that the asymptotic variance will be largewhen the zeros are close together, i.e. when 0. That this is thecase for estimated zeros is confirmed by comparing Fig. 2(c) and(e) which only differ in the angular position of the zeros.

The objective of this paper is to analyze the asymptotic variance(6)and even though Theorem 3.1provides a characterization of

this quantity for anyof the polynomialsAoFounder any stationaryexperimental condition, the result(8)requires further elaborationin order to provide useful insights. What remains to be doneis to characterize the function K and this in turn requires acharacterization of the rowspace of. We shall be able to providean explicit formula forK onlyfor some special cases but we willbeable to provide usefulboundsonK under quite general conditions.In fact, (8) contains structural information that we can extractwithout having to computeK explicitly. We pursue this idea next.

3.2. Lower bounds in terms of basis functions

Lemma 3.4 (Model Structure Comparison). Consider two differentmodel structures M1 and M2 defined as in (1)(2) corresponding to

predictor gradients 1

and 2

(see (5)), respectively, and supposethat the rowspace of 1 is containedin therowspace of 2. Suppose

that Go= Bo/(AoFo)and Ho= Co/(AoDo)can be described by bothmodel structures. Then the asymptotic variance for any root of AoFo,when M1 is used, is no greater than the corresponding asymptoticvariance whenM2is used.

Proof. The result follows directly from Theorem 3.1 and Lemma A.1.

However, we can also combineTheorem 3.1 andLemma 3.4

to obtain lower bounds on the asymptotic variance where K isreplaced by a quantity that is easier to characterize.

Lemma 3.5 (General Lower Bound). The asymptotic variance of aroot of any of the polynomials Ao, Bo, Co, Do or Fo is bounded frombelow according to

AsVarzo oLlb(zo)2 Klb(zo,zo), (13)

where

Klb(,z)

k

Bk ()Bk(z). (14)

Here{Bk} is any orthonormal basis for the rowspace of the vectorspecified inTable2 for the different polynomials.Table2also defines

the function Llb. In the table, Ry and Ru are transfer functions with allzeros strictly inside th e unit circle and no poles outside the unit circle,satisfying

|Ry(ej)|2 = |R(ej)|2 + o|Ho(ej)|2

|Go(ej)|2 ,

|Ru(ej)|2 = |R(ej)|2 + o|Ho(ej) K(ej)|2.(15)

Proof. The proof is given inAppendix C.

The usefulness of Lemma 3.5 is that the subspace which thebasis functions {Bk} should span is much simpler than thecorresponding space inTheorem 3.1,cf. (5)with the functionsgiven in Table 2. Infact, we will be able to use results in Appendix Ato provide explicit bounds and expressions for Klb.

Corollary 3.6. Equality holds in(13)for FIR and OE models. This alsoholds for BoxJenkins models when the condition

KGoSo

Bonb+nf is orthogonal to

1

CoDonc+nd (16)

is satisfied.

Proof. The proof is given inAppendix D.

Remark 3.7. Notice that(16)holds when the system is operatingin open loop, i.e.K 0.Remark 3.8. The ratio between the two terms

o|Ho|2|Go|2 and |R|

2 that

defineRy is the ratio of the parts of the output spectrum that are

due to the noise and the reference signal. Similarly, the ratio ofthe two terms that define Ru is the ratio of the parts of the inputspectrum that are due to the noise and the reference signal.

The following lemma provides bounds for Ry and Ru definedin(15).

Lemma 3.9 (Bounds on Ry and Ru).When Godoes not have zeros onthe unit circle, it holds that Ry, defined by(15)inLemma3.5,satisfies

1+inf

o|Ho(ej)|2|R(ej) Go(ej)|2

Ry(zo)R(zo)

2 1+sup

o|Ho(ej)|2

|R(ej) Go(ej

)

|2

, (17)

when R is not identically zero, and Ry=o Ho/Gowhen R0.


5/14


(a)n=3, =1/2, = /4. (b)n=5, =1/2, = /4. (c)n=3, =3/2, = /4.

(d)n=5, =3/2, = /4. (e)n=3, =3/2, = /10.

Fig. 2. Simulation results forExample 3.3.

Table 2Details forLemma 3.5.

Polynomial Llb(z) {Bk}spans rows of

AozHo (z)Go (z)So (z)Ry(z) Go SoRyHoAo na

BozHo (z)Go (z)Ru(z) GoSoRuHo BoFo nb+nf

CozHo (z)Ho (z) 1CoDo nc+nd

Do zHo (z)Ho (z) 1CoDo nc+ndFo

zHo (z)Go (z)So (z)Ru(z) GoSoRuHo BoFo nb+nf See Remark 3.2.

Furthermore when Hoand K are stable, Ru, also defined by(15)inLemma3.5,satisfies

1+inf

o|Ho(ej)K(ej)|2|R(ej)|2

Ru(zo)R(zo)2

1+sup

o|Ho(ej)K(ej)|2|R(ej)|2 , (18)

when R is not identically zero, and

Ru=oHoK wh en R0. (19)Proof. The proof is given inAppendix E.

3.3. Explicit lower and upper bounds

We now provide bounds which indicate how the asymptoticvariance of estimated roots depends on the model order.

Theorem 3.10. The asymptotic variance of a root of any of the

polynomials Ao, Bo, Co, Door Fois bounded according to

o|zo|2c11 |zo|2m2

1 |zo|2 AsVarzo

c21 |zo|2m2

1 |zo|2 , (20)

where m and c1 are given in Table 3 and where c2 = c2(Go, Ho,K, R, o)also depends on which polynomial that z

o belongs to (but

not on any of the orders of the estimated polynomials), and where

m=mabf na+nb+nf+mofor some finite mo0,for zo belongingto Ao, Boand Foand wherem=mcd na+ nb+ nc+ nd+ nf+ mo

for some finite mo0, for zo

belonging to Coand Do.

Proof. The proof is given inAppendix F.

Notice thatTheorem 3.10also applies to the case|zo| = 1, bysetting

1|zo|2m21|zo|2 = m in (20). This is relevant forAoand Bowhich

are allowed to have such roots, but not for the other polynomials

(byStanding Assumptions 2.1).

3.4. Explicit characterization for BoxJenkins models

Building on Corollary 3.6, we can provide explicit upper bounds

for the asymptotic variance of the roots for FIR, output error and

BoxJenkins models under certain restrictions. In some cases it

follows fromCorollary 3.6that these bounds are tight.


6/14


Table 3

Details forTheorem 3.10.

m c1

Ao na1

|Ao(zo )|2 min Ho (ej )Ao (ej )

So (ej )Go (ej )Ry (ej )

2Bo nb+nf 1|Bo (zo)Fo(zo )|2 min

Ho (ej )Bo (ej )Fo(ej )So (ej

)Go(ej )Ru(ej

)

2Co nc+nd 1|

Co(z

o)Do (zo )|2 min

Co(ej )Do(e

j )

2

Do nc+nd1

|Co(zo)Do (zo )|2 min Co(ej )Do(ej )2Fo nb+nf 1|Bo (zo)Fo(zo )|2 min

Ho (ej )Bo (ej )Fo(ej )So (ej

)Go(ej )Ru(ej

)

2

Theorem 3.11 (FIR, OE and BJ Models). Assume that the model

structure is FIR, OE or BoxJenkins, where for the latter we assume

that(16)holds.

The asymptotic variances of theroots of the Coand Dopolynomials

are given by

AsVarzo =|Co(1/zo)Do(1/z

o)|2|Z(zo)|2

|zo|2(nc+nd)(|zo|2 1) , (21)

where Z=CoDofor a root of Coand Z=DoCofor a root of Do.Furthermore, denote the denominator of GoSoRuBoFoHo (after cancella-

tions) by A and suppose in the following that the order n of this

polynomial is less than nb+nf.Denote the order of the numerator of GoSoRu

BoFoHo(after cancellations)

by m. Then the asymptotic variance of a root zo = 0of the Bo or Fopolynomials, with|zo| = 1, is bounded from above according to

AsVarzo 1(1 |zo|2)

o|Ho(zo)|2|Go(zo)So(zo)Ru(zo)|2

1|A(1/zo)|2

|A(zo)|2 |zo|2(nb+nf+m)

, (22)

where Ruis defined inLemma3.5.

For a root of Bo with magnitude 1, i.e.

|zo

| = 1, the asymptotic

variance is bounded from above according to

AsVarzo o|zoHo(z

o)|2|Go(zo)So(zo)Ru(zo)|2

nb+nf+mn+n

k=1

1 |k|2|zo k|2

, (23)

where the{k}nk=1are the roots of A.When m = 0, i.e. when the numerator of GoSoRu

BoFoHois constant,

equality holds in(22)and(23).

Proof. The proof is given inAppendix G.

In the following sections we will interpret and elaborate on theresults in this section.

4. Zeros and poles in the open unit disc|z| < 1

Due to the factor |zo|2m in the numerator of(20), Theorem 3.10shows that the asymptotic variance for zeros and poles with

magnitude less than one increases exponentially with the order of

the corresponding polynomial, cf.Example 3.3.Thus, the accuracycan be poor even when only a moderate number of parameters are

used for a certain polynomial. We alert the reader that the result

holds regardless of whether the system is operating in open orclosed loop. Also notice that the asymptotic variance for a root of

Co(Do) can belarge evenifnc(nd) is small. This happens ifnd(nc) is

large since m=nc+ndfor the Coand Dopolynomials (see Table 3).A similar conclusion holds for Boand Fo.

For open loop operation, or whenever (16) holds, an exactformula for the asymptotic variance of the roots of the noise modelpolynomials Co and Do is given in Theorem 3.11 by (21). Thistheorem also gives an exact expression for roots ofFounder certainconditions. Notice that these formulae exhibit the exponentialgrowth with the order discussed in the preceding paragraph.

We conclude that accurate identification of zeros and polesstrictly inside the unit circle is inherently difficult for model

structures of high orders.

5. Unstable poles

In this section we will study the asymptotic variance of esti-mates of poles located strictly outside the unit circle. By assump-tion, all such poles are contained in Ao , see the discussion belowStanding Assumptions 2.1.Since the theory covers stationary ex-perimental conditions, closed loop operation is necessary for theresults in this section to hold.

Theorem 3.10provides generally applicable lower and upperbounds for the asymptotic variance of an unstable pole. However,more transparent expressions canbe obtained by considering largemodel orders. We have the following result.

Theorem 5.1 (Unstable Poles).Consider a root zo of Ao with|zo| >1. The asymptotic variance AsVarzo increases monotonically to a

finite limit as n= na+ nb+ nc+ nd+ nf tends to infinity. Thelimit is bounded from above according to

limn

AsVarzo o Co(zo)Ao(zo)Do(zo)

2 K(zo)R(zo)2 1(1 |zo|2) (24)

and, provided na , bounded from below by

o

Co(zo)Ao(zo)Do(zo)2 K(zo)Ry(zo)

2 1(1 |zo|2) limn AsVarzo, (25)where Ry is defined by (15)inLemma3.5.

Proof. The proof is given inAppendix H.

The upper bound in Theorem 5.1 shows that, contrary to polesstrictly inside the unit circle (see Section 4), the asymptoticvariance for unstable poles converges to a finite limit as the modelorder increases. FromTheorem 5.1we see that the location of thepole is critical for the asymptotic variance when the number ofparameters in A is large. A pole close to the unit circle will havehigh asymptotic variance.Furthermore, we see that also the gain ofthe controller at the pole is important for the asymptotic variance.

Notice that Lemma 3.9 gives that|Ry(zo)| > |R(zo)| whichshows that the lower bound inTheorem 5.1is indeed lower thanthe upper bound in the same theorem. In the following subsectionswe will consider some special cases where Ry can be expressedexplicitly and which will shed further light on the asymptoticvariance of unstable poles.

5.1. High SNR at the output

Corollary 5.2. For a root zo of Aowith|zo|> 1, it holds

limna

AsVarzo

o

Co(zo)Ao(zo)Do(zo) 2 K(zo)R(zo) 2 1(1|zo|2) =1, (26)when

sup

o|Ho(ej)|2|R(ej) Go(ej)|2

0. (27)

Proof. In the limit(27)it follows from(17)that|R(zo)/Ry(zo)| 1 and hence the lower bound(25)equals the upper bound(24)inthe limit(27).


7/14


Corollary 5.2shows that at high SNR at the output, cf. Remark 3.8,the asymptotic variance of a pole of the system,located outside theunit disc, is approximately equal to the upper bound in(24).

5.2. Proportional signal and noise spectra at the output

When the two terms defining Ry, see Lemma 3.5, areproportional to each other, the lower bound in Theorem 5.1can

be expressed explicitly.

Corollary 5.3. Assume that R= Ho/Go for some > 0. Thenfor a root zo of Aowith|zo|> 1 it holds

1

1+ o

K(zo) Bo(zo)Ao(zo)Fo(zo)

2

1

(1 |zo|2) lim

naAsVarzo. (28)

Proof. With R = Ho/Go = Fo/(BoDo), it follows thatRy=

o+ Fo/(

BoDo) and(25)inTheorem 5.1gives the result.

5.3. Costless identification

The case when there is no external excitation, i.e. R 0, hasbeen coined costless in Bombois, Scorletti, Gevers, Van den Hof,and Hildebrand(2006) since identification can be performed inclosed loop without perturbing the system from normal operatingconditions.

Corollary 5.4. Assume R0. Then for a root zo of Aowith |zo| > 1,it holdsK(zo)

Bo(zo)

Ao(zo)Fo(zo)

21

(1 |zo|2) limna AsVarzo. (29)

Proof. Follows directly fromCorollary 5.3by setting

=0.

InCorollary 5.4we see again the critical role that the pole locationand the controller gainK(zo) play for the asymptotic variance of apole outside the unit disc.

6. Non-minimum phase zeros

Now we turn to the asymptotic variance of non-minimumphase zeros of the system. Here the results below cover both openand closed loop identification.

Theorem 6.1 (NMP-zeros).Consider a root zo with|zo| > 1of Bo.The asymptotic variance increases monotonically to a finite limit asn=na+nb+nc+nd+nftends to infinity.

The limit is bounded from above according to

limn

AsVarzo o|Ho(zo)|2|Go(zo)R(zo)|2 1(1 |zo|2) (30)and, provided nb+nf , bounded from below by

o|Ho(zo)|2|Go(zo)Ru(zo)|2 1(1 |zo|2) limn AsVarzo. (31)Equality holds in(31) for FIR and OE models as well as BoxJenkinsmodels subject to(16).

Proof. The proof is given inAppendix I.

As for poles outside the unit disc, we see that the asymptoticvariance of an NMP-zero converges to a finite limit as the modelorder increases. We see also that the location is important for the

asymptotic variance of a zero. An NMP-zero close to the unit circlewill have large asymptotic variance.

6.1. Open loop operation or high SNR at the input

Corollary 6.2. For a root zo of Bowith|zo|> 1, it holds

limnb+nf

AsVarzo

o

Ho(z

o)

Go(zo)R(zo)

2

1

(1|zo|2)

=1, (32)

when

sup

o|Ho(ej)K(ej)|2|R(ej)|2 0. (33)

In particular,(32)holds in open loop operation, i.e. when K 0.Proof. In the limit(33)it follows from(18)that|R(zo)/Ru(zo)| 1 and hence the lower bound(31)equals the upper bound(30)inthe limit(33).

From Corollary 6.2 we are able to conclude that the upper bound inTheorem 6.1 is attained when thesystem is operating in open loop.Furthermore, for closed loop operation at high SNR at the input, cf.Remark 3.8,and with a large number of parameters in Band/orF,the asymptotic variance becomes equal to the asymptotic variancein open loop operation.

6.2. Proportional signal and noise spectra at the input

When the two terms defining Ru, see Lemma 3.5, areproportional to each other, the lower bound in Theorem 5.1canbe expressed explicitly.

Corollary 6.3. Assume that R=HoK for some > 0 . Then fora root zo of Bowith|zo|> 1 it holds

1

1+ o

1K(zo) Bo(zo)Ao(zo)Fo(zo)

2

1

(1 |zo|2) limnb+nf AsVarzo. (34)

Equality holds in(34) for FIR and OE models as well as BoxJenkins

models subject to(16).Proof. WithR=HoK, it follows thatRu= o+ HoKand(31)inTheorem 5.1gives the result.

Corollary 6.3provides some insight regarding the role played bythe controller for the asymptotic variance of NMP-zeros. SinceK(zo) appears in the denominator in the lower bound(34),a smallcontroller gain will result in a large asymptotic varianceexactlythe opposite to what is the case for poles outside the unit disc, cf.Section5.

6.3. Costless identification

Corollary 6.4. Assume R0. Then for a root zo of Bowith|zo| > 1it holds

1K(zo) Bo(zo)Ao(zo)Fo(zo) 21

(1 |zo|2) limnb+nf AsVarzo. (35)

Equality holds in(35) for FIR and OE models as well as BoxJenkinsmodels subject to(16)

Proof. Follows directly fromCorollary 6.3by setting =0. Asin Corollary 6.3 we seethat Corollary 6.4 embodies the messagethat a small controller gain at the NMP-zero of interest implies alarge asymptotic variance.

6.4. Finite model order results

For FIR, OE and BoxJenkins models we can useTheorem 3.11to obtain results for finite model orders for NMP-zeros.


8/14


6.4.1. Upper bounds

Notice thatSo(zo)= 1 for a zero ofBo so the upper bound in

(22)is the same as the asymptotic limit in Corollary 6.2,save for

the factor1|A(1/z

o)|2|A(zo)|2

|zo|2(nb+nf+m)

(36)

which converges to 1 asnb+nf . The upper bound(22)canbe evaluated also for the remaining special cases ofRuused in the

preceding subsections, i.e. proportional noise and signal spectra at

the input and costless identification. The results are finite model

order upper boundsthat areequalto the asymptotic in model order

lower bounds(34)and (35),respectively, but with(36)as added

factor.

6.4.2. Exact results

Theorem 3.11can also be used to obtain exact expressions for

the asymptotic variance. For open loop operation we have the

following result.

Theorem 6.5 (Open Loop/FIR, OE and BJ Models). Assume that the

model structure is FIR, OE or BoxJenkins and that K 0. Assumefurther that Do= 1and that R has no zeros. Denote the polynomialF2o Co/R by Aand suppose that nb+ nfis larger than the order of this

polynomial. Then the asymptotic variance of a root zo =0 of Bo, with|zo| =1, is given by

AsVarzo = 1(1 |zo|2)

o|Ho(zo)|2|Go(zo)R(zo)|2

1|A(1/zo)|2

|A(zo)|2 |zo|2(nb+nf )

. (37)

When|zo| =1, the asymptotic variance of root zo of Bois given by

AsVarzo =

o|zo H

o(zo)

|2

|Go(zo)R(zo)|2

nb+nfn+n

k=1

1 |k|2|zo k|2

, (38)

where the{k}nk=1are the roots of A.Proof. Follows directly fromTheorem 3.11.

Notice that the above theorem is valid not only for NMP-zeros but

for any zeros. For closed loop operation we have the following

result.

Theorem 6.6 (Closed Loop/FIR and OE Models). Assume that the

model structure is FIR or OE and that the controller K= Kn/Kd= 0is such that Knand Kdare coprime stable polynomials such that

Fo=F1Knfor some polynomial F1 . Suppose further that R= K for some > 0and that nb+nfis at least the order of Fo(FoKd+Bo). Thenthe asymptotic variance of a root zo =0 of Bo, with |zo| = 1, is givenby

AsVarzo = 1(1 |zo|2)

1

1+ o

1

|G(zo)K(zo)|2

1|A(1/z

o)|2

|A(zo)

|2 |zo|2(nb+nf )

, (39)

where A=Fo(F1Kd+Bo).

When|zo| = 1, the asymptotic variance is given by

AsVarzo = |zo|2

1+ o

1

|G(zo)K(zo)|2

nb+nfn+n

k=1

1 |k|2|zo k|2

, (40)

where the{k}nk=1are the roots of A.Proof. Follows directly fromTheorem 3.11.

Theorem 6.6 is the NMP-zero counterpart to Theorem 4.1 inNinness and Hjalmarsson(2005b)where asymptotic variance forfrequency function estimates is considered. The condition on R canbe interpreted as that the feedback controller is u = K(r y)(rather than u=Kyras we have assumed throughout the paper)where the referenceris white noise.

7. Interpretations of the results

In the previous sections we have presented a number ofexpressions and simplified bounds for the asymptotic varianceof roots of the polynomials AoFo. We will now summarize and

interpret our major findings.

Roots inside versus outside the unit circle

An important result of this paper is that the asymptoticvariances of zeros and poles strictly inside the unit circle behavefundamentally different from the asymptotic variances of NMP-zeros and unstable poles when the model order is increased.Theorem 3.10 shows that the former grow exponentially whileTheorems 5.1 and 6.1 show that the latter converge to finite limits.Theorems 6.5 and 6.6 show that forFIR, OE andBoxJenkinsmodelsthe convergence for NMP-zeros is exponentially fast, with rate1/|zo|, under certain conditions. Thus roots closer to the unit circleimply slower convergence. Although this has not been proven, webelieve this to be true in general for both poles outsidethe unit disc

and NMP-zeros.

Roots on the unit circle

Combining the upper and lower bounds in Theorem 3.10givesthat for the borderline case with a root on the unit circle, theasymptotic variance grows linearly with the model order. This issimilar to what happens for frequency function estimates (Ljung,1985;Ninness & Hjalmarsson, 2004).

Roots close to each other

FromTheorem 3.1it follows that the asymptotic variance of arootof one ofthe polynomials blowsup whenthere isa nearbyrootof the same polynomial. The reason is that a root is very sensitive

to errors in the parameters of the polynomial in this case. Forexample, for the second order polynomial z2 +1z+2, it holdsthat

dz( )

d =1

2 1

2

2142

12142

T,

which tends to infinity as 21 42, i.e. when the polynomial hasa double root at1/2. In fact the standard asymptotic analysis,which is based on the first order Gauss approximation formula,breaks down for roots of higher multiplicity than 1. This does notimply that the variance of root estimates becomes infinite for,e.g., double roots, instead the convergence rate becomes slowerthan the 1/Nrate given in(7).We will not pursue this here butsimply conclude that the result above indicates that there are

significant problems associated with estimating roots of highermultiplicity than one.


9/14


NMP-zeros

The universal upper bound

limn

AsVarzo o|Ho(zo)|2

|

Go(zo)R(zo)|2

1

(1 |zo|2) (41)

fromTheorem 6.1is useful when the reference signal is non-zero.

Notice that the bound is model structure independent. It is alsotight in open loop operation and for high signal to noise ratios atthe input (see Corollary 6.2) in closed loop operation, and above wehave argued that the convergence rate with respect to the modelorder is likely to be exponentially fast (at least we have shown thisfor some special cases). Thus the right-hand side of(41)providesin many cases a quite reasonable approximation of the asymptoticvariance for a NMP-zero.

The reason why (41)becomes tight when the signal to noiseratio at the input increases can be understood from the fact thatthe sensitivity function of the closed loop becomes unity whenevaluated at a zero of the system. Thus the feedback does notcontribute to the excitation at a system zero and when the signaltonoise ratio atthe input ishighso thatthe noise fed backto theinput

does little to excite the system, the open loop case is recovered.The above observation has several implications. The firstconcerns how open loop identification compares to closed loopidentification. It is well known that for BoxJenkins models closedloop identification with a reference signal having spectral factorR,so that the part of the input due to the reference signal is SoR, isnever worse than open loop identification with a reference signalhaving spectral factor SoR (Agero & Goodwin, 2007; Bombois,Gevers, & Scorletti, 2005). For NMP-zeros, it follows from(41)thatthis holds for high model orders for any modelstructure also whenRis the spectral factor of the input in open loop identification.

Secondly, the universality of (41) also implies that, forhigh model orders, indirect and direct identification result insimilar asymptotic variance of NMP-zeros. Recall that in indirectidentificationris used as input rather than u(Ljung, 1999).

Thirdly, notice that very different input spectra will result insimilar asymptotic accuracy. We illustrate this with an example.

Example 7.1. Consider the first order ARX system

(10.9q1)yt= (q1 +1.1q2)ut+et, (42)where the input is given byut= rt 10.5q

11+0.3q1ytand wherertand

etare independent white noise, both with unit variance. This shallbe compared with the open loop case when the input is given byut= rt. The input spectra are plotted inFig. 3and in this example,they are very different in open and closed loop (for the closed loopcase both the total input spectrum u and

ru , which is the part

of the spectrum that originates from the reference signal rt, are

plotted). Despite this big difference, the variance of the estimatedzero does not change, at least not for high model orders.

The system (42) is simulated in both open and closed loops.From the closed loop simulations we use both direct andindirect identification. For the open loop case and for the directidentification in the closed loop case, an ARX model is estimatedfrom 1000 samples ofutandyt. For the indirect identification weestimate an ARMAX model from 1000 samples ofrt and y t. Thevariance of the estimated zero, from 5000MonteCarlosimulations,is presented inTable 4.The model orderno represents the order ofthe true system, i.e., na = 1 and nb = 2 in the ARX model andna = 3, nb = 3 and nc= 1 for the ARMAX model. When themodel order is increased, no + k means that na and nb has beenincreased by k , and nc is unchanged. The values in the table are

the variances from the simulations, multiplied byN= 1000. Theasymptotic value 5.8 is the upper bound given by(41).

Table 4

Variance of estimated non-minimum phase zeros from Monte Carlo simulations in

open and closed loops.

n Open loop Closed loop direct id. Closed loop indirect id.

no 2.3 0.21 4.7

no +2 4.0 1.8 6.6no +5 6.9 4.5 7.3no +10 6.9 7.2 6.2n

o

+15 6.0 6.5 5.5Asympt. 5.8 5.8 5.8

frequency

Inputspectrum

r

103

102

101

100

101

1020 0.5 1 1.5 2 2.5 3

u(), closed loop

u(), open loop

u(), closed loop

Fig. 3. Inputspectrum for Example7.1. Theinput spectra are very differentin open

and closed loops. This does not affect the accuracy of the zero estimates.

What we see is that the direct identification in closed loopperforms best for low model orders, but already fromno +5 andup, all three cases give about the same variance, just a little higherthan the asymptotic value.

Next we make some further comments on the accuracy ofestimated non-minimum phase zeros, based on(41).

Usually there is a tradeoff between bias and variance in the

sense that a low order model results in a biased estimate withlow(er) variance and a high order model gives an un-biasedestimate with high(er) variance. When it comes to NMP-zeros,this increase in the variance is typically small and, hence, highorder models can be used without the risk of loosing accuracy,unless the zero is close to the unit circle.

In many identification scenarios the input filter R(q) can bechosen by the user. The variance is proportional to 1/|R(zob )|2 sothat factor should be small. IfR(q) has a pole atzob this factor iszero, but that corresponds to an unstable, and hence infeasible,input filter. An interesting choiceis an autoregressive filter with

just one pole at 1/zob . Such an input filter is optimal in thesense that it has the lowest input power among all LTI filtersthat achieve the same variance, seeMrtensson, Jansson, andHjalmarsson(2005).

Consider open loop identification and suppose for a momentthat Go(q) is known and that only the part 1 zoz1 has tobe estimated. In this scenario,Go(q)= 1 and the input filterRshould be the original input filter in series with the originalGo(q), since the latter part of thesystem shapes theinput to thesystem 1 zoz1, which is to be identified.Notice now that theinput filterR(q)and the factorGo(q)enter the bound(41)as aproduct, i.e. the bound is the same if the product between theinput filterand thesystemthat is identified, excludingthe NMP-zero part, is the same. Thus in the above scenario, the boundremains thesameas when alsoGo(q) is identified. Furthermore,we have fromCorollary 6.2,that as nb+nf , the bound(41) is attained. Combining these observations we have that forhigh model ordersand open loop identification it is insignificantwhetherGo(q)is known or estimated, i.e. knowing Go(q)andonlyestimating 1zoz1 helpslittle for improving the accuracy.


10/14


Unstable poles

The universal upper bound

limn

AsVarzo o

Co(zo)

Ao(zo)Do(zo)

2

K(zo)

R(zo)

21

(1 |zo|2) (43)

fromTheorem 5.1is useful when the reference signal is non-zero.

Notice that the bound is model structure independent. It is also

tight for high signal to noise ratios at theoutput (see Corollary 5.2)

and above we have argued that the convergence rate with respect

to the model order is likely to be exponentially fast (at least we

have shown this for some special cases). Thus the right-hand side

of(43)provides in many cases a quite reasonable approximation

of the asymptotic variance for a pole located outside the unit disc.

The upper bound(43)and the corresponding lower bound(25)

indicate that the controller should be designed so that the gain at

the pole is small in order to ensure a small asymptotic variance.

In this context, we point out that if the feedback configuration is

changed to ut= K(q)(rt y t) instead ofu t= rt K(q)yt asabove, the controller dependence in(43)and in other expressions

for the asymptotic variance of unstable poles will disappear. This

is easy to see since this change in controller structure correspondsto replacingRbyKR.

Costless identification

Forcostlessidentification,i.e. when theexternal referenceis not

used to excite the system, the lower bounds 5.4and6.4show that

the controller gain should be small at unstable poles but large at

NMP-zeros in order for the asymptotic variance to be small. Thus

there is a conflict when designing suitable experiments when the

objective is to identify a NMP-zero and an unstable pole that are

close to each other.

8. Summary

In this paper we derive explicit expressions for the asymptotic

variance of estimated poles and zeros of dynamical systems. One

of the main conclusions is that the asymptotic variance of non-

minimum phase zeros and unstable poles is only slightly affected

by the model order, while the variance of minimum phase zeros

and stable poles is very sensitive and grows exponentially with the

model order. Another important observation is that nearby roots

have a very detrimental effect on the asymptotic variance.

The asymptotic variance expressions give structural insight to

how the variance is affected by e.g., model order, model structure

and input excitation.

Acknowledgments

The authors would like to express their gratitude to the

anonymous reviewers for their many insightful remarks and for

spurring us to a major revision which, we believe, significantly

improved the quality of the paper.

Appendix A. Technical results for reproducing kernels

Let{Bk}nk=1 be an orthonormal basis for an n-dimensionalsubspace Sn L2 and let K(,z)denote the reproducing kernelfor Sn, here defined as

K(,z)

nk=1

Bk ()Bk(z). (A.1)

The function K(,z)is called a reproducing kernelsince it holdsthat

f(),K(, ) =f(), f Snf(),K(, ) =ProjSn{f}(), f L2 (A.2)where ProjSn{f}denotes the orthogonal projection off L2ontoSn, see e.g.,Heuberger, Wahlberg, and den Hof(2005).

The reproducing kernel is unique and can also be expressed as

K(,z)=(), 1(z) (A.3)for any whose rowspace spans Sn(Heuberger et al., 2005).

In the following lemmas we present some properties of thereproducing kernels of different subspaces which are used in theproofs of the paper.

Lemma A.1. Consider two subspaces Sn andSm with reproducingkernelsK(,z) andK(,z) and letSnSm. Then it holds thatK(z,z)K(z,z). (A.4)Proof. Let{Bk}nk=1 and{Bk}mk=1 be orthonormal bases for Sn andSm, respectively, with Bk=

Bk,k=1, . . . , n. Then it is clear that

K(z,z) K(z,z)= mk=n+1Bk (z)Bk(z)0,

which concludes the proof.

Lemma A.2. Let the scalar-valued subspaces Sn andSm havereproducing kernels K(,z)andK(,z). Suppose that there existsa constant: 0 < 1 such that for every function f Sn thereexists a function gSmthat fulfills|f(z) g(z)| < |f(z)|, z ZwhereZ is some region including the unit circle. Then it holds that

K(z,z)

1+ 1

2K(z,z), z Z.

Proof. Since K(,z) itself, as a function ofz, belongs to Sn, thereis by assumption a family of functionsgSmsuch that|K(,z) g(z)|< |K(,z)|, z Z.Thus, we can write

K(,z)=g(z) +(z),where

|(z)|< |K(,z)|, z Zwhich also implies that()< K(, )since Z includesthe unit circle. Hence,

|K(,z)| |g(z)| + |K(,z)|,and by using the CauchySchwarz inequality and the fact thatK(, ),K(z, ) =K(,z) we get

|K(,z)| 11 |g(z)| =

1

1 |g(),K(z, )|

11 g()

K(z, ) = 11 g()

K(z,z)= 1

1 K(, ) ()K(z,z)

11 K(, ) + ()K(z,z)

11 (K(, ) + K(, ))

K(z,z)

= 1+ 1 K(, )K(z,z).


11/14


For =zit holds that K(z,z)= |K(z,z)|and then squaring theinequality above gives the result.

Lemma A.3. Consider the subspace

Sn Span

z1

M(z), . . . ,

zn

M(z)

, (A.5)

where

M(z)

mi=1

(1 iz1), |i|< 1 (A.6)

and nm. The reproducing kernel of the space Snis given by

K(z,z)=

|z|2

1 |z|2

1|M(1/z)|2

|M(z)|2 |z|2n

,

z=0 & |z| =1,nm+

mk=1

1 |k|2|z k|2

, |z| = 1.(A.7)

Furthermore, the reproducing kernel for the space spanned by zk,

k=1, 2, . . . can be expressed asK(z,z)= |z|

2

1 |z|2 , for|z|> 1. (A.8)

Proof. For z : z = 0,|z| = 1, the reproducing kernel for Sn isgiven by

K(,z)= 1 n()n(z)z 1 , (A.9)

where k(z) k

j=11jzzj , see Ninness, Hjalmarsson, and

Gustafsson(1998), and it can also be expressed in terms of thefunctionM(z) by noting that

n(z)= M(z1)

M(z)zn. (A.10)

Combining (A.9) and (A.10) gives the result (A.7) when |z| =1. Theresult when|z| =1 follows directly fromNinness and Gustafsson(1997).

For(A.8)we notice that Bk(z)= zk, k = 1, 2, . . . form anorthonormal basis and hence

K(z,z)=

k=1|z|2k = |z|

2

1 |z|2 , |z|> 1.

Lemma A.4. Consider the subspace

Sn Span Q(z)

M(z)z1, . . . , Q(z)

M(z)zn , (A.11)

where M(z) is given by(A.6)(with mn) andQ(z) q0+q1z1 + +qnqznq (A.12)has all zeros inside the unit circle. LetKnbe the reproducing kernel of

Sn. ThenKn(z,z) is bounded by

Kn(z,z) |z|21 |z|2

1|M(1/z)|

2

|M(z)|2 |z|2(n+nq)

, (A.13)

when z=0 is such that|z| =1 and when|z| =1 by

Kn(z,z)n+nqm+m

k=1

1

|k

|2

|z k|2 . (A.14)

Proof. Define SpasSp Span z1

M(z), . . . ,

zp

M(z)

. (A.15)

If we let p = n+ nq then Sn Sp and Lemma A.1 givesthat Kn(z,z)

Kp(z,z)where

Kp is the reproducing kernel of

Sp. Lemma A.3gives the right-hand sides of(A.13)and (A.14)asexpressions forKp(z,z). Lemma A.5. Consider the subspace

Sn Span

F(z)z1, . . . , F(z)zn

, (A.16)

where F is a rational stable transfer function. Let Kn be thereproducing kernel of Sn. ThenKn(z,z) is bounded according to

1 |z|2n21 |z|2 |F(z)|

2 1

sup

F(ej)2 Kn(z,z) 1 |z|

2n2

1

|z

|2 |F(z)|2

1

inf

F(ej)2

. (A.17)

Proof. Observe that the eigenvalues of

1

2

F(ej)2 n(ej)n(ej)d (A.18)are bounded from below by inf

F(ej)2 and from aboveby sup

F(ej)2, see Grenander and Szeg (1958). Hence theeigenvalues of the inverse of (A.18) are lower bounded by

1

sup|F(ej )|2 and hence

K(z,z)=|F(z)|2

n(z) 1

2

F(ej)2 n(ej)n(ej)d1

n(z)

|F(z)|2 1sup

F(ej)2 n(z)n(z),which gives the lower bound. The upper bound follows using thelower bound for the eigenvalues.

Lemma A.6. LetSnandKnbeasin Lemma A.4 andlet|z|> 1.Whenthe dimension n goes to infinity, the reproducing kernel is given by

limn

Kn(z,z)= |z|21 |z|2 . (A.19)

Proof. Consider the two spaces Sn andSp given by (A.11)and (A.15) and let Kn(,z) andKp(,z) be the associatedreproducing kernels. For any value p of the dimension of thesubspaceSp there is a number r(p) such that there existsparameters ksuch that1L(z) r(p)

k=0kz

k< 1p , z: |z| 1,

i.e. on and outside the unit circle the function 1/L(z) can beapproximated arbitrarily well with an FIR-filter by choosinga sufficiently large number r(p). This is a consequence ofMergelyans theorem, see e.g.Rudin(1987). Let this FIR-filter be

denotedS(z) r(p)

k=0 kzk. A functionf

Spcan be written as

f(z)= 1

M(z)

pk=1

kzk


12/14


for some arbitrary parameters k. Now notice that the functionL(z)S(z)f(z) belongs to the subspace Sn if we take n = n(p)=

p+r(p). Thus, for any functionfSp , there isag= LSf Snsuchthat

|f(z) g(z)| = |f(z) L(z)S(z)f(z)|= |1L(z)S(z)||f(z)|

1

p |f(z)|, z: |z| 1.Lemma A.2can now be applied and gives

1 1p

1+ 1p

2 Kp(z,z)Kn(p)(z,z).The sequence Kn(z,z), n= 1, 2, . . . is monotonically increasingand for|z| > 1 Lemma A.4 gives the finite upper boundKn(z,z) |z|

21|z|2 and hence it follows that Kn(z,z)has a limit.

The subsequence Kn(p)(z,z),p= 1, 2, . . . then converges to thesame limit and we can conclude that

limp

Kp(z,z) lim

n

Kn(z,z).

Lemma A.3gives that the lower bound approaches

limpKp(z,z)= |z|2

1 |z|2 ,

which is the same as the upper bound and that proves the result.

Appendix B. Proof ofTheorem 3.1

The starting point is the variance expression(6)which also canbe written as

AsVarzo =o(o), 1(o), (B.1)where is given by (5)and where ( ) is the gradient of the

pole/zero with respect to the parameters. This gradient is given bythe next lemma.

Lemma B.1. Let za(a) be a zero of A(q, a) of multiplicity1. Then

dza(a)

d a

a=oa

= zoaAo(zoa ) na (zoa ),

wheren(q)= [q1, . . . , qn]T. For zeros of the other polynomialsB, C , D and F we get analogous expressions.

Proof. The proof can be found in, e.g., Oppenheim and Schafer(1989).

Now we return to the expression(B.1)and consider a zero ofB(q),which is the most straightforward case. The result for the othercases will be commented at the end of the proof. Consider thefunction J( )= zb(b), which does not depend on a, c, d andf. Thus, fromLemma B.1,

(o)=

0na1

zobBo(zob ) nb (zob )0nc10nd10nf1

.It is now readily verified that

1(zob )

Ho(zob )Bo(z

ob )

Go(

zob )

So(

zob )

R(zo

b ) =

0nb (zob )

0

00

(B.2)

and hence 1(zob )L(z

ob )= (o)with L(z)given inTable 1since

So(zob )=1 regardless of the feedback. Thus, we can write (B.1)as

AsVarzo =o|L(zob )|21 (zob ), 11(zob ). (B.3)Now, using (A.3), we recognize 1 (), 11(z) as thereproducing kernel to the rowspace to and can be written as(A.1),i.e.(9).This now gives the result(8)inTheorem 3.1.

The other cases A, C, D and Ffollow in the same way, exceptfor one detail which is explained here. For brevity only the case

Ais considered here but the result holds also for the other cases.It should first be noted that is singular atzoa . This singularity iscanceled by the factor Ao in the function L , seeTable 1, and westill have that 1(z

oa )L(z

oa ) = (o). This pole/zero cancellation

between L and K i must be made for the expression (8) to makesense.

Appendix C. Proof ofLemma 3.5

First we note that existence of Ry and Ru satisfying (15)is ensured by the spectral factorization theorem. Poles on theunit circle can be extracted before the spectral factorization and

reinserted afterwards and are thus left unchanged. Furthermore,notice that byStanding Assumptions 2.1, R has all zeros strictlyinside the unit circle. Thus the right-hand sides of the equationsin(15)are non-zero on the unit circle and thus Ry and Ru can betaken to have all their zeros strictly inside the unit circle.

Consider now a zero ofBo. Lemma 3.4 gives that the asymptoticvariance when only b and f are used is no greater than theasymptotic variance when allparameters areused. Considering thecase when only band fare used will hence give a lower bound.

Forthis case, it is straightforward to verifythat (ej)(ej)=(ej)(ej) where

G oSoRuHo

1

BoTnb

1

FoTnf

Tand that

(zob ) Ho(z

ob )Bo(z

ob )

Go(zob )So(z

ob )Ru(z

ob )

=nb (zob )

0

.

Thus, if we consider a zero ofBwe can, analogously to the proof ofTheorem 3.1,write(B.1)as

AsVarzo =o|L(zob )|2(zob ), 1(zob ), (C.1)with L(z)= zHo(z)/(Go(z)So(z)Ru(z)). Since So(zob )= 1 we canremove So from L. Now

(), 1(z) is the reproducingkernel for the rowspace of . However, since Boand Foare coprimethis is thesame subspace as the one generated by GoSoRu

BoFoHonb+nf, see

Ninness and Hjalmarsson(2004). This proves(13)for zeros ofBo.The proofs for zeros of the other polynomials are analogous.

Appendix D. Proof ofCorollary 3.6

That we have equality in (13)when only b and f are used,i.e. for FIR and OE models, follows directly from the proof ofLemma 3.5.

We now turn to the BoxJenkins case when(16)holds. Then

=

GoHo

So

BoRnb (q)

KGoSo

Bo

o nb (q)

0 1Co

o nc(q)

01

Doo nd (q)

GoHo

SoFo

Rnf (q) KGoSo

Foo nf (q)

.


13/14


14/14


For thelower bound (25) we invoke Lemma 3.5 and then notice

that Lemma A.6 is applicable toKlb(zo,zo) under the conditions in

the theorem. Evaluation ofLlbfor AousingTable 2now gives(25).

Appendix I. Proof ofTheorem 6.1

The proof parallels the proof ofTheorem 5.1,see Appendix H

but in the proof of(30) L is evaluated for Bo instead. Similarly, inthe proof of(31),Llb is evaluated forBousingTable 2.

That equality holds in(31)for OE and FIR models as well as forBoxJenkins models subject to (16) follows from Corollary 3.6. Thisconcludes the proof.

References

Agero, J., & Goodwin, G. (2007). Choosing between open- and closed-loopexperiments in linear system identification. IEEE Transactions on AutomaticControl,52(8), 14751480.

Bombois, X., Gevers, M., & Scorletti, G. (2005). Open-loop versus closed-loopidentification of BoxJenkins models: A new variance analysis. In 44th IEEEconference on decision and control and the European control conference 2005 (pp.31173122).

Bombois, X., Scorletti, G., Gevers, M., Van den Hof, P., & Hildebrand, R. (2006). Leastcostly identification experiment for control.Automatica,42(10), 16511662.

Forssell, U., & Ljung, L. (1999). Closed-loop identification revisited. Automatica,35(7), 12151241.

Gevers, M., Bazanella, A. S., Bombois, X., & Miskovic, L. (2009). Identification andthe information matrix: How to get just sufficiently rich? IEEE Transactions onAutomatic Control(in press). To appear December 2009.

Gevers, M., Ljung, L., & Van den Hof, P. (2001). Asymptotic variance expressions forclosed-loop identification.Automatica,37(5), 781786.

Grenander, U., & Szeg, G. (1958).Toeplitz forms and their applications. Berkley, CA:University of California Press.

Heuberger, P., Wahlberg, B., & den Hof, P. V. (Eds.). (2005). Modeling andidentification with rational orthogonal basis functions. Springer Verlag.

Hjalmarsson, H., & Lindqvist, K. (2002). Identification of performance limitations incontrol using ARX-models. InProceedings of The 15th IFAC world congress.

Hjalmarsson,H., & Mrtensson,J. (2007). A geometricapproach to varianceanalysisin system identification: Theory and nonlinear systems. In Proceedings of the46th IEEE conference on decision and control (pp. 50925097).

Hjalmarsson, H., & Ninness, B. (2006). Least-squares estimation of a class offrequency functions: A finite sample variance expression. Automatica, 42(4),

589600.Lindqvist, K. (2001). On experiment design in identification of smooth linear

systems.Licentiate thesis. Royal Institute of Technology, Stockholm, Sweden.Ljung, L. (1985). Asymptotic variance expressions for identified BlackBox transfer

function models.IEEE Transactions Automatic Control,30(9), 834844.Ljung, L. (1999).System identification: Theory for the user(second ed.). Prentice Hall.Mrtensson, J., & Hjalmarsson, H. (2003). Identification of performance limitations

in control using general SISO-models. In Proceedings of the 13th IFAC symposiumon system identification(pp. 519524).

Mrtensson, J., & Hjalmarsson, H. (2005a). Closed loop identification of unstablepoles and non-minimum phase zeros. In Proceedings of the 16th IFAC worldcongress on automatic control.

Mrtensson, J., & Hjalmarsson, H. (2005b). Exact quantification of the variance ofestimatedzeros.In Proceedingsof the 44thIEEE conferenceon decision and controland European control conference (pp. 42754280).

Mrtensson,J., & Hjalmarsson,H. (2007). A geometricapproach to varianceanalysisin system identification: Linear time-invariant systems. In Proceedings of the46th IEEE conference on decision and control (pp. 42694274).

Mrtensson, J., Jansson,H., & Hjalmarsson,H. (2005).Input design for identificationof zeros. InProceedings of the 16th IFAC world congress on automatic control.

Ninness,B., & Gustafsson,F. (1997).A unifyingconstructionof orthonormalbasesforsystem identification.IEEE Transactions on Automatic Control,42(4), 515521.

Ninness, B., & Hjalmarsson, H. (2004). Variance error quantifications that are exactfor finite model order. IEEE Transactions on AutomaticControl, 49(8), 12751291.

Ninness, B., & Hjalmarsson, H. (2005a). Analysis of the variability of jointinputoutput estimation methods.Automatica,41(7), 11231132.

Ninness,B., & Hjalmarsson,H. (2005b).On thefrequency domain accuracyof closed-loop estimates.Automatica,41(7), 11091122.

Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1998). Generalized Fourier andToeplitz results for rational orthonormal bases. SIAM Journal on Control andOptimization,37(2), 429460.

Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1999a). Asymptotic varianceexpressions for output error model structures. In14th IFAC world congress, Vol.H(pp. 367372).

Ninness, B., Hjalmarsson, H., & Gustafsson, F. (1999b). The fundamental roleof general orthonormal bases in system identification. IEEE Transactions onAutomatic Control,44(7), 13841406.

Oppenheim, A., & Schafer, R. (1989). Discrete-time signal processing. EnglewoodCliffs, NJ: Prentice-Hall.

Rudin, W. (1987).Real and complex analysis(third ed.). McGraw-Hill, Inc..Skogestad, S., & Postlethwaite, I. (1996). Multivariable feedback controlAnalysis and

design. Chichester: John Wiley.Sderstrm, T., & Stoica, P. (1989). Prentice Hall International series in systems and

control engineering,System identification.Vuerinckx, R., Pintelon, R., Schoukens, J., & Rolain, Y. (2001). Obtaining accurate

confidence regions for the estimated zeros and poles in system identification

problems.IEEE Transactions on Automatic Control,46(4), 656659.Xie, L., & Ljung, L. (2001). Asymptotic variance expressions for estimated frequencyfunctions.IEEE Transactions on Automatic Control,46(12), 18871899.

Xie, L., & Ljung, L. (2004). Variance expressions for spectra estimated using auto-regressions.Journal of Econometrics,118(12), 247256.

Jonas Mrtenssonreceived his M.Sc. degree in VehicleEngineering and his Ph.D. degree in Automatic Controlfrom KTH Royal Institute of Technology, Stockholm,Sweden, in 2002 and 2007 respectively. Since 2007 heis a researcher at the School of Electrical Engineering atKTH. His research interests include system identification,presently focusing on modeling and control of internalcombustion engines.

Hkan Hjalmarsson was born in 1962. He received theM.S. degree in Electrical Engineering in 1988, and theLicentiate degree and the Ph.D. degree in AutomaticControl in 1990 and 1993, respectively, all from LinkpingUniversity,Sweden.He hasheld visitingresearch positionsat California Institute of Technology, Louvain Universityand at the University of Newcastle, Australia. He hasserved as an AssociateEditor for Automatica (19962001),and IEEE Transactions on Automatic Control (20052007)and has been Guest Editor for European Journal of Controland Control Engineering Practice. He is Professor at the

School of Electrical Engineering, KTH, Stockholm, Sweden. He is Chair of the IFACTechnical Committee on Modeling, Identification and Signal Processing. In 2001 hereceived the KTH award for outstanding contribution to undergraduate education.His research interests include system identification, signal processing, control andestimation in communication networks and automated tuning of controllers.

Documents

Artigo erro variância identificação.pdf