Using Generalized Fibonacci Sequences for Solving the One-dimensional Lqr Problem and Its Discrete-time Riccati Equation

In this article we develop a method of solving general one-dimensional Linear Quadratic Regulator (LQR) problems in optimal control theory, using a generalized form of Fibonacci numbers. We find the solution R (k) of the corresponding discrete-time Riccati equation in terms of ratios of generalized Fibonacci numbers. An explicit Binet type formula for R (k) is also found, removing the need for recursively finding the solution at a given timestep. Moreover, we show that it is also possible to express the feedback gain, the penalty functional and the controller state in terms of these ratios. A generalized golden ratio appears in the corresponding infinite horizon problem. Finally, we show the use of the method in a few examples.


Introduction
In the optimal control theory, one focuses on the problem of controlling a dynamic system under minimization of a penalty (or cost) functional.One of the most well-known optimal control problems is Linear Quadratic control, where the dynamic system is described by a set of linear differential equations, or linear difference equations, depending on whether continuous-or discrete-time is used.Moreover, the cost is described by a quadratic functional.In order to solve the discrete-time, finite horizon, linear quadratic control problem, typically one solves a nonlinear Riccati difference equation recursively.We refer to the books Kwakernaak and Sivan (1972) and Lewis and Syrmos (1995) for more information on optimal control theory.
The Italian matematician Leonardo of Pisa, known as Fibonacci, studied in the 13th century the growth of an idealised rabbit population and arrived in the nowadays famous Fibonacci sequence (F n ) = (0, 1, 1, 2, 3, 5, 8, 13, . ..), described by the recursive equation These remarkable numbers have been shown to appear in such diverse fields as nature, art, geometry, architecture, music and even for calculating π.Moreover, one of the most fascinating facts is that the ratio F n /F n−1 of two consecutive numbers converge to the golden ratio (or golden section) ϕ given by are e.g. the books Dunlap (2003), Huntley (1970), Livio (2002) and Walser (2001).
There is in general not much known about the connection between optimal control and Fibonacci numbers.In Benavoli et al. (2009) relationships between the Fibonacci sequence and the Kalman filter gain, respectively its estimation error covariance, are derived for a simple plant model x (t + 1) = x (t) + w (t) , y (t + 1) = x (t) + v (t) , where w and v are noise terms of equal variance.It is shown that the steady-state Kalman gain and the estimation error covariance are then equal to the golden section and its conjugate, respectively.
For the more general case of a state equation and non-equal noise variances, the authors show that two particular choices of a recover a steady-state Kalman filter gain equal to the golden section, respectively the conjugate golden section.Similar golden section filter gain situations are derived in Capponi et al. (2010) for the continuous-time Kalman filter and for the non-linear Benes filter.Also somewhat related, it is shown in the article Lang (2004) that generalized Fibonacci numbers can be generated by a continuous-time Riccati differential equation.
In this article we investigate the relation between the one-dimensional linear quadratic control problem and Fibonacci numbers.More specific, we show that we can solve the corresponding discrete-time Riccati equation by introducing a generalized sequence of Fibonacci numbers and express different quantities in the LQR problem as ratios of generalized Fibonacci numbers.We also find explicit Binet type formulae based on these generalized Fibonacci numbers, removing the need to recursively find the solution at a given timestep.Finally, we show how to practically use the generalized Fibonacci sequences by numerically solving four linear quadratic examples in the last section.

General Setting of a Finite
Horizon Discrete-Time LQR Consider an r-dimensional discrete-time system x (k + 1) = Φx (k) + Γu (k) , k = 0, 1, 2, . . ., N, (1) where x (•) ∈ R r is the state vector, u (•) ∈ R s the input vector, y (•) ∈ R t the output vector, Φ ∈ R r×r the state matrix, Γ ∈ R r×s the input matrix and ∆ ∈ R t×r the output matrix.Moreover, we define the cost functional J = J (x, u) by where P is a positive definite matrix and Q is a positive semidefinite matrix.The cost functional is then minimized (see e.g.Åström and Wittenmark (1996)) by the optimal feedback control law where the feedback gain is given by and R (k) is the symmetric and positive semidefinite matrix that is the solution of the discrete-time Riccati difference equation This can be simplified as (see e.g.Balchen (1977)) where R (k) is the solution of the difference equation We will show that this particular system in dimension r = 1 has a special connection to a recurrence equation of Fibonacci type.For simplicity, we first consider the case when Φ 2 = I = 1 and later generalize the result to any Φ.

Remark 1
The above results are also true for a more general cost functional 6) It is also possible to include linear terms in the penalty functional by making appropriate transformations.
We will from hereon assume that b > 0 and a = 0 are real numbers, even though a lot of the results in this section easily can be generalized in the case of complex coefficients a and b.However, due to the identification between these coefficients and the physical parameters arising from the LQR-problem, it is natural to use the imposed restrictions.
Moreover, we introduce the negative generalized Fibonacci numbers by which in particular gives Depending on the values of a and b, we have the following situations: Theorem 3 Assume that we have defined the sequence (f n ) n=0,1,2,3,... = (f 0 , f 1 , f 2 , . ..) as in (7) above.Then Proof.Cases (1) and (3) are obvious.For case (2) with 0 < a < 1, consider the recurrence equation Then we have the two basis cases Hence the proof is complete if we can show that Indeed, the assumptions Thus the proof follows by the induction principle.In the same way, it easily follows in the case −1 < a < 0 that the n th generalized Fibonacci number is which also implies that By using a so called Binet formula, it is possible to find the n th generalized Fibonacci number f n explicitly without the need for previous values.The formula is given in the following theorem: Theorem 4 The n th generalized Fibonacci number f n is given by Proof.Let us denote and ϕ 2 = a − ϕ the other, since The difference between the solutions is then We thus need to prove the formula It easily follows that

Now assume that
Then Thus the proof follows by the induction principle.
Remark 5 The quantity ϕ is the well known Golden ratio We define the sequence of Fibonacci ratios (G n ) n=1,2,3,... by We then find the consecutive elements G n by rewriting (7) as or reversed with initial value G 1 = a.Clearly, a > 0 implies that G n > 0 and a < 0 implies that G n < 0 for all n, i.e.
Therefore aG n = a 2 + a 2 b aGn−1 ≥ a 2 and thus also aG n = a 2 + a 2 b aGn−1 ≤ a 2 +b for n = 1, 2, 3, . . . .Hence, it follows that a 2 ≤ aG n ≤ a 2 + b, with equality only for n = 1 and n = 2, respectively.In a similar fashion, it follows that the reciprocal sequence (H n ) n=1,2,3,... defined by can be generated by the recursive relation and is limited by for n = 2, 3, 4, . . ., with equality only for n = 2 and n = 3, respectively.As a consequence of the Binet formula (9), we get explicit expressions for G n and H n by the formulae Theorem 6 Both (G n ) and (H n ) are Cauchy sequences.
Proof.The identity for n = 2, 3, 4, . . . .By rewriting the multiplicative contraction factor as and using the fact that a 2 < aG n ≤ a 2 + b, we see that Thus the difference between consecutive elements in (G n ) becomes smaller and smaller.By an analogous derivation it is possible to show that for n = 2, 3, 4, . . ., where the multiplicative factor is It thus follows that both sequences are Cauchy.
Corollary 7 Hence the sequences (G n ) and (H n ) converge to the unique and finite limits Proof.The sequences are convergent since they are Cauchy.The limit for (G n ) is found by letting n → ∞ in equation ( 11) above, which yields Obviously √ a 2 + 4b > |a| .Therefore, since all elements G n are positive if a > 0 and negative if a < 0, we conclude that the limit must be where the positive sign is chosen if a > 0 and the negative sign is chosen if a < 0. The corresponding proof for (H n ) follows analogously from equation ( 12).
The sequence (G n ) oscillates around G ∞ with consecutive elements being above and below G ∞ , since if aG n−1 < aG ∞ , we get that and vice versa.The same is true for the sequence (H n ) , since Hn−1 a < H∞ a implies that The rate of convergence to the corresponding limit elements follows from the identities where the last equality follows from (15).
Moreover, since A similar calculation shows that the corresponding result is also true for the sequence (H n ) , with the constant of contraction again being Since consecutive elements of (G n ) oscillate around the limit G ∞ , it follows that the subsequences with oddindexed and even-indexed elements of (G n ) must have all elements either above or below the limit.Now hence the elements of (aG n ) n=1,3,5,... all lie below aG ∞ and the elements of (aG n ) n=2,4,6,... all lie above aG ∞ .
For this particular problem, we are only interested in the odd-indexed subsequences G o n = G 2n−1 and H o n = H 2n−1 .By using the recurrence formula twice, we can find expressions for the elements of these subsequences.Indeed, G 1 = a.
Similarly, we find the corresponding recurrence equation for the odd-indexed subsequence of (H n ) to be Alternatively, with use of the negative generalized Fibonacci numbers (8) we might also include the first negative indexed elements as initial values for the recurrence equations.
Theorem 8 The odd-indexed and even-indexed subsequences of (G n ) and (H n ) are monotone. Proof.

A Special One-Dimensional LQR Problem
In this section, we deal with the special case Φ 2 = 1 for the LQR problem (1) in dimension r = 1.We thus consider the optimal feedback control of the system subject to minimization of the functional where Q ≥ 0 and P > 0.
Remark 9 The degenerate case Q = 0 gives the explicit solutions , where Q 0 is taken from (6).Specifically, we have that R (k) = 0 and L (k) = 0 for all k when Q 0 = 0. We therefore only consider the case Q > 0 from hereon.
To ensure controllability of the system, we also require Γ = 0. Then the corresponding Riccati difference equation (5) becomes To begin with, we note that this implies that The feedback control gain (4) then becomes Inverting this relation yields Replacing R (k + 1) in ( 21) with ( 22) gives the corresponding recurrence equation for the feedback gain as , with the initial value If we divide L (k) with Φ, we get the equivalent recurrence equation However, since R (N − 1) = Q, we can also use as initial value.Moreover, by multiplying (20) with Γ/Q, we get an equivalent difference equation in the variable ,( 24) We can here alternatively use as initial value.This leads us to the theorem: Theorem 10 Assume that Φ 2 = 1 and let f n be the generalized Fibonacci numbers generated by Then the solution R (N − k) , k = 1, 2, . . ., N, of the Riccati equation ( 20) is given by and the feedback gain L (N − k) is given by Proof.Consider the definition ( 7) with Then the recurrence equation ( 17) for the odd-indexed subsequence of (G n ) becomes If we do not want to use negative-indexed values, we can replace the initial value with G 1 = a = Γ.By making the change of variable we see that this is exactly the same recurrence equation with same initial value as equation ( 24) for Γ Q R (k) , when Φ 2 = 1.By uniqueness of the solution of this kind of recurrence equations (function iteration), we find that Similarly, we get that the recurrence equation ( 18) for the odd-indexed subsequence of (H n ) becomes If we do not want to use negative-indexed values, we can replace the initial value with H 1 = 0. Using Φ 2 = 1, we note that this is exactly the same recurrence equation with same initial value as equation ( 23) for L (k) /Φ, where the relationship between indices n and Hence we find that Knowing the feedback gain, we can now compute the state and input sequences.The results are contained in the following theorem: Theorem 11 Given the initial value x (0) , we can express the state and input values x (k) and u (k) , k = 0, 1, 2, . . ., N − 1, in the system (19) as Proof.If the formula for x (k) is true, it easily follows from (27) that First, we note that the formula for x (k) is trivially true when k = 0. Now assume that the formula is true for k = p, i.e.
where the last equality follows from the definition of generalized Fibonacci numbers with m = 2 (N − p) − 1 (odd number).The induction principle completes the proof.
Moreover, we have the following result for the subsequent values of the penalty functional: Theorem 12 Let us define the value J k at timestep k of the penalty functional by Then the subsequent values J k , k = N −1, N −2, . . ., 0, are given by Proof.From (21) we have that Moreover, inversion of (20) yields that .
When m = N − 1, we trivially have that Let us now assume that (31) holds for k = p + 1, i.e.
Then it follows that Using the induction principle concludes the first part of (31).Secondly, using ( 26) and ( 28), we find that the consecutive values J k of the penalty functional can be expressed as Remark 13 Let Φ 2 = 1.By first using definition (30) together with expressions ( 28) and ( 29) to compute J 0 and then comparing with the result 2 from formula (31), one obtains the identity Moreover, by using (34) we see that By using the explicit Binet type formulae ( 13) and ( 14), we may also conclude that Corresponding Binet type formulae for x (k) , u (k) and J k may obviously be found in an analogous way.Now consider the case Γ > 0. It is obvious that (R (N − k)) k is monotonically increasing (as k increases from k = 0 to k = N ), since the corresponding generalized Fibonacci ratios are monotonically increasing by Theorem 8.More specific, in the case with infinite horizon (i.e.N → ∞) , we have that R ∞ (k) and L ∞ (k) converge to the limit values where subscript ∞ denotes infinite horizon and The case Γ < 0 follows analogously with the corresponding sign corrections.
Remark 14 The more general cost functional (6) leads to the same conclusions as above, however with the initial conditions f 0 and f 1 of the generalized Fibonacci recurrence equation ( 25) replaced with due to the change of initial value R (N ) = Q 0 .In this case we also have to redefine negative-indexed generalized Fibonacci numbers accordingly, since f 0 = 0. Thus we obtain

The General One-Dimensional Case
Let us now consider the case without restrictions on Φ, i.e. the optimal feedback control of the system subject to minimization of the functional where Q ≥ 0 and P > 0. Again, we only consider the case Q > 0, see Remark 9. We also discard the degenerate case Φ = 0, since it results in zero feedback by ( 21).Furthermore, to ensure controllability of the system we require Γ = 0. Since in general Φ 2 = 1, it is impossible to use our previous definition (25) of a generalized Fibonacci sequence to solve the recurrence equations Instead, we look for a sequence f n defined by the recurrence equation where we allow the coefficients a n and b n to vary with n.Then Moreover, we define Hence G −1 = 0, and similarly Thus we see that if we make the choice This means that we can identify . .) defined above can be split in the subsequences where f2m−1 , for m = 0, 1, 2, . . . .It is worth noting that also in this case are G n and H n well defined for n = 1, 2, 3, . . ., independent of Q 0 , since the corresponding bounds for the elements are This result is summarized in the main theorem of this article: Theorem 15 Assume that µ n is periodically varying as Let f n be the generalized Fibonacci numbers generated by and define Then the solution R (N − k) , k = 1, 2, . . ., N, of the Riccati equation ( 20) is given by and the feedback gain L (N − k) is given by Moreover, corresponding theorems (Theorem 11 and Theorem 12) for the state and input sequences and the value of the penalty functional hold true also in this case, since no other property from this more general definition have been used in their corresponding proofs.

Remark 16
In fact, if we know that By using the definition ( 42) in ( 39) and letting m → ∞, we see that where that is, Similarly, we get that Remark 17 Note that the limits G e ∞ and G o ∞ are independent of the initial values f 0 and f 1 .
In order to find explicit Binet type expressions for L (k) and R (k) , we need to be able to express the generalized Fibonacci numbers by some kind of recurrence equation with constant coefficents.First, we note that both (G o n ) and (H o n ) have even-indexed generalized Fibonacci numbers in the numerator and oddindexed generalized Fibonacci numbers in the denominator.We therefore try to find two recurrence equations with constant coefficients for every second generalized Fibonacci number, one for the odd-indexed subsequence and one for the even-indexed subsequence.Indeed, we have for n odd and for n even, i.e. the same recurrence relation.The corresponding initial values are for the odd-indexed subsequence for the even-indexed subsequence f e = (f e n ) n=0,1,2,... = (f −2 , f 0 , f 2 , . ..) .Now consider the sequence (g n ) = (g −1 , g 0 , g 1 , g 2 , ...) generated by Then If we compare this with recurrence equation ( 44) and its corresponding initial values (46), we see that the odd-indexed subsequence g o = (g −1 , g 1 , g 3 , . ..) coincides with f o = (f −1 , f 1 , f 3 , . ..) if we make the identification Thus we can generate the odd-indexed subsequence f o by taking every second element g −1 , g 1 , g 3 , . . .generated by the recurrence equation By following the same idea as in the proof of the Binet formula ( 9), we see that we can write where are the roots of the corresponding characteristic equation Hence we can conclude that Similarly, we can generate the even-indexed subsequence (f e n ) by taking every second element (h −2 , h 0 , h 2 , . ..) generated by the recurrence equation In this case, the corresponding Binet type formula becomes i.e. we can conclude that where Remark 18 Note that all formulae in the special case Φ 2 = 1 coincide with those given in the previous section.
Remark 19 Another way to show this Binet type formula is to transform the Riccati difference equation (equivalent to eq. ( 38)), by the change of variables This yields the equivalent one-parameter difference equation A second change of variables linearizes this equation to the resulting linear second order difference equation in y n y k+2 − y k+1 + ρy k = 0, which easily can be solved.For details, see e.g. the books Camouzis and Ladas (2008), Elaydi (2005) and Kulenović and Ladas (2002).
Remark 20 To handle the more general cost functional (6), one needs to replace the initial values with or equivalently,

Some Numerical Examples
We will illustrate the use of generalized Fibonacci sequences in four different examples.
Example 21 Let N = 6.We consider the process under the maximization of the functional Moreover, we assume that we start with a value of ξ(0) = 9.First, we see that the change of variable x (k) = ξ (k) − 10 transforms the problem to the equivalent LQR problem with initial value x (0) = −1, subject to minimization of the functional The change of variable implies that the optimal path ξ (k) is offset by 10, i.e. ξ (k) = x (k) + 10.Here P = Q = 1, Φ = 1 and Γ = 1, thus by using expressions (20), ( 21), ( 19), ( 2) and (30), we can generate Table 1.To see the relation with the sequence of Fibonacci numbers F n , we need to consider the consecutive ratios of the numbers F n generated by the recurrence formula In this case, we get the familiar sequence in Table 2. Clearly, the values of x (k) , u (k) and J k all coincide with the predicted values from ( 28), ( 29) and ( 31).Moreover, we generate the sequences of consecutive ratios G n = Fn+1 Example 22 Let N = 5.We consider the process with initial value x (0) = 1, subject to minimization of the functional Here P = 15, Q = 5 2 , Φ = −1 and Γ = 5.Thus by using expressions ( 20), ( 21), ( 19), ( 2) and ( 30), we can generate Table 4.In this case, we need to find the generalized Fibonacci numbers f n generated by the recurrence formula f n = Γf n−1 + P Q f n−2 = 5f n−1 + 6f n−2 .This gives the sequence in Table 5, resulting in the sequences of consecutive ratios G n = fn+1 fn and H n = fn−1 fn given by Table 6.Again, we note that the odd-indexed elements of G n multiplied with Q Γ = 1 2 (bold face) coincide with R (k) and the odd-indexed elements of H n (bold face) multiplied with Φ = −1 coincide with L (k), where n = 2 (N − k) − 1 = 9 − 2k, k = 0, 1, 2, 3, 4. The characteristic equation is in this case with solutions Thus formulae (32) and ( 33) give the explicit expres- Moreover, if we denote we note that R (k) is getting closer and closer to Q Γ G ∞ = 3 and that L (k) is getting closer and closer to Φ 1 G∞ = − 1 6 as k decreases.We also see that the error times for every k (decreasing), as predicted by formula ( 16) taken two times (only odd-indexed G n in R (k)).
Example 23 Let N = 4.We consider the process with initial value x (0) = 3, subject to minimization of the functional Here Q 0 = 6, P = 15, Q = 5 2 , Φ = 1 and Γ = −5.Thus by using expressions (20), ( 21), ( 19), ( 2) and (30) with R (N ) = Q 0 , we can generate Table 7.In this case, we need to find the generalized Fibonacci numbers f n generated by the recurrence formula f Here we need to replace the initial values f 0 and f 1 by This gives the sequence in given by Table 9.Again, we note that the oddindexed elements of G n multiplied with Q Γ = − 1 2 (bold face) coincide with R (k) and the odd-indexed elements of H n (bold face) multiplied with Φ = 1 coincide with L (k), where n = 2 (N − k)−1 = 7−2k, k = 0, 1, 2, 3, 4. If we compare with Example 22, we see that also in this case will R (k) come closer and closer to independent of the initial value Q 0 (and thus also independent of f 0 and f 1 ).
Example 24 Let N = 4.We consider the process with initial value x (0) = 1, subject to minimization of the functional Here P = 15, Q = 5 2 , Φ = 3 and Γ = 5.Thus by using expressions (20), ( 21), ( 19), ( 2) and (30), we can generate Table 10.In this case, we need to find the generalized Fibonacci numbers f n generated by the recurrence formula f This gives us the sequence in Table 11, resulting in the sequences of consecutive ratios G n = fn+1 fn and H n = fn−1 fn given by Table 12.As before, we note that the odd-indexed elements of G n (bold face) multiplied with Q Γ = 1 2 coincide with R (k) and the odd-indexed elements of H n multiplied with Φ = 3 (bold face) coincide with L (k), where n Another method to find G n and H n (using recurrence equations with constant coefficients) is to take every second element g −1 , g 1 , g 3 , ... generated by the recurrence equation and every second element (h −2 , h 0 , h 2 , ...) generated by the recurrence equation .
Then the odd-indexed elements of (g n ) (bold face) coincide with the odd-indexed elements of (f n ) and the even-indexed elements of (h n ) (bold face) coincide with the even-indexed elements of (f n ) , see Table 13.These two subsequences form the basis of the Binet type for- Table 9: Ratios of consecutive numbers from Table 8. mulae ( 48) and ( 49), which in this case become Finally, we note that R (k) is getting closer and closer to and that L (k) is getting closer and closer to as k decreases.2795225 Table 13: Another method to find generalized Fibonacci numbers corresponding to Example 24.

Discussion and Conclusions
In this article, we found an efficient way to solve onedimensional LQR-problems.We showed that to each specific problem, there is a corresponding Fibonacci type sequence in terms of which the solution can be written.More precisely, it turns out that the solution of the corresponding Riccati equation, the feedback gain, the controller state, the input and the subsequent penalty functional values all can be expressed as ratios of subsequent values generated by this Fibonacci type sequence.Moreover, this way of solving the LQR problem naturally implies that we can find direct explicit formulae (so called Binet type formulae) that computes the k th desired value of some property, without knowledge of previous values.The article is concluded by showing the use of the theory and the formulae in four separate examples of increasing complexity.

Table 1 :
Numerical values corresponding to Example 21.

Table 2 :
Fibonacci numbers corresponding to Example 21.

Table 3 :
Ratios of consecutive numbers from Table2.

Table 4 :
Numerical values corresponding to Example 22.

Table 5 :
Generalized Fibonacci numbers corresponding to Example 22.

Table 6 :
Ratios of consecutive numbers from Table5.

Table 7 :
Numerical values corresponding to Example 23.Moreover, the negative indexed numbers f −1 and f −2

Table 10 :
Numerical values corresponding to Example 24.

Table 11 :
Generalized Fibonacci numbers corresponding to Example 24.

Table 12 :
Ratios of consecutive numbers from Table11.