VIEWS: 2 PAGES: 64 POSTED ON: 2/24/2013
Solutions for Nonlinear Equations Lecture 8 Alessandra Nardi Thanks to Prof. Newton, Prof. Sangiovanni, Prof. White, Jaime Peraire, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Last Lecture Review • How to represent circuits – MNA – Voltage sources – How to assemble, X-stamp • How to solve linear systems – Gaussian elimination – LU decomposition Outline • Nonlinear problems • Iterative Methods • Newton’s Method – Derivation of Newton – Quadratic Convergence – Examples – Convergence Testing • Multidimensonal Newton Method – Basic Algorithm – Quadratic convergence – Application to circuits Nonlinear Problems - Example 1 Vd I1 Ir Id I d I s (e Vt 1) 0 0 Need to Solve I r I d I1 0 e1 1 e1 I s (e 1) I1 0 Vt g (e1 ) I1 R Nonlinear Equations • Given g(V)=I • It can be expressed as: f(V)=g(V)-I Solve g(V)=I equivalent to solve f(V)=0 Hard to find analytical solution for f(x)=0 Solve iteratively Nonlinear Equations – Iterative Methods • Start from an initial value x0 • Generate a sequence of iterate xn-1, xn, xn+1 which hopefully converges to the solution x* • Iterates are generated according to an iteration function F: xn+1=F(xn) Ask • When does it converge to correct solution ? • What is the convergence rate ? Newton-Raphson (NR) Method Consists of linearizing the system. Want to solve f(x)=0 Replace f(x) with its linearized version and solve. df * f ( x) f ( x ) ( x )( x x* ) * Taylor Ser ies dx k 1 df k k 1 k f ( x ) f ( x ) ( x )( x x ) k dx 1 k 1 df k x x ( x ) f ( x ) k k Iteration function dx Note: at each step need to evaluate f and f’ Newton-Raphson Method – Graphical View Newton-Raphson Method – Algorithm Define iteration Do k = 0 to …. 1 k 1 df k x x ( x ) f ( x k ) k dx until convergence • How about convergence? • An iteration {x(k)} is said to converge with order q if there exists a vector norm such that for each k N: k 1 q x x x x ˆ k ˆ Newton-Raphson Method – Convergence df k * k d 2 f 0 f ( x* ) f ( x k ) ( x )( x x ) 2 ( x)( x* x k ) 2 dx dx some x [ x k , x* ] Mean Value theorem truncates Taylor series But df k k 1 k by Newton 0 f ( x ) ( x )( x x ) k dx definition Newton-Raphson Method – Convergence df k k 1 * d 2 f Subtracting ( x )( x x ) 2 ( x)( x x ) k * 2 dx d x df k 1 d 2 f Dividing through ( x k 1 x* ) [ ( x )] ( x)( x k x* ) 2 dx d 2x df k 1 d 2 f Let [ ( x )] 2 ( x) K k dx d x k 1 * 2 then x x K x x * k k Convergence is quadratic Newton-Raphson Method – Convergence Local Convergence Theorem If df a) bounded away from zero dx K is bounded d2 f b) bounded dx 2 Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic) Newton-Raphson Method – Convergence Example 1 f ( x) x 2 1 0, find x ( x* 1) df k ( x ) 2 xk dx k 2x (x k 1 x ) x k k 2 1 x 2 2 k 1 k 2x (x x ) 2x (x x ) x * k * k k * k 1 1 or ( x x ) k ( x k x* ) 2 * 2x Convergence is quadratic Newton-Raphson Method – Convergence Example 2 f ( x) x 0, x 0 2 * 1 df df k Note : not bounded ( x ) 2 xk dx dx away from zero k 1 2 x ( x 0) ( x 0) k k 2 k 1 x 0 x 0 1 k 2 for x k x* 0 1 or ( xk 1 x ) ( xk x ) * * 2 Convergence is linear Newton-Raphson Method – Convergence Example 1, 2 Newton-Raphson Method – Convergence x0 = Initial Guess, k 0 Repeat { f x k x x k 1 x k f x k k k 1 } Until ? x k 1 x threshold ? k f x k 1 threshold ? Newton-Raphson Method – Convergence Convergence Checks Need a "delta-x" check to avoid false convergence f(x) x k 1 x xa xr x k k 1 k 1 k * x x x X f x k 1 fa Newton-Raphson Method – Convergence Convergence Checks Also need an "f x " check to avoid false convergence f(x) f x k 1 fa * x X x k 1 x k x k 1 x xa xr x k k 1 Newton-Raphson Method – Convergence demo2 Newton-Raphson Method – Convergence Local Convergence Convergence Depends on a Good Initial Guess f(x) 1 1 x x 2 0 0 X x x x Newton-Raphson Method – Convergence Local Convergence Convergence Depends on a Good Initial Guess Nonlinear Problems – Multidimensional Example v1 b i2 + v2 - v2 Nodal Analysis i1 i3 At Node 1: i1 i2 0 + + g v1 g v1 v2 0 b b v v 1 3 At Node 2: i3 i2 0 - - g v3 g v1 v2 0 Nonlinear Resistors Two coupled i g v nonlinear equations in two unknowns Multidimensional Newton Method Problem: Find x such that F x 0 * * x * N and F : N N F ( x) F ( x ) J ( x )( x x ) * * * Taylor Ser ies F1 ( x) F1 ( x) x x N 1 J ( x) Jacobian Matrix FN ( x) FN ( x) x1 x N k 1 k 1 x x J (x ) F (x ) k k Iteration function Multidimensional Newton Method Computational Aspects Iteration : x k 1 x k J ( x k ) 1 F ( x k ) k 1 Do not compute J ( x ) (it is not sparse). Instead solve : J ( x k )( x k 1 x k ) F ( x k ) Each iteration requires: 1. Evaluation of F(xk) 2. Computation of J(xk) 3. Solution of a linear system of algebraic equations whose coefficient matrix is J(xk) and whose RHS is -F(xk) Multidimensional Newton Method Algorithm x0 = Initial Guess, k 0 Repeat { Compute F x k , J F x k Solve J F x x x F x for x k k 1 k k k 1 k k 1 } Until x k 1 x k , f x k 1 small enough Multidimensional Newton Method Convergence Local Convergence Theorem If a) J F1 x k Inverse is bounded b) J F x J F y x y Derivative is Lipschitz Cont Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic) Application of NR to Circuit Equations 1 I1 Ir Id f Vk 1 f Vk f Vk Vk 1 Vk 0 0 1 V1 f V V g V 1 I1 , g V ISe 1 V R1 1 Vk1 Vk1 1 I V f Vk 1 k I S e I1 S e Vk 1 Vk 0 V V R1 R1 V vk 1 Ik rdk Application of NR to Circuit Equations Companion Network – MNA templates Note: G0 and Id depend on the iteration count k G0=G0(k) and Id=Id(k) Application of NR to Circuit Equations Companion Network – MNA templates Modeling a MOSFET (MOS Level 1, linear regime) d Modeling a MOSFET (MOS Level 1, linear regime) Modeling a MOSFET • Need continuous models with continuous derivatives • Example: – simple MOS model valid from weak inversion to saturation X2 I DS K 1 X VGS VT X 2V ln 1 e 2V DC Analysis Flow Diagram For each state variable in the system Implications • Device model equations must be continuous with continuous derivatives and derivative calculation must be accurate derivative of function (not all models do this - Poor diode models and breakdown models don’t - be sure models are decent - beware of user-supplied models) • Watch out for floating nodes (If a node becomes disconnected, then J(x) is singular) • Give good initial guess for x(0) • Most model computations produce errors in function values and derivatives. Want to have convergence criteria || x(k+1) - x(k) || < such that > than model errors. Summary • Nonlinear problems • Iterative Methods • Newton’s Method – Derivation of Newton – Quadratic Convergence – Examples – Convergence Testing • Multidimensional Newton Method – Basic Algorithm – Quadratic convergence – Application to circuits Methods for Ordinary Differential Equations Lecture 10 Alessandra Nardi Thanks to Prof. Jacob White, Deepak Ramaswamy Jaime Peraire, Michal Rewienski, and Karen Veroy Outline • Transient Analysis of dynamical circuits – i.e., circuits containing C and/or L • Examples • Solution of Ordinary Differential Equations (Initial Value Problems – IVP) – Forward Euler (FE), Backward Euler (BE) and Trapezoidal Rule (TR) – Multistep methods – Convergence Application Problems Signal Transmission in an Integrated Circuit Signal Wire Wire has resistance Wire and ground plane form a capacitor Logic Logic Gate Gate Ground Plane • Metal Wires carry signals from gate to gate. • How long is the signal delayed? Application Problems Signal Transmission in an IC – Circuit Model resistor capacitor Constructing the Model • Cut the wire into sections. • Model wire resistance with resistors. • Model wire-plane capacitance with capacitors. Application Problems Signal Transmission in an IC – 2x2 example v1 iR 2 v2 Constitutive Conservation Equations Laws iC1 iR1 R2 iR3 iC2 ic C dvc iC1 iR1 iR2 0 C1 R1 R3 C2 dt 1 iR vR iC2 iR3 iR2 0 R Nodal Equations Yields 2x2 System dv1 1 1 1 R R C1 0 dt R2 v1 1 2 0 C2 dv2 1 1 1 v2 dt R 2 R3 R2 Application Problems Signal Transmission in an IC – 2x2 example v1 (0) 1 v2 (0) 0 Notice two time scale behavior • v1 and v2 come together quickly (fast eigenmode). • v1 and v2 decay to zero slowly (slow eigenmode). Circuit Equation Formulation • For dynamical circuits equations can be written compactly: dx(t ) F( , x, t ) 0 dt x(0) x0 where x is the vector of circuit va riables • For sake of simplicity, we shall discuss first order ODEs in the form: dx(t ) f ( x, t ) dt Ordinary Differential Equations Initial Value Problems (IVP) Solve Initial Value Problem (IVP) : dx(t ) f ( x, t ) dt x(t0 ) x0 in an interval [t0 ,T] given the initial condition x0 . Typically analytic solutions are not available solve it numerically Ordinary Differential Equations Assumptions and Simplifications • Not necessarily a solution exists and is unique for: dy F ( , y, t ) 0 dt • It turns out that, under rather mild conditions on the continuity and differentiability of F, it can be proven that there exists a unique solution. dy We shall assume that F ( , y, t ) 0 has a unique solution dt • Also, for sake of simplicity only consider dx(t ) Ax(t ) linear case: dt x(t0 ) x0 Finite Difference Methods Basic Concepts First - Discretize Time t t t 0 t1 t2 t L 1 tL T Second - Represent x(t) using values at ti ˆ x2 ˆ3 x4 ˆ l ˆ x 1 x x ˆ Approx. x(tl ) Exact sol’n sol’n 0 t1 t2 t3 tL Third - Approximate d x(tl ) using the discrete x 's ˆl dt d x l x l 1 ˆ ˆ x l 1 x l ˆ ˆ Example: x(tl ) or dt tl tl 1 Finite Difference Methods Forward Euler Approximation d slope x(tl ) x dt d x(tl 1 ) x(tl ) x(tl ) A x(tl ) dt t x(tl 1 ) x(tl ) or slope t x(tl 1 ) x(tl ) t A x(tl ) tl tl 1 t x(tl 1 ) x(tl ) t A x(tl ) Finite Difference Methods Forward Euler Algorithm x(t1 ) x1 x(0) tAx 0 ˆ x x(t2 ) x 2 x1 tAx1 ˆ ˆ ˆ tAx1 ˆ tAx(0) x(t L ) x L x L 1 tAx L 1 ˆ ˆ ˆ t1 t2 t3 t Finite Difference Methods Backward Euler Approximation d slope x(tl 1 ) x dt d x(tl 1 ) x(tl ) x(tl 1 ) A x(tl 1 ) dt t x(tl 1 ) x(tl ) or slope t x(tl 1 ) x(tl ) t A x(tl 1 ) tl tl 1 t x(tl 1 ) x(tl ) t A x(tl 1 ) Finite Difference Methods Backward Euler Algorithm Solve with Gaussian Elimination x x(t1 ) x1 x(0) tAx1 ˆ ˆ [ I tA] x1 x(0) ˆ tAx 2 ˆ x(t2 ) x 2 [ I tA] 1 x1 ˆ ˆ tAx1 ˆ t1 t2 t x(t L ) x L [ I tA] 1 x L 1 ˆ ˆ Finite Difference Methods Trapezoidal Rule Approximation 1 d d d ( x(tl 1 ) x(tl )) slope dt x(tl ) d 2 dt dt slope x(tl 1 ) x dt 1 ( Ax(tl 1 ) Ax(tl )) 2 x(tl 1 ) x(tl ) x(tl 1 ) x(tl ) slope t t 1 x(tl 1 ) x(tl ) tA( x(tl 1 ) x(tl )) 2 t 1 1 ( x(tl 1 ) tAx(tl )) ( x(tl ) tAx(tl 1 )) 2 2 Finite Difference Methods Trapezoidal Rule Algorithm Solve with Gaussian Elimination t x(t1 ) x x(0) Ax(0) Ax1 ˆ 1 ˆ x 2 t 1 t I A x I A x (0) ˆ 2 2 1 t t ˆ x ( t2 ) x2 I ˆ A I A x 1 2 2 t1 t2 t 1 t t x (t L ) ˆL I x A I ˆ A x L1 2 2 Finite Difference Methods Numerical Integration View d tl 1 x(t ) A x (t ) x(tl 1 ) x(tl ) Ax( )d dt tl t Ax(tl ) Ax(tl ) Trap tl 1 tl Ax( )d 2 tAx(tl 1 ) BE tAx(tl ) FE tl tl 1 Finite Difference Methods Summary of Basic Concepts Trap Rule, Forward-Euler, Backward-Euler Are all one-step methods ˆ l is computed using only x l 1 , not x l 2 , x l 3 , etc. x ˆ ˆ ˆ Forward-Euler is simplest No equation solution explicit method. Boxcar approximation to integral Backward-Euler is more expensive Equation solution each step implicit method Trapezoidal Rule might be more accurate Equation solution each step implicit method Trapezoidal approximation to integral Convergence Analysis Convergence Definition Definition: A finite-difference method for solving initial value problems on [0,T] is said to be convergent if given any A and any initial condition max T x x l t 0 as t 0 ˆl l0, t x computed with t ˆl t ˆ l x computed with 2 xexact Convergence Analysis Order-p Convergence Definition: A multi-step method for solving initial value problems on [0,T] is said to be order p convergent if given any A and any initial condition x x l t C t p max T ˆl l0, t for all t less than a given t0 Forward- and Backward-Euler are order 1 convergent Trapezoidal Rule is order 2 convergent Multistep Methods – Convergence Analysis Two types of error The Local Truncation Error (LTE) of an integratio n method at tl 1 is the difference between th e computed value x l 1 and the exact ˆ value of the solution x(tl 1 ), assuming no previous error has been made. The Global Truncation Error (GTE) of an integratio n method at tl 1 is the difference between the computed value x l 1 and the exact ˆ value of the solution x(tl 1 ), assuming that only the initial condition is known exactly. Multistep Methods – Convergence Analysis Two conditions for Convergence • For convergence we need to look at max error over the whole time interval [0,T] – We look at GTE • Not enough to look at LTE, in fact: – As I take smaller and smaller timesteps t, I would like my solution to approach exact solution better and better over the whole time interval, even though I have to add up LTE from more timesteps. Multistep Methods – Convergence Analysis Two conditions for Convergence 1) Local Condition: One step errors are small (consistency) Typically verified using Taylor Series 2) Global Condition: The single step errors do not grow too quickly (stability) All one-step methods are stable in this sense. One-step Methods – Convergence Analysis Consistency definition Definition: A one-step method for solving initial value problems on an interval [0,T] is said to be consistent if for any A and any initial condition x x t ˆ 1 0 as t 0 t 2 bounded derivatives of x are x 2 d t ˆ t x 1x Proves the theorem if 2 td 2 d gnitcartbus dna )0(xA )0(x taht gnitoN 2 td x 2 d t 0 xd t 0 x ) t (x 2 td 2 td sdleiy orez tuoba t ni gnidnapxE ˆ t ,0 0 xAt 0 x 1x noitinifed reluE-drawroF Consistency for Forward Euler One-step Methods – Convergence Analysis x 2 d 2 2 ] T,0 [ xam 5.0 C erehw , t C le td yb dednuob rorre "pets-eno" eht si le erehw l e t l xAt t l x t 1 l x sdleiy t l tuoba t ni gnidnapxE l ˆ ˆ ˆ xAt l x 1 l x noitinifed reluE-drawroF Convergence Analysis for Forward Euler One-step Methods – Convergence Analysis 2 l 1 l t C E At I E 2 l t C E A t 1 l e no dnuob eht gnisu dna smron gnikaT l e l E At I 1 l E ˆ tl x lx lE rorre "labolG" eht enifeD l ˆ ˆ e t l x lx At I t 1 l x 1 lx snoitauqe edils suoiverp eht gnitcartbuS Convergence Analysis for Forward Euler One-step Methods – Convergence Analysis Example V2 R V1 1 dV 1 R V1 C 1 V2 0 dt R + C 1 1 VS - I1 V1 V2 I1 0 R R V2 VS BE FE 1 1 Vk11 Vk1 1 2 1 1 Vk11 Vk1 1 2 Vk 1 C Vk 1 0 Vk C Vk 0 R t R R t R 1 1 1 1 1 1 2 Vk 1 Vk21 I k 1 0 Vk Vk I k 0 1 1 2R R 2R R Vk 1 VS Vk VS 1 1 Vk11 1 2 Vk1 Vk11 1 1 Vk1 1 2 Vk 1 C Vk 1 C C Vk C Vk R t R t t R t R 1 1 1 1 1 1 2 Vk 1 Vk21 I k 1 0 Vk Vk I k 0 1 1 2R R 2R R Vk 1 VS Vk VS Conclusions • Introduced basic non-linear equation solution techniques – How to get good initial points? – Is it efficient? • Introduced basic differential eqns solution techniques – Stability? – How to choose time-step – Stiffness – Other methods?