Solutions to Assignment 3 by evx17276


									         MATH 335                                                                           Winter 2010

                     Solutions to Assignment 3
Problem 1: Haar Wavelets [20 Points]

One practically important basis is the class of Haar functions (wavelets). We defined in class Haar functions
as follows.

                                                        1, if 0 ≤ x ≤ 1
                                         Ψ0,0 (x) =                                                               (1)
                                                        0 else

                                                     if 0 ≤ x < 1/2
                                       Φ0,0 (x) = −1, if 1/2 ≤ x ≤ 1                                              (2)
                                                   0  else

Let for n ∈ N+ and k ∈ {0, 1, 2, . . . , 2n − 1}

                                               1, if k2−n ≤ x ≤ (k + 1)2−n
                                 Ψn,k (x) =                                                                       (3)
                                               0 else

                                    2n/2 ,
                                            if k2−n ≤ x < (k + 1/2)2−n
                          Φn,k (x) = −2n/2 , if (k + 1/2)2−n ≤ x ≤ (k + 1)2−n                                     (4)
                                     0       else

Show that the family
                                  {Ψ0,0 , Φn,k , n ∈ Z+ , k ∈ {0, 1, 2, . . . , 2n − 1}}
is complete and orthonormal in L2 ([0, 1]; R) and is a sequence (that is, countable).
Hint: You may wish to use/assume the fact that the space of R−valued continuous functions on [0, 1] are
dense in L2 ([0, 1]; R).

We will show that the set is orthonormal, countable and complete.
For any two pair (n, k), (n′ , k ′ ) which are different, we will show that φn,k , φn′ ,k′ = 0. If n = n′ , suppose
n > n′ ; then φn′ k′ is either constant on the support set (that is the set of points on which the function is
non-zero) of φn,k that is for the interval [ 2k , k+1 ]. If n = n′ , then for k = k ′ , the support sets are disjoint.
                                               n   2n

Finally, for any allowable pair n, k
                                                ||φn,k || = 2n 2−n = 1.

As such, the family is orthonormal.
The sequence is countable, since we can enumerate the entries starting from 0, 0 to {n, k, {k ∈ 0, 1, 2, . . . , 2n −
1}, n ∈ N+ }, and by this ordering ensure that every element will eventually be included in the counting
The sequence is also complete. To see this observe the following. A function which has a non-zero norm
should be one which has a non-zero value on some interval; otherwise by the definition of the Lebesgue
integral, the function would have zero integration and zero L2 −norm.
As such, suppose there is a function, which has non-zero norm which is orthogonal to all of the elements
in the countable set described above. By, the argument above, there must exist an open interval on which
the function is non-zero. Call this open interval (a, b), 0 ≤ a < b ≤ 1. Now, there exists an (n, k) pair such
that, 0 ≤ k ≤ 2n − 1, k, n ∈ N pair such that, a < k2−n < (k + 1)2−n b (for example by picking n to be
greater than or equal to ⌈log2 (b − a)⌉). Without any loss, suppose that the function takes a non-negative
value in this interval (if it can take negative values also, restrict (a, b) to an even smaller interval, or only
consider the negative portion).
Furthermore, there exists a linear combination of the elements in the set

                                    {Ψ0,0 , Φn,k , k ∈ {0, 1, 2, . . . , 2n − 1}},

such that the linear combination is constant in the interval [k2−n , (k + 1)2−n ]. But then, the inner product
of this constant function and the function defined on the portion (a, b) is non-zero.
As such, a function, which is not zero almost everywhere, cannot be orthogonal to every element in the
countable set.
A function which is zero almost everywhere is the null-element under the L2 −norm. This concludes the

Problem 2: Linear Functionals on a Normed Space

Let f be a linear functional on a normed linear space X (mapping X to R). Suppose that we define f to
be bounded if there is a constant M such that |f (x)| ≤ M ||x|| for all x ∈ X.
The smallest such M is called the norm of f and is denoted by ||f ||. The space of all bounded, linear
functionals on X is called the dual space of X.
Show that a linear functional on a normed linear space is bounded if and only if it is continuous.

a) Boundedness implies continuity: For every ǫ > 0, there exists a δ > 0 such that ||x − y|| ≤ ǫ implies that

                                 |f (x) − f (y)| = |f (x − y)| ≤ M ||x − y|| ≤ ǫ,

with δ = ǫ/M .
b) Continuity implies boundedness: Suppose the linear function is not bounded.
Let us make the following observation. A linear function on a normed linear space is continuous at any
point if and only if it is continuous at the null element, as we discussed earlier.
Suppose the function is not bounded. This means that there exists an element x∗ ∈ X such that |f (x∗ )| ≥
M ||x∗ || for all M ∈ R. We can construct a sequence of signals {x∗ } = n x∗ such that x∗ → 0. By linearity,
                                                                  n                     n
for every n ∈ N,
                                    x∗     1         1
                                 f ( ) = f (x∗ ) ≥ M ||x∗ ||, ∀M ∈ R,
                                    n      n         n
and that there is a uniform bound L such that f (xn∗ ) ≥ L for all n.

But the sequence {f (x∗ )} does not converge to 0, for otherwise for every ǫ there would exist an N such
that for all n > N , |f (x∗ )| ≤ ǫ. This is not possible by the argument in the previous paragraph.

Problem 3: Riesz Representation Theorem

Let f be a linear functional on l2 (N; R); thus, mapping l2 (N; R) to R.
Show that for every such f , there exists a vector κ ∈ l2 (N; R) such that

                                           f (x) = x, κ ,   ∀x ∈ l2 (N; R),
where the inner-product is the one giving rise to the l2 -norm in l2 (N; R).
Hint: We showed this in class for the case when the space is Rn for some n ∈ N.

Let ei ∈ l2 (N; R), i ∈ N be given by:
                                              ei (m) = 1i=m ,      m∈N

Note that {ei , i ∈ N} is a complete, orthonormal sequence in l2 (N; R). Thus, for every x ∈ l2 (N; R), we can
                                       x=      x, ei em =      x(m)em
                                              m∈N                 m∈N

                             f (x) = f (       x(m)em ) =          x(m)f (em ) = x, κ ,
                                         m∈N                m∈N

                                              κ(k) = f (ek ),     ∀k ∈ N

Problem 4: Weak Convergence

In this problem we will show that the usual notion of convergence (that is strong convergence) implies
weak convergence for a Hilbert space. Let {xn } ∈ L2 (R+ ; R).
a) Show that if {xn } → x, then {< xn , f >} → x, f             ∀f ∈ L2 (R+ ; R), that is convergence in strong sense
implies convergence in weak sense.
b) Possibly via a counterexample, show that {xn } → x weakly does not imply convergence in the strong
Hint: Recall that {xn } → x means that {||xn − x||2 } → 0. You may think about the example we gave in
class for l2 (N; R).

a) The proof is immediate once one observes:

                < xn , f > − < x∗ , f >=< xn − x∗ , f >≤ ||xn − x∗ ||2 ||f ||2 ≤ M ||xn − x∗ ||2 ,
for some M < ∞. Hence, if xn → x∗ , weak convergence follows.
b) Consider the sequence:
                                                 xn (m) = 1m=n

                                            ||xn − 0|| = 1,      ∀n ∈ N

On the other hand, for any v ∈ L2 (R+ ; R):

                                                   xn , v = v(n)

Since ||v||2 < ∞, it follows that v(n) → 0 as n → ∞.
This implies that the sequence
                                                        {xn }
converges to the null element 0 weakly, but not in the l2 norm.

Problem 5: [Matlab Assignment]

Let C([−1, 1]) denote the space of continuous functions from [−1, 1] to R. We observed in class that
polynomials can be used to approximate any function in this space with arbitrary precision, under the
supremum norm (Weierstrass Theorem).
The polynomials we considered were in the family {fi , fi (t) = ti , i ∈ N}. Since C([−1, 1]) is dense in
L2 ([−1, 1]; R), these polynomials can be used to generate a complete orthonormal sequence in L2 ([−1, 1]; R).
These polynomials are not orthonormal, but we could orthonormalize them via the Gram-Schmidt proce-
dure. In fact, this leads to the Legendre Polynomials given by:
                                                        (−1)n     dn
                                 en (t) =    2n + 1/2                 (1 − t2 )n .
                                                         2n n!    dtn

Now, let us consider L2 ([0, 1]; R) (as the representation for the polynomials are somewhat simpler when
the domain is [0, 1] instead of [−1, 1], but the extension is a minor technicality).
One class of polynomials which can be used to provide the approximation is the family of Bernstein
polynomials, defined as follows: Let for some f ∈ C([0, 1]):
                                                         k n k
                                    Bn,f (t) =         f( )  t (1 − t)n−k
                                                         n k

Write a Matlab function which admits a trigonometric function f ∈ C([0, 1]), and n as its inputs and
generates the Bernstein polynomial approximation of the signal.
You may take f to be for example f = 2 sin(10πt) + 3 cos(20πt).
Given a trigonometric function of your choice, compute the Bernstein approximations of order n, where n
can take three values and verify that the supremum difference
                                              sup |f (t) − Bn,f (t)|,

decreases as n increases.
Thus, we have now learned three families of functions which can be used to generate complete orthonormal
sequences in L2 ([0, 1]; R): The Fourier series, Polynomials and Haar Wavelets. These are all possible since
L2 ([0, 1]; R) is a Hilbert space, which is separable. The same discussion applies to an arbitrary interval
[a, b], a, b ∈ R, and even the unbounded intervals of the form [a, ∞) (but with further technical details), as
a result of separability.

To top