2015年11月26日 星期四

[線性代數] Orthonormal Basis 與 Gram-Schmidt Process (1)

延續前篇 [線性代數] Orthonormal Basis 與 Gram-Schmidt Process (0) 的問題,以下我們正式引入 Gram-Schmidt Process


Theorem: Gram-Schmidt Process
令 $V$ 為 有限維度內積空間 且 令 $W\neq \{ {\bf 0}\}$ 為 $V$中的 $m$-維子空間。則此子空間 $W$ 存在一組正交基底 $T =\{{\bf w}_1,...{\bf w}_m\}$

Proof:
我們首先建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2...,{\bf v}_m\}$ for $W$。由於 $W$ 為 $V$ 的子空間,故我們可在 $W$ 其上選取一組基底,令 $S=\{{\bf u}_1,...,{\bf u}_m\} $ 接著我們選取其中任意一個向量,比如說 ${\bf u}_1 \in S$ 並稱此向量為 ${\bf v}_1$ 亦即我們重新定義
\[
{\bf v}_1 := {\bf u}_1
\]注意到此 ${\bf v}_1 \in  W_1:=span\{ v_1 \}$ 其中 $W_1$ 為 $W$ 的子空間

接著我們要尋找 ${\bf v}_2$,我們希望此向量 ${\bf v}_2$ 落在 $W$ 子空間 $W_2 = span\{ {\bf u}_1, {\bf u}_2\} $ 且 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal。但注意到我們有 ${\bf v}_1 := {\bf u}_1 $ 故 ${\bf v}_2  \in W_2 = span\{ {\bf u}_1 , {\bf u}_2\}  =  span\{ {\bf v}_1, {\bf u}_2\} $  也就是說 ${\bf v}_2$ 可透過 ${\bf v}_1$ 與 ${\bf u}_2$ 做線性組合
\[
{\bf v}_2 = a_1 {\bf v}_1 + a_2 {\bf u}_2
\]其中 $a_1, a_2$ 待定。 注意到由於我們要讓 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal 故 $\langle {\bf v}_2, {\bf v}_1 \rangle = 0$ 故現在觀察
\[\begin{array}{l}
\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle  = 0\\
 \Rightarrow \langle {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2},{{\bf{v}}_1}\rangle  = 0\\
 \Rightarrow {a_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle  + {a_2}\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle  = 0\\
 \Rightarrow {a_1} =  - {a_2}\frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }} \;\;\;\; (*)
\end{array}\]注意到上式中 $\langle {\bf v}_1, {\bf v}_2 \rangle \neq 0$ 因為 ${\bf v}_1 = {\bf u}_1 \in S$ 且 $S$ 為 非零子空間 $W$ 的基底 。注意到 $(*)$ 為一條方程式兩個未知數 $a_1,a_2$ 故可任意令 $a_2 \in \mathbb{R}^1$ 為自由變數解得 $a_1$ 。為了計算方便起見我們選 $a_2 :=1$ 則
\[{a_1} =  - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}
\]此說明了
\[\begin{array}{l}
{{\bf{v}}_2} = {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} =  - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} + {{\bf{u}}_2} = {{\bf{u}}_2} - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1}
\end{array}\]至此我們有了一組 orthogonal subset $\{{\bf v}_1, {\bf v}_2\}$ for $W$。

接著我們尋找 ${\bf v}_3 \in W_3 := span\{{\bf u}_1, {\bf u}_2, {\bf u}_3\}$ 且 ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal。注意到
\[{W_3}: = span\{ {{\bf{u}}_1},{{\bf{u}}_2},{{\bf{u}}_3}\}  = span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \]故
\[\begin{array}{l}
{{\bf{v}}_3} \in span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \\
 \Rightarrow {{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}
\end{array}\]且又因為我們要求  ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal故
\[\begin{array}{l}
{{\bf{v}}_3},{{\bf{v}}_1}\rangle  = 0\\
 \Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_1}\rangle  = 0\\
 \Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle  + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle  + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle  = 0\\
\\
\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle  = 0\\
 \Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_2}\rangle  = 0\\
 \Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle  + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle  + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle  = 0
\end{array}\]也就是說我們有一組聯立方程
\[\begin{array}{l}
\left\{ {\begin{array}{*{20}{l}}
{\langle {{\bf{v}}_3},{{\bf{v}}_1}\rangle  = 0}\\
{\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle  = 0}
\end{array}} \right.\\
 \Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle  + {b_2}\underbrace {\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle }_{ = 0} + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle  = 0}\\
{{b_1}\underbrace {\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle }_{ = 0} + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle  + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle  = 0}
\end{array}} \right.\\
 \Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle  + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle  = 0}\\
{{b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle  + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle  = 0}
\end{array}} \right.
\end{array}\]注意到 ${\bf v}_2 \neq {\bf 0}$ 因為 ${\bf v}_2$ 需與 ${\bf v}_1$ 正交,故兩條方程三個未知數 $b_1,b_2,b_3$,可指定一自由變數,故選 $b_3 :=1 \in \mathbb{R}^1$ 則我們可解得 $b_1, b_2$ 如下
\[\left\{ {\begin{array}{*{20}{l}}
{{b_1} =  - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}}\\
{{b_2} =  - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}}
\end{array}} \right.\]
也就是說
\[\begin{array}{l}
{{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}\\
 \Rightarrow {{\bf{v}}_3} =  - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2} + {{\bf{u}}_3}\\
 \Rightarrow {{\bf{v}}_3} = {{\bf{u}}_3} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2}
\end{array}
\]至此我們有了一組 orthogonal subset $\{{\bf v}_1,{\bf v}_2,{\bf v}_3\} $ for $W$

重複上述步驟 (by induction)我們可建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2,...,{\bf v}_m\}$ 最後我們對每一組 ${\bf v}_i$ 做正規化,定義
\[
{\bf w}_i := \frac{1}{||{\bf v}_i||} {\bf v}_i
\]則我們得到一組 orthonormal basis $T := \{{\bf w}_1,...,{\bf w}_m\}$

讀者可用以下幾個例子做練習

Example 1: 令 $S=\{[1\;\;2]^T, [-3\;\;4]^T\}$ 為 ordered basis for $ V:= \mathbb{R}^2$ 且其上內積為標準內積。
(a) 試利用 Gram-Schmidt process 找出 orthogonal basis
(b) 試利用 Gram-Schmidt process 找出 orthonormal basis

Example 2: 令 $V := P_3$ 且其上的內積定義為
\[
\langle p(t), q(t) \rangle := \int_0^1 p(t) q(t) dt
\] 現在令 $W$ 為 $P_3$ 子空間且基底為 $\{t,t^2\}$ 試求 orthonormal basis for $W$

[線性代數] Orthonormal Basis 與 Gram-Schmidt Process (0)

首先引入 一組向量彼此互為標準正交的定義

===================
Definition: Orthonormal Set
令 $V$ 為有限維度的內積空間 且 令 $S$ 為 $V$ 上的一組 集合滿足 $S =\{{\bf v}_1,..{\bf v}_n\}$ 。則我們稱 $S$ 為 orthonormal set 若
\[\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle  = \left\{ \begin{array}{l}
0\begin{array}{*{20}{c}}
{}&{}
\end{array}i \ne j\\
1\begin{array}{*{20}{c}}
{}&{}
\end{array}i = j
\end{array} \right.\]其中 $\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle $ 為 $V$ 上的內積運算。
====================

Comment:
1. 給定一個向量空間我們如果有 orthonormal basis 則其上的任意向量將可以被非常容易地表示 (why?) 比如說我們考慮 $V:= \mathbb{R}^2$ 且 具備一組標準基底 $S:=\{{\bf s}_1, {\bf s}_2\} = \{[1 \;0]^T, [0\;1]^T\}$ 則此基底為 orthonormal 。現在若給訂任意向量 ${\bf v} := [100, -99]^T\in V$ 則此向量可以非常容易透過 基底 $S$ 做線性組合來組出 ${\bf v}$亦即
\[\underbrace {\left[ \begin{array}{l}
100\\
 - 99
\end{array} \right]}_{ = {\bf{v}}} = 100\underbrace {\left[ \begin{array}{l}
1\\
0
\end{array} \right]}_{ = {{\bf{s}}_1}} + \left( { - 99} \right)\underbrace {\left[ \begin{array}{l}
0\\
1
\end{array} \right]}_{ = {{\bf{s}}_2}}\]
2. 上述觀點事實上到無窮維仍然成立,也就是說我們可以將正交的概念推廣到函數空間上面,並且說明什麼叫做兩個"函數" 彼此正交。以下我們看個無窮維函數空間的例子:


Example (Infinite-dimension Case)
令 $V:= C[0, 2 \pi]$ 且配備內積 \[
(f(t),g(t)) := \frac{1}{2 \pi}\int_0^{2 \pi} f(t) \bar{g} (t) dt
\]則 下列集合
\[
S :=\{f_n(t): f_n(t) := e^{jnt} =\cos nt + j \sin nt, \; n \in \mathbb{Z}\}
\]為 orthonormal set
Proof:
取 $f_n(t), g_m(t) \in S$ 觀察
\[\begin{array}{l}
({f_n}(t),{g_m}(t)) = \frac{1}{{2\pi }}\int_0^{2\pi } {{f_n}} (t){{\bar g}_m}(t)dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{jnt}}} {e^{ - jmt}}dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{j\left( {n - m} \right)t}}} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\left[ {\cos \left( {\left( {n - m} \right)t} \right) + j\sin \left( {\left( {n - m} \right)t} \right)} \right]} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\cos \left( {\left( {n - m} \right)t} \right)} dt + \frac{j}{{2\pi }}\int_0^{2\pi } {\sin \left( {\left( {n - m} \right)t} \right)} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{}
\end{array}n = m\\
0,\begin{array}{*{20}{c}}
{}&{}
\end{array}n \ne m
\end{array} \right. \;\;\;\;\;\;\;\;\; \square
\end{array}\]

=====================
Definition: Orthonormal Basis
若 $S$ 為 內積空間$V$上的一組有序基底,且 $S$ 為 orthnormal set 則我們稱此 $S$ 為 Orthnormal basis。
=====================


以下定理給出了 orthonormal basis 可以快速決定任意向量用該基底做線性組合的係數。

====================
Theorem: 
令 $S = \{{\bf u}_1, {\bf u}_2,...,{\bf u}_n\}$ 為一組 orthonormal basis 對有限維度向量空間 $V$ 且令 ${\bf v} \in V$ 則
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]其中 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$
====================

Comment:
上述定理中提及的 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle$ 在幾何意義上為 ${\bf v}$ 在 ${\bf u}_i$ 上的分量,且 $c_i$ 在數學上又稱為 Fourier Coefficient


以下我們給出證明:

Proof:
由於 ${\bf v} \in V$故此向量 ${\bf v}$ 可透過 $V$ 上的基底作唯一線性組合表示。
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]
故我們只需證明 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$

現在固定任意 $i$ ,並且觀察
\[\begin{array}{l}
\left\langle {{\bf{v}},{{\bf{u}}_i}} \right\rangle  = \left\langle {{c_1}{{\bf{u}}_1} + {c_2}{{\bf{u}}_2} + ... + {c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left\langle {{c_1}{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle  + \left\langle {{c_2}{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle  + ... + \left\langle {{c_i}{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle  + ... + \left\langle {{c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle  + {c_2}\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle  + ... + {c_i}\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle  + ... + {c_n}\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\underbrace {\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle }_{ = 0} + {c_2}\underbrace {\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle }_{ = 0} + ... + {c_i}\underbrace {\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle }_{ = 1} + ... + {c_n}\underbrace {\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle }_{ = 0}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_i}
\end{array}
\]注意到上述結果使用了 $S$ 基底為 orthonormal 故當 $i\neq j$ 時候, $\langle {\bf u_i}, {\bf u}_j\rangle =0$  $\square$

以下我們看個例子
Example 1:
令 $S = \{{\bf u}_1, {\bf u}_2\}$ 為 $\mathbb{R}^2$ 的一組基底,其中
\[{{\bf{u}}_1} = \frac{1}{\sqrt{2} }\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right];{{\bf{u}}_1} = \frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]\]
(a) 確認 $S$ 為一組 orthonormal basis
(b) 令 ${\bf v} := [3\;\;4]^T$ 試決定其透過 $S$ 基底所構成的線性組合

Proof:
(a) 注意到 $\mathbb{R}^2$ 為內積空間,我們可在其上定義內積運算為
\[
\langle {\bf u}, {\bf v} \rangle := {\bf u}^T {\bf v}
\]
 現在我們檢驗內積 $\langle {\bf u}_1, {\bf u}_2 \rangle$
\[\langle {{\bf{u}}_1},{{\bf{u}}_2}\rangle  = \frac{1}{2} \left[ {\begin{array}{*{20}{c}}
1&1
\end{array}} \right] \left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right] = 0\]故此說明了 ${\bf u}_1, {\bf u}_2$ 為 orthogonal 接著我們驗證此基底具有 unit length
\[\left\| {{{\bf{u}}_1}} \right\| = \sqrt {\langle {{\bf{u}}_1},{{\bf{u}}_1}\rangle }  = 1;\begin{array}{*{20}{c}}
{}&{}
\end{array}\left\| {{{\bf{u}}_2}} \right\| = \sqrt {\langle {{\bf{u}}_2},{{\bf{u}}_2}\rangle }  = 1\]
綜上所述, $S$ 為 orthonormal basis。

(b) 現在令 ${\bf v}:= [3\;\;4]^T \in \mathbb{R}^2$ 故此向量可透過 $S$ 基底做線性組合表示
\[
{\bf v} = c_1 {\bf u}_1 + c_2 {\bf u}_2
\] 又因為 $S$ 為 orthonormal basis 故由前述定理可知 上式中的係數可透過內積求得
\[\begin{array}{l}
{c_1} = \langle {\bf{v}},{{\bf{u}}_1}\rangle  = {{\bf{v}}^T}{{\bf{u}}_1} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right]} \right) = \frac{7}{\sqrt{2}}\\
{c_2} = \langle {\bf{v}},{{\bf{u}}_2}\rangle  = {{\bf{v}}^T}{{\bf{u}}_2} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]} \right) = \frac{1}{\sqrt{2}}\;\;\;\;\; \square
\end{array}\]

現在我們可以考慮以下問題:
若給定一個有限維度向量空間 $V$ 伴隨一組基底 $S$。那麼我們想進一步詢問是否可從這組基底 $S$ 中找出另外一組基底 $T$ 且 $T$ 基底元素彼此互相正交 且 單位長度為 $1$ ? 亦即我們想問是否可以從一組不是 orthonormal basis $S$ 來建構一組 orthonormal basis $T$ ,答案是肯定的,此構造方法稱為 Gram-Schmidt Process 我們之後會再行介紹。

[數學分析] 三角多項式 與 三角級數 (2)- Generalized Fourier Series

現在我們定義廣義 Fourier Series :

=================
Definition: (Orthogonal System of Functions)
令 $\{\phi_n\}$, $n \in \mathbb{N}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence 且滿足下列積分
\[
\int_a^b \phi_n(x) \phi_m^*(x) dx =0, \;\; \text{ if $n \neq m$}
\]那麼我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthogonal 或稱 (orthogonal system of functions on $[a,b]$) 。除此之外,若積分
\[
\int_a^b \phi_n(x) \phi_n^*(x) dx =1
\]我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthonormal 或稱 (orthonormal system of functions on $[a,b]$) 。
=====================

Comments: 
一般而言,若我們取 $\{\phi_n\}$  $n \in \mathbb{N}$ 為在 $[0, 2\pi]$ 上 的 Complex-valued 函數 sequence 且滿足 $\phi_n(x):= exp(inx)$  則讀者可自行驗證此 函數 sequence 為 orthogonal


=====================
Definition: (n-th Fourier Coefficient of $f$)
若 $\{\phi_n\}$ 為 orthonormal on $[a,b]$ 且 對任意 $n \in \mathbb{N}$,
\[
c_n:=\int_a^b f(x) \phi_n^*(x) dx
\]我們稱 $c_n$ 為 $n$-th Fourier coefficient of $f$ (relative to $\{\phi_n\}$)
=====================
上述 $^*$ 為 complex conjugate。


=====================
Definition: Generalized Fourier Series
Generalized Fourier Series of $f$ (relative to $\{\phi_n\}$) 定義為
\[
f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)
\]其中 $c_n=\int_a^b f(x) \phi_n^*(x) dx $。
====================

Theorem: Bessel's inequality
若 $\{\phi_n\}$ 為 orthnormal on $[a,b]$ 且若 $f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)$ 則 \[
\sum_{n=1}^\infty |c_n|^2 \le \int_a^b |f(x)|^2 dx
\]


====================
Theorem: Best Approximation of Fourier Series 
令 $\{\phi_n\}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence ,且 $\phi_n$ 為 orthogonal。現在定義 $n$-th partial sum of Fourier Series of $f$ 如下
\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x)
\]且令 $t_n$ 為任意 series 如下
\[
t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]其中 $\gamma_n \in \mathbb{C}$ 則
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 且 上式等式成立 若且為若 $\gamma_n = c_n$ 對任意 $n$。
==================

Proof:

\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x);\;\;\; t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]
我們首先證明下列不等式成立
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 首先觀察
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {t_n}(x)} \right){{\left( {f(x) - {t_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){t_n}^*(x) - {t_n}(x){f^*}(x) + {t_n}(x){t_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4} \ \ \ \ \ \ (*)
\end{array}\]接著對上式逐項觀察,首先檢驗 term 2:
\[\begin{array}{l}
\int_a^b {f(x){t_n}^*(x)} dx = {\int_a^b {f(x)\left[ {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x)\sum\limits_{m = 1}^n {{\gamma _m}^*} {\phi _m}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} \underbrace {\int_a^b {f(x){\phi _m}^*(x)} dx}_{ = {c_m}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} {c_m}
\end{array}\]
接著我們檢驗 term 3:
\[\begin{array}{l}
\int_a^b {{t_n}(x){f^*}(x)} dx = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x){f^*}(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \underbrace {\int_a^b {{f^*}(x){\phi _m}(x)} dx}_{ = c_m^*}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} c_m^*
\end{array}\]
最後檢驗 term 4:
\[\begin{array}{l}
\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx = \int_a^b {{t_n}(x)t_n^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = {\int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\left[ {\sum\limits_{k = 1}^n {{\gamma _k}} {\phi _k}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\sum\limits_{k = 1}^n {{\gamma _k}^*} {\phi _k}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \sum\limits_{k = 1}^n {{\gamma _k}^*} \underbrace {\int_a^b {{\phi _m}(x){\phi _k}^*(x)} dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}m = k}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m= 1}^n {{\gamma _m}{\gamma _m}^*}  = \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]
故現在將上述結果 帶回 $(*)$ 可得
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}}  - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*}  + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}  \ \ \ \ (\star)
\end{array}\]
注意到下列 FACT:
\[\begin{array}{l}
\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}}  = \sum\limits_{m = 1}^n {\left( {{c_n} - {\gamma _m}} \right){{\left( {{c_n} - {\gamma _m}} \right)}^*}}  = \sum\limits_{m = 1}^n {\left( {{c_n}{c_n}^* - {c_n}{\gamma _m}^* - {\gamma _m}{c_n}^* + {\gamma _m}{\gamma _m}^*} \right)} \\
 \Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}}  = \sum\limits_{m = 1}^n {{c_n}{c_n}^*}  - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*}  - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*}  + \sum\limits_{m = 1}^n {{\gamma _m}{\gamma _m}^*} \\
 \Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}}  - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}}  =  - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*}  - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*}  + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]與
\[\begin{array}{l}
\int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {s_n}(x)} \right){{\left( {f(x) - {s_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){s_n}^*(x) - {s_n}(x){f^*}(x) + {s_n}(x){s_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x){f^*}(x)} dx - \int_a^b {f(x){s_n}^*(x)} dx - \int_a^b {{s_n}(x){f^*}(x)} dx + \int_a^b {{s_n}(x){s_n}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \int_a^b {\left( {f(x)\sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} } \right)} dx - \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} {f^*}(x)} \right)} dx + \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} \sum\limits_{k = 1}^n {{c_k}^*{\phi _k}^*\left( x \right)} } \right)} dx\\
{\rm{since}}\begin{array}{*{20}{c}}
{}
\end{array}{s_n}(x): = \sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)}  \Rightarrow s_n^*(x): = {\left[ {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} } \right]^*} = \sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} \int_a^b {\left( {f(x){\phi _m}^*\left( x \right)} \right)} dx - \sum\limits_{m = 1}^n {{c_m}} \int_a^b {{f^*}(x){\phi _m}\left( x \right)} dx + \sum\limits_{m = 1}^n {{c_m}} \sum\limits_{k = 1}^n {{c_k}^*} \int_a^b {{\phi _m}\left( x \right){\phi _k}^*\left( x \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m} - \sum\limits_{m = 1}^n {{c_m}} c_m^* + \sum\limits_{m = 1}^n {{c_m}{c_m}^*} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m}\\
 \Rightarrow \int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_m}} \right|}^2}}
\end{array}\]

故 $\star$ 可進一步改寫
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}}  - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*}  + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}}  - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} }_{ = \int_a^b | f(x) - {s_n}(x){|^2}dx} + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b | f(x) - {s_n}(x){|^2}dx + \underbrace {\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} }_{ \ge 0}\\
 \Rightarrow \int_a^b | f(x) - {t_n}(x){|^2}dx \ge \int_a^b | f(x) - {s_n}(x){|^2}dx\\

\end{array}\]
且若 $c_m = \gamma_m$ 則等式成立。 $\square$

[數學分析] 三角多項式 與 三角級數 (1)

三角多項式表示一個函數可以透過 多個三角函數 方式表示,具體定義如下。

============================
Definition: Trigonometric polynomial
我們說 $f(x)$ 為一個 三角多項式( trigonometric polynomial) 若 $f$ 具有下列形式:
\[
f(x) := \sum_{n=0}^N a_n \cos(nx) + b_n \sin (nx) \ \ \ \ \ (*)
\]其中 $a_n, b_n \in \mathbb{C}$ 且 $x \in \mathbb{R}$。;或者上式可等價寫為 複數形式
\[
f(x) := \sum_{n=-N}^N c_n e^{i n x}
\]對任意 $c_n \in \mathbb{C}$ 與 $x \in \mathbb{R}$
============================

Comment:
注意到對於 式子 $(*)$ 可改寫為
\[f(x) = \sum\limits_{n = 0}^N {{a_n}} \cos (nx) + {b_n}\sin (nx) = {a_0} + \sum\limits_{n = 1}^N {{a_n}} \cos (nx) + {b_n}\sin (nx)\]

==========================
FACT 1: Trigonometric polynomial $f$ 為週期函數且週期為 $2 \pi$。
==========================

Proof: 亦即我們要證明 $f(x+2 \pi) = f(x)$,故
\[\begin{array}{l}
f(x + 2\pi ): = \sum\limits_{n =  - N}^N {{c_n}} {e^{in\left( {x + 2\pi } \right)}} = \sum\limits_{n =  - N}^N {{c_n}} {e^{in\left( x \right)}}{e^{in\left( {2\pi } \right)}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \sum\limits_{n =  - N}^N {{c_n}} {e^{in\left( x \right)}}\underbrace {\left[ {\cos n2\pi  + i\sin n2\pi } \right]}_{ = 1} = \sum\limits_{n =  - N}^N {{c_n}} {e^{in\left( x \right)}} = f\left( x \right) \ \ \ \ \square
\end{array}\]

==========================
FACT 2: 下列等式成立:
\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi  {{e^{imx}}} {e^{ - inx}}dx = \left\{ \begin{array}{l}
0,\begin{array}{*{20}{c}}
{}
\end{array}n \ne m\\
1,\begin{array}{*{20}{c}}
{}
\end{array}n = m
\end{array} \right.\]==========================
Proof: omitted.

==========================
FACT 3: Trigonometric polynomial $f$ 的係數 $c_n$ 可由下列積分決定:
\[
c_n = \frac{1}{2 \pi}\int_{-\pi}^\pi f(x) e^{-inx}dx
\]==========================
Proof:
\[\begin{array}{*{20}{l}}
{\frac{1}{{2\pi }}\int_{ - \pi }^\pi  f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi  {\sum\limits_{m =  - N}^N {{c_m}} {e^{imx}}} {e^{ - inx}}dx}\\
{\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m =  - N}^N {{c_m}} \int_{ - \pi }^\pi  {{e^{imx}}} {e^{ - inx}}dx}
\end{array}\]利用 FACT 2 可得
\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi  f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\sum\limits_{m =  - N}^N {{c_m}} \underbrace {\int_{ - \pi }^\pi  {{e^{imx}}} {e^{ - inx}}dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}n = m} = {c_n} \ \ \ \ \square\]

===================
FACT 4: Trigonometric polynomial $f$ 為 實數函數 若且唯若 $c_n^* = c_{-n}$ ( 其中$( \cdot )^*$) 表示 complex conjugate。
===================

Proof:
$(\Rightarrow)$ 假設 Trigonometric polynomial $f$ 為 Real-valued 函數,我們要證明 $c_n^* = c_{-n}$ 。故由於 $f$ 為 Real-valued 函數,我們有 $f^* = f$;亦即
\[\begin{array}{l}
c_n^* = {\left( {\frac{1}{{2\pi }}\int_{ - \pi }^\pi  f (x){e^{ - inx}}dx} \right)^*} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi  {{f^*}} (x){e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi  f (x){e^{inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi  {\sum\limits_{m =  - N}^N {{c_m}} {e^{imx}}} {e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m =  - N}^N {{c_m}} \int_{ - \pi }^\pi  {{e^{imx}}} {e^{inx}}dx = {c_{ - n}}.
\end{array}\]

$(\Leftarrow)$ 假設  $c_n^* = c_{-n}$ ,我們要證 Trigonometric polynomial $f$ 為 Real-valued 函數。亦即要證明 $f^* = f$。現在觀察
\[{f^*}(x) = {\left( {\sum\limits_{n =  - N}^N {{c_n}} {e^{inx}}} \right)^*} = \sum\limits_{n =  - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{n =  - N}^N {c_{ - n}^{}} {e^{ - inx}}\]令 $m:=-n$ 可得
\[{f^*}(x) = \sum\limits_{n =  - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{m =  - N}^N {c_m^{}} {e^{imx}} = f\left( x \right) .\ \ \ \ \square
\]

接著我們可以定義 Trigonometric Series 如下:

========================
Definition: Trigonometric Series 
我們說 Trigonometric Series 為具有下列形式的無窮級數
\[
\sum_{n=-\infty}^\infty c_n e^{-inx}
\]且由先前 Trigonometric polynomial 的係數定法,我們可以一樣定義對一個週期函數 $f$ 的 $m$-th Fourier Coefficient :
\[
c_m := \frac{1}{2 \pi} \int_{-\pi}^\pi f(x) e^{-imx}dx
\]========================

========================
Definition: Fourier Series 
Fourier Series 為一個 Trigonometric Series 且其係數為 Fourier coefficient of $f$,我們將 Fourier Series 記做
\[
f \sim \sum_{n = -\infty}^\infty c_n e^{inx}
\]========================
注意:上述並非等號;單純表示 $c_n$ 是來自 $f$ 的 Fourier Series coefficient。

故我們想問 "何時可以讓 $f$ 與 Fourier Series 等號成立? " 或者說 基於怎樣的測量基準之下,此兩者可以被適當的逼近?

我們將回答此問題於更廣義的 Fourier Series 之上,在後面的文章會在做介紹。


[數學分析] 內積空間的不等式 Cauchy-Schwarz Inequality 與 Triangular Inequality

==============================
Theorem: (Cauchy-Schwarz Inequality) 
令 $V$ 為實數內積空間,且  ${\bf u}, {\bf v} \in V$ 則\[
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||
\]==============================


先看幾個例子
Example 1: 歐幾里德平面空間對應的 柯西不等式 
$V:=\mathbb{R}^2$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$,現令 ${\bf u}:=[u_1\;\;u_2]^T; \; {\bf v}:=[v_1\;\;v_2]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|\\
 \Rightarrow {\left| {{{\bf{u}}^T}{\bf{v}}} \right|^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)\\
 \Rightarrow {\left| {{u_1}{v_1} + {u_2}{v_2}} \right|^2} \le \left( {u_1^2 + u_2^2} \right)\left( {v_1^2 + v_2^2} \right)
\end{array}\]

Example 2: 有限維歐幾里德空間對應的 柯西不等式
$V:=\mathbb{R}^n$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$,現令 ${\bf u}:=[u_1,...,\;\;u_n]^T; \; {\bf v}:=[v_1,...,\;\;v_n]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {{{\bf{u}}^T}{\bf{v}}} \right|}^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)}\\
{ \Rightarrow {{\left| {{u_1}{v_1} + {u_2}{v_2} + ... + {u_n}{v_n}} \right|}^2} \le \left( {u_1^2 + u_2^2 + ... + u_n^2} \right)\left( {v_1^2 + v_2^2 + ... + v_n^n} \right)}
\end{array}\]

Example 3: 無窮維 實數連續函數空間 對應的 柯西不等式
$V:=C[0,1]$ 且配備內積 $(f(t), g(t)) := \int_0^1 f(t) g(t) dt$,現令 ${\bf u}:=f(t); \; {\bf v}:=g(t)$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {f\left( t \right),g\left( t \right)} \right)} \right| \le \left\| {f\left( t \right)} \right\|\left\| {g\left( t \right)} \right\|\\
 \Rightarrow {\left| {\int_0^1 {f\left( t \right)g\left( t \right)dt} } \right|^2} \le \left( {\int_0^1 {{f^2}\left( t \right)dt} } \right)\left( {\int_0^1 {{g^2}\left( t \right)dt} } \right)
\end{array}\]

Example 4: 實數矩陣空間對應的柯西不等式
令 $V:= M_{n \times n}$ 且配備內積 $(A, B) := tr(B^T A)$ 現令 ${\bf u}:=A; \; {\bf v}:=B$ 為 $n \times n$ 矩陣,則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {tr\left( {{B^T}A} \right)} \right|}^2} \le tr\left( {{A^T}A} \right)tr\left( {{B^T}B} \right)}
\end{array}\]

Example 5: 隨機變數所成的 $L^2$ 空間之柯西不等式:
令 $V:= L^p :=\{X: E[|X|^2] < \infty\}$ 且配備內積 $(X,Y) := E[XY]$,其中 $X,Y$ 為隨機變數,$E[\cdot]$ 表期望值。現令 ${\bf u} := X$ 且 ${\bf v} := Y$ 則上述的 Cauchy-Schwarz Inequality 可表為
\begin{align*}
  & \left| {\left( {{\mathbf{u}},{\mathbf{v}}} \right)} \right| \leq \left\| {\mathbf{u}} \right\|\left\| {\mathbf{v}} \right\| \hfill \\
 &  \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \left\| X \right\|\left\| Y \right\| \hfill \\
  & \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \sqrt {E\left[ {{X^2}} \right]} \sqrt {E\left[ {{Y^2}} \right]}  \hfill \\
\end{align*}



Comments:
1.上述幾個例子展示了儘管所表現的樣式非常不同,但從抽象化觀點而言是同一件事情。
2. 柯西等式何時成立?

以下我們給出 Cauchy-Schwarz Inequality 的證明,此證明頗具巧思有興趣的讀者可細細品味。

Proof of Cauchy-Schwarz Inequality
令 $c \in \mathbb{R}^1$ 現在觀察
\[\begin{array}{l}
\underbrace {\left( {{\bf{u}} - c{\bf{v}},{\bf{u}} - c{\bf{v}}} \right)}_{ = {{\left\| {{\bf{u}} - c{\bf{v}}} \right\|}^2}} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( { - c{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}}, - c{\bf{v}}} \right) + \left( { - c{\bf{v}}, - c{\bf{v}}} \right)\\
 = \left( {{\bf{u}},{\bf{u}}} \right) - c\left( {{\bf{v}},{\bf{u}}} \right) - c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
 = \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right) \;\;\;\; (*)
\end{array}\]
若 ${\bf{v}} \ne 0$ 則 $({\bf v},{\bf v})>0$ 我們可取 \[c: = \frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\]將此 $c$ 帶入 $(*)$ 可得
\[\begin{array}{l}
{\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} = \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
 = \left( {{\bf{u}},{\bf{u}}} \right) - 2\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\left( {{\bf{u}},{\bf{v}}} \right) + {\left( {\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}} \right)^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
 = \left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}
\end{array}\]但由於 ${\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} \ge 0$ 故
\[\left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}} \ge 0 \Rightarrow \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}\]
上式結果說明在 ${\bf v} \neq 0$ 時 Cauchy-Schwarz Inequality 成立。另外我們回頭檢驗 ${\bf v} = 0$ 的情況,則此時 $\left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}$ 自動滿足。故不論如何我們都有
\[{\left( {{\bf{u}},{\bf{v}}} \right)^2} \le \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right)\]至此證畢。$\square$

Comments:
1. 注意到若 ${\bf u} = c {\bf v} $ 則 Cauchy-Schwarz 等式成立。此結果背後蘊含最小平方的最佳化觀點但我們在此不作贅述。
2. 上述 Cauchy-Schwarz Inequality 可引出 Triangular Inequality

Corollary:  Triangular Inequality
令 $V$ 為實數內積空間,若 ${\bf u}, {\bf v} \in V $ 則
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]
Proof:
觀察
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} = \left( {{\bf{u}} + {\bf{v}},{\bf{u}} + {\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + 2\left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {\left\| {\bf{u}} \right\|^2} + 2\left( {{\bf{v}},{\bf{u}}} \right) + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}
\end{array}\]由 Cauchy-Schwarz Inequality 我們有 $
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||$ 故
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2||{\bf{u}}||\;||{\bf{v}}|| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left( {\left\| {\bf{u}} \right\| + \left\| {\bf{v}} \right\|} \right)^2}
\end{array}\]對兩邊同開根號我們得到
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]即為所求 $\square$

Comment:
若 ${\bf u}, {\bf v}$ 互為正交,亦即 $({\bf u},{\bf v}) = 0$ 則我們有
\[
||{\bf u} + {\bf v}||^2 = ||{\bf u}||^2 + ||{\bf v}||^2
\]上述等式 可視為 畢氏定理 在向量空間中的推廣。

2015年11月19日 星期四

[機率論] 非負連續隨機變數 的期望值

令 $Y $ 為 任意非負 連續隨機變數 配備機率密度 $f_Y$,則我們有以下非常簡潔的結果來描述 $Y$ 的期望值 $E[Y]$。

============
Lemma:
\[
E[Y] = \int_0^\infty P(Y>y) dy
\]============
Proof:
首先觀察等式右方,由於 $P\left( {Y > y} \right) = \int_y^\infty  {{f_Y}\left( x \right)dx} $ 故
\[\begin{array}{l}
\int_0^\infty  {P\left( {Y > y} \right)dy}  = \int_0^\infty  {\left( {\int_y^\infty  {{f_Y}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {\left( {\int_0^\infty  {{f_Y}\left( x \right){1_{\left\{ {x \ge y} \right\}}}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {\left( {\int_0^\infty  {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dx} } \right)dy}
\end{array}\]由於 integrand 非負,由 Fubini Theorem 我們可互換積分順序並得到如下結果
\[\begin{array}{l}
\int_0^\infty  {P\left( {Y > y} \right)dy}  = \int_0^\infty  {\int_0^\infty  {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dydx} } \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {{f_Y}\left( x \right)\left( {\int_0^\infty  {{1_{\left\{ {y \le x} \right\}}}\left( y \right)dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {{f_Y}\left( x \right)\left( {\int_0^x {1dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {{f_Y}\left( x \right)x} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty  {x{f_Y}\left( x \right)} dx = E[Y] \;\;\;\;\;\;\; \square
\end{array}\]

Comments:
1. 前述假設 非負隨機變數是指 $Y \ge 0$ almost surely, 亦即 $ P(Y \ge 0) = 1$
2. 上述證明中採用的符號 $1_{A} (x)$ 表示 指示函數 (indicator function),我們給出定義如下:令 $X$ 為任意集合 則我們可定義對其上的任意子集 $A  \subset X$ 所對應的指示函數( indicator function of a subset $A$ of a set $X$ ) 為 $1_A: X \to \{0,1\}$ 滿足 \[{1_A}\left( x \right): = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{x \in A}
\end{array}\\
0,\begin{array}{*{20}{c}}
{}&{x \notin A}
\end{array}
\end{array} \right.\]
3. 上述結果可用 distribution function 改寫,令 $F_Y(y) := P(Y \leq y)$則
\[
E[Y] = \int_0^\infty P(Y>y) dy = \int_0^\infty (1 - F_Y(y)) dy
\]這個結果可以使得我們在計算期望值的同時,不用再困擾需要先求出 pdf ,只要有分配函數 即可計算期望值。

2015年11月14日 星期六

[線性代數] 淺論有限維 實數內積空間 (0)

這次要介紹 有限維度的內積空間 (Inner Product Space),簡而言之就是有限維度向量空間 $(V, \oplus, \odot)$ 上額外定義內積運算,則我們稱此類空間為 內積空間。

Comments: 
1. 有限維度內積空間稱為 歐式空間 (Euclidean Space)
2. 若為無窮維度的內積空間我們稱為 Pre-Hilbert Space,若此無窮維度內積空間為完備空間,則稱之為 Hilbert Space
3. 為何好好的向量空間不用還要多此一舉另外又定一個 內積空間?主因是向量空間本身只定義了加法 與純量乘法的運算,如果我們想討論在向量空間中某元素的大小 或者 某兩元素之間的關係則無從得知。但是如果我們引入 內積運算 到向量空間中,則可以在原本的向量空間上將 代數 與 幾何 的概念做直接的連結,也就是我們可以透過內積引入 其上的兩元素是否 垂直 (正交) 的概念,亦可針對某元素來探討其 長度與大小 概念 。
4. 讀者可回憶 高中所學習過的 點積 (dot product),此文所探討的內積 即為 點積 的推廣。



首先定義內積

==================
Definition: Inner Product on Vector Space
令 $V$ 為實數向量空間,則 Inner Product on $V$ 為函數 $(\cdot, \cdot): V\times V \to \mathbb{R}$ 滿足下列條件
(a) $({\bf u}, {\bf u}) \ge 0$:且 $({\bf u}, {\bf u}) = 0$ 若且唯若 ${\bf u} = {\bf 0}_V$
(b) $({\bf u}, {\bf v}) = ({\bf v}, {\bf u}), \; \forall {\bf u,v} \in V$
(c) $({\bf u} + {\bf v}, {\bf w}) = ({\bf u}, {\bf w}) + ({\bf u}, {\bf v}), \; \forall {\bf u,v,w} \in V$
(d) $(c {\bf u}, {\bf v}) = c({\bf u}, {\bf v}),\; \forall {\bf u,v} \in V, c \in \mathbb{R}$
==================

Comment:
1. 透過內積我們亦可定義 ${\bf u}$ 的大小,記作 $||{\bf u}|| = \sqrt{ ({\bf u}, {\bf u} )} $
2. 透過內積我們亦可定義在內積空間中兩向量是否垂直:亦即 若 ${\bf u}, {\bf v} \in V$ 稱為 垂直 或 正交 若 $({\bf u}, {\bf v}) = 0$

2. 前述內積定義可立即有以下衍生結果:
Fact 1: $\left( {{\bf{u}},c{\bf{v}}} \right) = c\left( {{\bf{u}},{\bf{v}}} \right)$

Fact 2: $\left( {{\bf{u}},{\bf{v}} + {\bf{w}}} \right) = \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{u}},{\bf{w}}} \right)$

讀者可自行驗證上述兩個結果。

=========
Claim 1:  給定 ${\bf v}, {\bf w} \in V$ ,若對任意 ${\bf u} \in V$ 我們有 $
({\bf u},{\bf v}) = ({\bf u}, {\bf w})$ 則 ${\bf v} = {\bf w}$
=========
Proof:
由於我們要證明 ${\bf v} = {\bf w}$,故我們僅需證明 $\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = 0$ 現在觀察兩者之差的內積
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) + \left( {{\bf{v}} - {\bf{w}}, - {\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)
\end{array}\]注意到 ${\bf v} - {\bf w} \in V$ 故 若我們令 ${\bf u}:= {\bf v} - {\bf w} $ 則 由已知條件可知
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{v}}} \right) - \left( {{\bf{u}},{\bf{w}}} \right) = 0. \;\;\; \square
\end{array}\]


由上述結果我們有以下衍生定理
=========
Corollary of Claim 1:
若 對任意 ${\bf u} \in V$ 我們有 $({\bf u}, {\bf v})=0$ 則 ${\bf v} = 0$
=========
Proof: omitted.



=========
Claim: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量, 則此運算為一種內積。
=========
Proof:
我們現在驗證 $f$ 確實為內積,故我們需驗證其滿足前述四項 (a,b,c,d)條件:首先驗證 $(a)$:
\[f({\bf{u}},{\bf{u}}) = {{\bf{u}}^T}{\bf{u}} = u_1^2 + ...u_2^2 \ge 0\]且 $f({\bf{u}},{\bf{u}}) = 0$ 若且唯若 ${\bf u} = [0,...,0]^T$

$(b)$ 令 ${\bf u,v} \in \mathbb{R}^n$ 現在我們觀察
\[\begin{array}{l}
f({\bf{v}},{\bf{u}}) = {{\bf{v}}^T}{\bf{u}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {v_1}{u_1} + ... + {v_n}{u_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {u_1}{v_1} + ... + {u_n}{v_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
 \vdots \\
{v_n}
\end{array} \right] = {{\bf{u}}^T}{\bf{v}} = f\left( {{\bf{u}},{\bf{v}}} \right)
\end{array}
\]
$(c)$ 令 $\bf u,v,w$$\in \mathbb{R}^n$ 接著我們觀察
\[\begin{array}{l}
f({\bf{u}} + {\bf{v}},{\bf{w}}) = {\left( {{\bf{u}} + {\bf{v}}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1} + {v_1}}&{{u_2} + {v_2}}&{...}&{{u_n} + {v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
 \vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1} + {v_1}} \right){w_1} + ...\left( {{u_n} + {v_n}} \right){w_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1}{w_1} + ... + {u_n}{w_n}} \right) + \left( {{v_1}{w_1} + ... + {v_n}{w_n}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
 \vdots \\
{w_n}
\end{array} \right] + \left[ {\begin{array}{*{20}{c}}
{{v_1}}&{{v_2}}&{...}&{{v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
 \vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {\left( {\bf{u}} \right)^T}{\bf{w}} + {\left( {\bf{v}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = f({\bf{u}},{\bf{w}}) + f({\bf{v}},{\bf{w}})
\end{array}
\]
$(d)$ 同理我們觀察
\[\begin{array}{l}
f(c{\bf{u}},{\bf{v}}) = {\left( {c{\bf{u}}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{c{u_1}}&{c{u_2}}&{...}&{c{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
 \vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
 \vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c{\left( {\bf{u}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = cf({\bf{u}},{\bf{v}})
\end{array}\]

Claim 2:  令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量, 則此運算為 $\mathbb{R}^n$ 一種內積。
Proof: omitted

Claim 3:  令向量空間 $ V := C[a,b]$ ,若 $f,g \in V$ 令
\[\left( {f,g} \right): = \int_a^b {f\left( t \right)g\left( t \right)dt} \] 則 $(f,g)$ 為 $C[a,b]$ 上的一種內積。
Proof: omitted


以下我們接著介紹對任意有限維度向量空間,則其上的內積可以用一個透過基底表示的矩陣 $C$ 來完全決定。

==================
Theorem: 令 $S=\{{\bf u}_1,...,{\bf u}_n\}$ 為 向量空間 $V$ 的 ordered basis 且假設我們可在 $V$ 上定義內積,現在令 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且 $C=[c_{ij}]$ 矩陣 則
對任意 ${\bf v,w} \in V$, 存在 $C = [c_{ij}]$ 矩陣 使得 $({\bf v},{\bf w}) = [{\bf v}]_S^T C [{\bf w}]_S$
==================

Proof : 給定  ${\bf v,w} \in V$ 則我們有
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n}\\
{\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}
\end{array}
\]
現在我們觀察內積 \[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
 = ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
 = {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})
\end{array}\]又因為 ${\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}$ 故
\[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
 = ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
 = {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})\\
 = {a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + {a_2}({{\bf{u}}_2},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + ... + {a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n})\\
 = \left[ {{a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1}) + {a_1}({{\bf{u}}_1},{b_2}{{\bf{u}}_2}) + ... + {a_1}({{\bf{u}}_1},{b_n}{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1}) + {a_n}({{\bf{u}}_n},{b_2}{{\bf{u}}_2})... + {a_n}({{\bf{u}}_n},{b_n}{{\bf{u}}_n})} \right]\\
 = \left[ {{a_1}{b_1}({{\bf{u}}_1},{{\bf{u}}_1}) + {a_1}{b_2}({{\bf{u}}_1},{{\bf{u}}_2}) + ... + {a_1}{b_n}({{\bf{u}}_1},{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}{b_1}({{\bf{u}}_n},{{\bf{u}}_1}) + {a_n}{b_2}({{\bf{u}}_n},{{\bf{u}}_2})... + {a_n}{b_n}({{\bf{u}}_n},{{\bf{u}}_n})} \right]\\
 = \sum\limits_{j = 1}^n {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j})}  + \sum\limits_{j = 1}^n {{a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j})}  + ... + \sum\limits_{j = 1}^n {{a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \\
 = \sum\limits_{j = 1}^n {\left( {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j}) + {a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j}) + ... + {a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \right)} \\
 = \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}({{\bf{u}}_i},{{\bf{u}}_j})} }
\end{array}\]
現在令 $c_{ij} = (a_i, b_j)$ ,則我們有
\[({\bf{v}},{\bf{w}}) = \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}{c_{ij}}} } =[{\bf v}]_S C [{\bf w}]_S\]

Comments: 
1. 前述的 $C$ 矩陣亦稱為 matrix of the inner produce with respect to the ordered basis $S$
2. 由於我們有 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且利用內積定義 $ ({\bf u}_i, {\bf u}_j)=({\bf u}_j, {\bf u}_i) $ 我們得到 $c_{ij} = c_{ji}$,故 前述的 $C = [c_{ij}]$ 矩陣為對稱矩陣。


上述 inner product 不但可用來引出 norm 的概念,更保持了連續性,以下我們給出相關結果。

============================
Lemma: (Continuity of Inner Product)
令 $u_n \to u$ 且 $v_n \to v$ 且收斂在某內積空間 (或 Pre-Hibert Space) ,則
\[
(u_n,v_n) \to (u,v)
\] ============================

Proof: 直接觀察
\begin{align*}
  \left| {({u_n},{v_n}) - (u,v)} \right| &= \left| {({u_n},{v_n}) - \left( {{u_n},v} \right) + \left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
  & \leqslant \left| {({u_n},{v_n}) - \left( {{u_n},v} \right)} \right| + \left| {\left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
   &= \left| {({u_n},{v_n} - v)} \right| + \left| {\left( {{u_n} - u,v} \right)} \right| \hfill \\
\end{align*} 回憶 Cauchy Schwarz Inequality $|(x,y)| \leq ||x|| ||y||$,我們有
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \left\| {{u_n}} \right\|\left\| {{v_n} - v} \right\| + \left\| {{u_n} - u} \right\|\left\| v \right\|\;\;\;\; (*)
\]注意到因為 $\{u_n\}$數列為收斂數列,故 $||u_n|| < \infty$,另外 $v$ 為 收斂數列 $\{v_n\}$ 的極限,故 $||v|| < \infty$。除此之外,由假設  $u_n \to u$ 且 $v_n \to v$ 可知
\[
||u_n - u|| \to 0;\;\;\;\;\; ||v_n \to v|| \to 0
\]因此,上式 $(*)$ 取極限後可得
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \underbrace {\left\| {{u_n}} \right\|}_{ < \infty }\underbrace {\left\| {{v_n} - v} \right\|}_{ \to 0} + \underbrace {\left\| {{u_n} - u} \right\|}_{ \to 0}\underbrace {\left\| v \right\|}_{ < \infty } \to 0 + 0 = 0\]

2015年11月1日 星期日

[MATLAB] 將 symbolic expression 轉成 latex 程式碼

一般而言在 MATLAB 使用中不免會碰到使用 symbolic toolbox 情況,但有時表示式非常繁雜,如果又想要把該表示方程式轉寫成 latex 貼到論文中該怎麼辦?

MATLAB 提供一個非常方便的功能

latex(.)

可以幫助我們直接轉換 MATLAB symbolic expression 成為 LATEX code