2. Review of Vectors and Matrices

 

 

VECTORS

 

1. Definition

 

For the purposes of this course, a vector is an object which has magnitude and direction.  Examples include forces, electric fields, and the normal to a surface.  A vector is often represented pictorially as an arrow and symbolically by an underlined letter $\underset{\_}{a}$ or using bold type $a$.  Its magnitude is denoted $\left|\underset{\_}{a}\right|$ or $\left|a\right|$.  There are two special cases of vectors: the unit vector $n$ has $\left|n\right|=1$; and the null vector $0$ has $\left|0\right|=0$.

 

2. Vector Operations

 

 Addition

 

Let $a$ and $b$ be vectors.  Then $c=a+b$ is also a vector.  The vector $c$ may be shown diagramatically by placing arrows representing $a$ and $b$ head to tail, as shown in the figure.

 

 Multiplication

 

1.      Multiplication by a scalar. Let $a$ be a vector, and $\alpha$ a scalar.  Then $b=\alpha a$ is a vector.  The direction of $b$ is parallel to $a$ and its magnitude is given by $\left|b\right|=\alpha \left|a\right|$.

Note that you can form a unit vector n which is parallel to a by setting $n=\frac{a}{\left|a\right|}$.

2.      Dot Product (also called the scalar product). Let a and b be two vectors.  The dot product of a and b is a scalar denoted by $\alpha =a\cdot b$, and is defined by

$a\cdot b=\left|a\right|\left|b\right|\cos \theta (a,b)$,

where $\theta (a,b)$ is the angle subtended by a and b. Note that $a\cdot b=b\cdot a$, and $a\cdot a={\left|a\right|}^2$.  If $\left|a\right|\ne 0$ and $\left|b\right|\ne 0$ then $a\cdot b=0$ if and only if $\cos \theta (a,b)=0$; i.e. a and b are perpendicular.

3.      Cross Product (also called the vector product).  Let a and b be two vectors.  The cross product of a and b is a vector denoted by $c=a\times b$.  The direction of c is perpendicular to a and b, and is chosen so that (a,b,c) form a right handed triad, Fig. 3.  The magnitude of c is given by

$\left|c\right|=\left|a\times b\right|=\left|a\right|\left|b\right|\sin \theta (a,b)$

Note that $a\times b=-b\times a$ and $a\cdot (a\times b)=b\cdot (a\times b)=0$.

 

 Some useful vector identities

$\begin{array}{l}a\cdot (b\times c)=b\cdot (c\times a)=c\cdot (a\times b)\\ a\times (b\times c)=(a\cdot c)b-(a\cdot b)c\\ (a\times b)\cdot (c\times d)=(a\cdot c)(b\cdot d)-(b\cdot c)(a\cdot d)\end{array}$

 

3. Cartesian components of vectors

 

Let $(e_1,e_2,e_3)$ be three mutually perpendicular unit vectors which form a right handed triad, Fig. 4.  Then $\left\{e_1,e_2,e_3\right\}$ are said to form and orthonormal basis. The vectors satisfy

$\left|e_1\right|=\left|e_2\right|=\left|e_3\right|=1e_1\times e_2=e_3,e_1\times e_3=-e_2{e}_2\times e_3=e_1$

We may express any vector a as a suitable combination of the unit vectors $e_1$, $e_2$ and $e_3$.  For example, we may write

$a=a_1{e}_1+a_2{e}_2+a_3{e}_3={\displaystyle \sum _{i=1}^3{a}_i{e}_i}$

where $(a_1,a_2,a_3)$ are scalars, called the components of a in the basis $\left\{e_1,e_2,e_3\right\}$.   The components of a have a simple physical interpretation.  For example, if we evaluate the dot product $a\cdot e_1$ we find that

$a\cdot e_1=(a_1{e}_1+a_2{e}_2+a_3{e}_3)\cdot e_1=a_1$

in view of the properties of the three vectors $e_1$, $e_2$ and $e_3$.  Recall that

$a\cdot e_1=\left|a\right|\left|e_1\right|\cos \theta (a,e_1)$

 

Then, noting that $\left|e_1\right|=1$, we have

$a_1=a\cdot e_1=\left|a\right|\cos \theta (a,e_1)$

 

Thus, $a_1$ represents the projected length of the vector a  in the direction of $e_1$, as illustrated in the figure.  Similarly, $a_2$ and $a_3$ may be shown to represent the projection of $a$ in the directions $e_2$ and $e_3$, respectively.

 

The advantage of representing vectors in a Cartesian basis is that vector addition and multiplication can be expressed as simple operations on the components of the vectors.  For example, let a, b and c be vectors, with components $(a_1,a_2,a_3)$, $(b_1,b_2,b_3)$ and $(c_1,c_2,c_3)$, respectively.  Then, it is straightforward to show that

$\begin{array}{c}c=a+b\iff c_1=a_1+b_1;c_2=a_2+b_2;c_3=a_3+b_3\\ a\cdot b={\displaystyle \sum _{i=1}^3{a}_i{b}_i}\\ c=a\times b\iff c_1=a_2{b}_3-a_3{b}_2;c_2=a_3{b}_1-a_1{b}_3;c_3=a_1{b}_2-a_2{b}_1\end{array}$

 

 

4. Change of basis

 

Let a be a vector, and let $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis.  Suppose that the components of a in the basis $\left\{e_1,e_2,e_3\right\}$ are known to be $(a_1,a_2,a_3)$.  Now, suppose that we wish to compute the components of a in a second Cartesian basis, $\left\{m_1,m_2,m_3\right\}$.  This means we wish to find components $({\alpha }_1,{\alpha }_2,{\alpha }_3)$, such that

$a={\alpha }_1{m}_1+{\alpha }_2{m}_2+{\alpha }_3{m}_3$

To do so, note that

$\begin{array}{l}{\alpha }_1=a\cdot m_1=a_1{e}_1\cdot m_1+a_2{e}_2\cdot m_1+a_3{e}_3\cdot m_1\\ {\alpha }_2=a\cdot m_2=a_1{e}_1\cdot m_2+a_2{e}_2\cdot m_2+a_3{e}_3\cdot m_2\\ {\alpha }_3=a\cdot m_3=a_1{e}_1\cdot m_3+a_2{e}_2\cdot m_3+a_3{e}_3\cdot m_3\end{array}$

This transformation is conveniently written as a matrix operation

$\left[\alpha \right]=\left[Q\right]\left[a\right]$,

where $\left[\alpha \right]$ is a matrix consisting of the components of a in the basis $\left\{m_1,m_2,m_3\right\}$, $\left[a\right]$ is a matrix consisting of the components of a in the basis $\left\{e_1,e_2,e_3\right\}$, and $\left[Q\right]$ is a `rotation matrix’ as follows

$\left[\alpha \right]=\left[\begin{array}{l}{\alpha }_1\\ {\alpha }_2\\ {\alpha }_3\end{array}\right]\left[a\right]=\left[\begin{array}{l}{a}_1\\ a_2\\ a_3\end{array}\right]\left[Q\right]=\left[\begin{array}{l}{m}_1\cdot e_1{m}_1\cdot e_2{m}_1\cdot e_3\\ m_2\cdot e_1{m}_2\cdot e_2{m}_2\cdot e_3\\ m_3\cdot e_1{m}_3\cdot e_2{m}_3\cdot e_3\end{array}\right]$

Note that the elements of $\left[Q\right]$ have a simple physical interpretation.  For example, $m_1\cdot e_1=\cos \theta (m_1,e_1)$, where $\theta (m_1,e_1)$ is the angle between the $m_1$ and $e_1$ axes.  Similarly $m_1\cdot e_2=\cos \theta (m_1,e_2)$ where $\theta (m_1,e_2)$ is the angle between the $m_1$ and $e_2$ axes.  In practice, we usually know the angles between the axes that make up the two bases, so it is simplest to assemble the elements of $\left[Q\right]$ by putting the cosines of the known angles in the appropriate places.

 

Index notation provides another convenient way to write this transformation:

${\alpha }_i=Q_{ij}{a}_j,Q_{ij}=e_i\cdot m_j$

You don’t need to know index notation in detail to understand this $–$ all you need to know is that

$Q_{ij}{a}_j\equiv {\displaystyle \sum _{j=1}^3{Q}_{ij}{a}_j}$

 

The same approach may be used to find an expression for $a_i$ in terms of ${\alpha }_i$.  If you work through the details, you will find that

$\left[\begin{array}{l}{a}_1\\ a_2\\ a_3\end{array}\right]=\left[\begin{array}{ccc}{m}_1\cdot e_1& m_2\cdot e_1& m_3\cdot e_1\\ m_1\cdot e_2& m_2\cdot e_2& m_3\cdot e_2\\ m_1\cdot e_3& m_2\cdot e_3& m_3\cdot e_3\end{array}\right]\left[\begin{array}{l}{\alpha }_1\\ {\alpha }_2\\ {\alpha }_3\end{array}\right]$

Comparing this result with the formula for ${\alpha }_i$ in terms of $a_i$, we see that

$\left[a\right]={\left[Q\right]}^T\left[\alpha \right]$

where the superscript T denotes the transpose (rows and columns interchanged). The transformation matrix $\left[Q\right]$ is therefore orthogonal, and satisfies

${\left[Q\right]}^{-1}={\left[Q\right]}^T\left[Q\right]{\left[Q\right]}^T={\left[Q\right]}^T\left[Q\right]=\left[I\right]$

where [I] is the identity matrix.

 

 

 

 

 

 

 

 

 

5. Useful vector operations

 Calculating areas

The area of a triangle bounded by vectors a, b¸and b-a is

$A=\frac{1}{2}|a\times b|$

The area of the parallelogram shown in the picture is 2A.

 

 Calculating angles

The angle between two vectors a and b is

$\theta ={\cos }^{-1}\left(a\cdot b/\left|a\right|\left|b\right|\right)$

 

 Calculating the normal to a surface.

If two vectors a and b can be found which are known to lie in the surface, then the unit normal to the surface is

$n=\pm \frac{a\times b}{\left|a\times b\right|}$

If the surface is specified by a parametric equation of the form $r=r(s,t)$, where s and t are two parameters and r is the position vector of a point on the surface, then two vectors which lie in the plane may be computed from

$a=\frac{\partial r}{\partial s},b=\frac{\partial r}{\partial t}$

 

 Calculating Volumes

The volume of the parallelopiped defined by three vectors a, b, c is

$V=|c\cdot \left(a\times b\right)|$

The volume of the tetrahedron shown outlined in red is V/6.

 

 

 

 

 

 VECTOR FIELDS AND VECTOR CALCULUS

 

 

1. Scalar field.

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  A scalar field is a scalar valued function of position in space.  A scalar field is a function of the components of the position vector, and so may be expressed as $\varphi (x_1,x_2,x_3)$. The value of $\varphi$ at a particular point in space must be independent of the choice of basis vectors.  A scalar field may be a function of time (and possibly other parameters) as well as position in space.

 

 

2. Vector field

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  A vector field is a vector valued function of position in space.  A vector field is a function of the components of the position vector, and so may be expressed as $v(x_1,x_2,x_3)$.  The vector may also be expressed as components in the basis $\left\{e_1,e_2,e_3\right\}$

$v(x_1,x_2,x_3)=v_1(x_1,x_2,x_3)e_1+v_2(x_1,x_2,x_3)e_2+v_3(x_1,x_2,x_3)e_3$

The magnitude and direction of $v$ at a particular point in space is independent of the choice of basis vectors.   A vector field may be a function of time (and possibly other parameters) as well as position in space.

 

 

3. Change of basis for scalar fields.

Let $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space. Express the position vector of a point relative to O in $\left\{e_1,e_2,e_3\right\}$ as

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

and let $\varphi (x_1,x_2,x_3)$ be a scalar field.

Let $\left\{m_1,m_2,m_3\right\}$ be a second Cartesian basis, with origin P.  Let $c\equiv \overrightarrow{OP}$ denote the position vector of  P relative to O. Express the position vector of a point relative to P in $\left\{m_1,m_2,m_3\right\}$ as

$p={\xi }_1{m}_1+{\xi }_2{m}_2+{\xi }_3{m}_3$

 

To find $\varphi ({\xi }_1,{\xi }_2,{\xi }_3)$, use the following procedure.  First, express  p as components in the basis $\left\{e_1,e_2,e_3\right\}$, using the procedure outlined in Section 1.4:

$p=p_1{e}_1+p_2{e}_2+p_3{e}_3$

where

$\begin{array}{l}{p}_1={\xi }_1{e}_1\cdot m_1+{\xi }_2{e}_2\cdot m_1+{\xi }_3{e}_3\cdot m_1\\ p_2={\xi }_1{e}_1\cdot m_2+{\xi }_2{e}_2\cdot m_2+{\xi }_3{e}_3\cdot m_2\\ p_3={\xi }_1{e}_1\cdot m_3+{\xi }_2{e}_2\cdot m_3+{\xi }_3{e}_3\cdot m_3\end{array}$

or, using index notation

$p_i=Q_{ij}{\xi }_j$

where the transformation matrix $Q_{ij}$ is defined in Sect 1.4.

Now, express c as components in $\left\{e_1,e_2,e_3\right\}$, and note that

$\begin{array}{l}r=p+c\\ \Rightarrow x_1{e}_1+x_2{e}_2+x_3{e}_3=p_1{e}_1+p_2{e}_2+p_3{e}_3+c_1{e}_1+c_2{e}_2+c_3{e}_3\\ \Rightarrow x_1=p_1+c_1,x_2=p_2+c_2,x_3=p_3+c_3\\ \Rightarrow x_i=Q_{ij}{\xi }_j+c_i\end{array}$

so that

$\begin{array}{l}\varphi (x_1,x_2,x_3)=\varphi (p_1+c_1,p_2+c_2,p_3+c_3)\\ =\varphi (Q_{1j}{\xi }_j+c_1,Q_{2j}{\xi }_j+c_2,Q_{3j}{\xi }_j+c_3)\end{array}$

 

 

 

4. Change of basis for vector fields.

Let $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space. Express the position vector of a point relative to O in $\left\{e_1,e_2,e_3\right\}$ as

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

and let $v(x_1,x_2,x_3)$ be a vector  field, with components

$v(x_1,x_2,x_3)=v_1(x_1,x_2,x_3)e_1+v_2(x_1,x_2,x_3)e_2+v_3(x_1,x_2,x_3)e_3$

Let $\left\{m_1,m_2,m_3\right\}$ be a second Cartesian basis, with origin P.  Let $c\equiv \overrightarrow{OP}$ denote the position vector of  P relative to O. Express the position vector of a point relative to P in $\left\{m_1,m_2,m_3\right\}$ as

$p={\xi }_1{m}_1+{\xi }_2{m}_2+{\xi }_3{m}_3$

 

To express the vector field as components in $\left\{m_1,m_2,m_3\right\}$ and as a function of the components of p, use the following procedure.  First, express $(v_1,v_2,v_3)$ in terms of $({\xi }_1,{\xi }_2,{\xi }_3)$ using the procedure outlined for scalar fields in the preceding section

$\begin{array}{l}{v}_k(x_1,x_2,x_3)=v_k(p_1+c_1,p_2+c_2,p_3+c_3)\\ =v_k(Q_{1j}{\xi }_j+c_1,Q_{2j}{\xi }_j+c_2,Q_{3j}{\xi }_j+c_3)\end{array}$

for k=1,2,3.  Now, find the components  of v in $\left\{m_1,m_2,m_3\right\}$ using the procedure outlined in Section 1.4.  Using index notation, the result is

$\begin{array}{l}v=Q_{1i}{v}_i(Q_{1j}{\xi }_j+c_1,Q_{2j}{\xi }_j+c_2,Q_{3j}{\xi }_j+c_3)e_1\\ +Q_{2i}{v}_i(Q_{1j}{\xi }_j+c_1,Q_{2j}{\xi }_j+c_2,Q_{3j}{\xi }_j+c_3)e_2\\ +Q_{2i}{v}_i(Q_{1j}{\xi }_j+c_1,Q_{2j}{\xi }_j+c_2,Q_{3j}{\xi }_j+c_3)e_3\end{array}$

 

 

5. Time derivatives of vectors

 

Let a(t) be a vector whose magnitude and direction vary with time, t.  Suppose that $\left\{i,j,k\right\}$ is a fixed basis, i.e. independent of time.  We may express a(t) in terms of components $(a_x,a_y,a_z)$ in the basis $\left\{i,j,k\right\}$ as

$a(t)=a_{x}i+a_{y}j+a_{z}k$.

The time derivative of a is defined using the usual rules of calculus

$\dot{a}(t)=\frac{d}{dt}a(t)=\underset{\in \to 0}{\lim }\frac{a(t+\in )-a(t)}{\in }$,

or in component form as

$\dot{a}(t)={\dot{a}}_{x}i+{\dot{a}}_{y}j+{\dot{a}}_{z}k$

The definition of the time derivative of a vector may be used to show the following rules

$\begin{array}{l}\frac{d}{dt}\left[\alpha (t)a(t)\right]=\dot{\alpha }(t)a(t)+\alpha (t)\dot{a}(t)\\ \frac{d}{dt}\left[a(t)\cdot b(t)\right]=\dot{a}(t)\cdot b(t)+a(t)\cdot \dot{b}(t)\\ \frac{d}{dt}\left[a(t)\times b(t)\right]=\dot{a}(t)\times b(t)+a(t)\times \dot{b}(t)\end{array}$

 

 

6. Using a rotating basis

 

It is often convenient to express position vectors as components in a basis which rotates with time.  To write equations of motion one must evaluate time derivatives of rotating vectors.

 

Let $\left\{e_1,e_2,e_3\right\}$ be a basis which rotates with instantaneous angular velocity $\Omega$.  Then,

$\frac{d{e}_1}{dt}=\Omega \times e_1,\frac{d{e}_2}{dt}=\Omega \times e_2,\frac{d{e}_3}{dt}=\Omega \times e_3$

 

 

7. Gradient of a scalar field.

 

Let $\varphi$ be a scalar field in three dimensional space.  The gradient of $\varphi$ is a vector field denoted by $\text{grad}(\varphi )$ or $\nabla \varphi$, and is defined so that

$(\nabla \varphi )\cdot a=\underset{\in \to 0}{\lim }\frac{\varphi (r+\in a)-\varphi (r)}{\in }$

for every position r in space and for every vector a.

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  Express $\varphi$ as a function of the components of r $\varphi =\varphi (x_1,x_2,x_3)$.  The gradient of  $\varphi$ in this basis is then given by

$\nabla \varphi =\frac{\partial \varphi }{\partial x_1}{e}_1+\frac{\partial \varphi }{\partial x_2}{e}_2+\frac{\partial \varphi }{\partial x_3}{e}_3$

 

8. Gradient of a vector field

 

Let v be a vector field in three dimensional space.  The gradient of v is a tensor field denoted by $\text{grad}(v)$ or $\nabla v$, and is defined so that

$(\nabla v)\cdot a=\underset{\in \to 0}{\lim }\frac{v(r+\in a)-v(r)}{\in }$

for every position r in space and for every vector a.

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  Express v as a function of the components of r, so that $v=v(x_1,x_2,x_3)$.  The gradient of  v in this basis is then given by

$\nabla v=\left[\begin{array}{ccc}\frac{\partial v_1}{\partial x_1}& \frac{\partial v_1}{\partial x_2}& \frac{\partial v_1}{\partial x_3}\\ \frac{\partial v_2}{\partial x_1}& \frac{\partial v_2}{\partial x_2}& \frac{\partial v_2}{\partial x_3}\\ \frac{\partial v_3}{\partial x_1}& \frac{\partial v_3}{\partial x_2}& \frac{\partial v_3}{\partial x_3}\end{array}\right]$

Alternatively, in index notation

${\left[\nabla v\right]}_{ij}\equiv \frac{\partial v_i}{\partial x_j}$

 

9. Divergence of a vector field

 

Let v be a vector field in three dimensional space.  The divergence of v is a scalar field denoted by $\text{div}(v)$ or $\nabla \cdot v$.  Formally, it is defined as $\text{trace(grad(}v\text{))}$ (the trace of a tensor is the sum of its diagonal terms). 

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  Express v as a function of the components of r: $v=v(x_1,x_2,x_3)$. The divergence of v is then

 

$\text{div(}v\text{)=}\frac{\partial v_1}{\partial x_1}+\frac{\partial v_2}{\partial x_2}+\frac{\partial v_3}{\partial x_3}$

 

 

10. Curl of a vector field.

 

Let v be a vector field in three dimensional space.  The curl of  v  is a vector field denoted by $\text{curl}(v)$ or $\nabla \times v$.  It is best defined in terms of its components in a given basis, although its magnitude and direction are not dependent on the choice of basis.

 

Let  $\left\{e_1,e_2,e_3\right\}$ be a Cartesian basis with origin O in three dimensional space.  Let

$r=x_1{e}_1+x_2{e}_2+x_3{e}_3$

denote the position vector of a point in space.  Express v as a function of the components of r  $v=v(x_1,x_2,x_3)$. The curl of  v in this basis is then given by

$\nabla \times v=\left|\begin{array}{ccc}{e}_1& e_2& e_3\\ \frac{\partial }{\partial x_1}& \frac{\partial }{\partial x_2}& \frac{\partial }{\partial x_3}\\ v_1& v_2& v_3\end{array}\right|=\left(\frac{\partial v_3}{\partial x_2}-\frac{\partial v_2}{\partial x_3}\right)e_1+\left(\frac{\partial v_1}{\partial x_3}-\frac{\partial v_3}{\partial x_1}\right)e_2+\left(\frac{\partial v_2}{\partial x_1}-\frac{\partial v_1}{\partial x_2}\right)e_3$

Using index notation, this may be expressed as

${\left[\nabla \times v\right]}_i={\in }_{ijk}\frac{\partial v_k}{\partial x_j}$

 

 

11. The Divergence Theorem.

Let V be a closed region in three dimensional space, bounded by an orientable surface S. Let n  denote the unit vector normal to S, taken so that n points out of V. Let u be a vector field which is continuous and has continuous first partial derivatives in some domain containing T.  Then

${\displaystyle \underset{V}{\int }\text{div(}u\text{)}}dV={\displaystyle \underset{S}{\int }u\cdot n}dA$

alternatively, expressed in index notation

${\displaystyle \underset{V}{\int }\frac{\partial u_i}{\partial x_i}dV={\displaystyle \underset{S}{\int }{u}_i{n}_{i}dA}}$

For a proof of this extremely useful theorem consult e.g. Kreyzig, Advanced Engineering Mathematics, Wiley, New York, (1998).

 

 

 

MATRICES

 

1. Definition

 

An $(n\times m)$ matrix $[A]$ is a set of numbers, arranged in m rows and n columns

$\left[A\right]=\left[\begin{array}{l}{a}_{11}{a}_{12}{a}_{13}\cdots a_{1n}\\ a_{21}{a}_{22}{a}_{23}\cdots a_{2n}\\ ⋮⋮⋮\ddots ⋮\\ a_{m1}{a}_{m2}{a}_{m3}\cdots a_{mn}\end{array}\right]$

 

 A square matrix has equal numbers of rows and columns

 A diagonal matrix is a square matrix with elements such that $a_{ij}=0$ for $i\ne j$

 The identity matrix $\left[I\right]$ is a diagonal matrix for which all diagonal elements $a_{ii}=1$

 A symmetric matrix is a square matrix with elements such that $a_{ij}=a_{ji}$

 A skew symmetric matrix is a square matrix with elements such that $a_{ij}=-a_{ji}$

 

 

2. Matrix operations

 

 Addition  Let  $\left[A\right]$ and $\left[B\right]$ be two matrices of order $(m\times n)$ with elements $a_{ij}$ and $b_{ij}$.  Then

$\left[C\right]=\left[A\right]+\left[B\right]\iff c_{ij}=a_{ij}+b_{ij}$

 

 

 Multiplication by a scalar.  Let $\left[A\right]$ be a matrix with elements $a_{ij}$, and let k be a scalar.  Then

$\left[B\right]=k\left[A\right]\iff b_{ij}=k{a}_{ij}$

 

 

 Multiplication by a matrix. Let $\left[A\right]$ be a matrix of order $(m\times n)$ with elements $a_{ij}$, and let $\left[B\right]$ be a matrix of order $(p\times q)$ with elements $b_{ij}$.  The product $\left[C\right]=\left[A\right]\left[B\right]$ is defined only if n=p, and is an $(m\times q)$ matrix such that

$\left[C\right]=\left[A\right]\left[B\right]\iff c_{ij}={\displaystyle \sum _{k=1}^n{a}_{ik}{b}_{kj}}$

Note that multiplication is distributive and associative, but not commutative, i.e.

$\left[A\right]\left(\left[B\right]+\left[C\right]\right)=\left[A\right]\left[B\right]+\left[A\right]\left[C\right]\left[A\right]\left(\left[B\right]\left[C\right]\right)=\left(\left[A\right]\left[B\right]\right)\left[C\right]\left[A\right]\left[B\right]\ne \left[B\right]\left[A\right]$

The multiplication of a vector by a matrix is a particularly important operation.  Let b and c be two vectors with n components, which we think of as $(1\times n)$ matrices.  Let $\left[A\right]$ be an $(m\times n)$ matrix.  Thus

$b=\left[\begin{array}{l}{b}_1\\ b_2\\ b_3\\ ⋮\\ b_n\end{array}\right]c=\left[\begin{array}{l}{c}_1\\ c_2\\ c_3\\ ⋮\\ c_n\end{array}\right]\left[A\right]=\left[\begin{array}{l}{a}_{11}{a}_{12}{a}_{13}\cdots a_{1n}\\ a_{21}{a}_{22}{a}_{23}\cdots a_{2n}\\ ⋮⋮⋮\ddots ⋮\\ a_{m1}{a}_{m2}{a}_{m3}\cdots a_{mn}\end{array}\right]$

Now,

$c=\left[A\right]b\iff c_i={\displaystyle \sum _{j=1}^n{a}_{ij}{b}_j}$

i.e.

$\begin{array}{l}{c}_1=a_{11}{b}_1+a_{12}{b}_2+a_{13}{b}_3+\cdots a_{1n}{b}_n\\ c_2=a_{21}{b}_1+a_{22}{b}_2+a_{23}{b}_3+\cdots a_{2n}{b}_n\\ ⋮\\ c_m=a_{m1}{b}_1+a_{m2}{b}_2+a_{m3}{b}_3+\cdots a_{mn}{b}_n\end{array}$

 

 Transpose. Let $\left[A\right]$ be a matrix of order $(m\times n)$ with elements $a_{ij}$.  The transpose of $\left[A\right]$ is denoted ${\left[A\right]}^T$.  If $\left[B\right]$ is an $(n\times m)$ matrix such that $\left[B\right]={\left[A\right]}^T$, then $b_{ij}=a_{ji}$, i.e.

${\left[A\right]}^T={\left[\begin{array}{l}{a}_{11}{a}_{12}{a}_{13}\cdots a_{1n}\\ a_{21}{a}_{22}{a}_{23}\cdots a_{2n}\\ ⋮⋮⋮\ddots ⋮\\ a_{m1}{a}_{m2}{a}_{m3}\cdots a_{mn}\end{array}\right]}^T=\left[\begin{array}{l}{a}_{11}{a}_{21}{a}_{31}\cdots a_{n1}\\ a_{12}{a}_{22}{a}_3{}_2\cdots a_{n2}\\ ⋮⋮⋮\ddots ⋮\\ a_{1m}{a}_{2m}{a}_{3m}\cdots a_{nm}\end{array}\right]$

Note that

${\left(\left[A\right]\left[B\right]\right)}^T={\left[B\right]}^T{\left[A\right]}^T$

 

 Determinant  The determinant is defined only for a square matrix.  Let $\left[A\right]$ be a $(2\times 2)$ matrix with components $a_{ij}$.  The determinant of  $\left[A\right]$ is denoted by $\det \left[A\right]$ or $\left|A\right|$ and is given by

$\left|A\right|=\left|\begin{array}{l}{a}_{11}{a}_{12}\\ a_{21}{a}_{22}\end{array}\right|=a_{11}{a}_{22}-a_{12}{a}_{21}$

Now, let $\left[A\right]$ be an $\left(n\times n\right)$ matrix.  Define the minors $M_{ij}$ of $\left[A\right]$ as the determinant formed by omitting the ith row and jth column of $\left[A\right]$.  For example, the minors $M_{11}$ and $M_{12}$ for a $\left(3\times 3\right)$ matrix are computed as follows.   Let

$\left[A\right]=\left[\begin{array}{l}{a}_{11}{a}_{12}{a}_{13}\\ a_{21}{a}_{22}{a}_{23}\\ a_{31}{a}_{32}{a}_{33}\end{array}\right]$

Then

$M_{11}=\left|\begin{array}{l}{a}_{22}{a}_{23}\\ a_{32}{a}_{33}\end{array}\right|=a_{22}{a}_{33}-a_{32}{a}_{23}{M}_{12}=\left|\begin{array}{l}{a}_{21}{a}_{23}\\ a_{31}{a}_{33}\end{array}\right|=a_{21}{a}_{33}-a_{31}{a}_{23}$

Define the cofactors $C_{ij}$ of $\left[A\right]$ as

$C_{ij}={\left(-1\right)}^{i+j}{M}_{ij}$

Then, the determinant of the $\left(n\times n\right)$ matrix $\left[A\right]$ is computed as follows

$\left|A\right|={\displaystyle \sum _{j=1}^n{a}_{ij}{C}_{ij}}$

The result is the same whichever row i is chosen for the expansion.  For the particular case of a $\left(3\times 3\right)$ matrix

$\det \left[A\right]=\det \left[\begin{array}{l}{a}_{11}{a}_{12}{a}_{13}\\ a_{21}{a}_{22}{a}_{23}\\ a_{31}{a}_{32}{a}_{33}\end{array}\right]=a_{11}(a_{22}{a}_{33}-a_{23}{a}_{32})+a_{12}(a_{23}{a}_{31}-a_{21}{a}_{33})+a_{13}(a_{21}{a}_{32}-a_{31}{a}_{22})$

The determinant may also be evaluated by summing over rows, i.e.

$\left|A\right|={\displaystyle \sum _{i=1}^n{a}_{ij}{C}_{ij}}$

and as before the result is the same for each choice of column j.  Finally, note that

$\det {\left[A\right]}^T=\det \left[A\right]\det \left(\left[A\right]\left[B\right]\right)=\det \left[A\right]\det \left[B\right]$

 

 Inversion.  Let $\left[A\right]$ be an $\left(n\times n\right)$ matrix.  The inverse of $\left[A\right]$ is denoted by ${\left[A\right]}^{-1}$ and is defined such that

${\left[A\right]}^{-1}\left[A\right]=\left[I\right]$

The inverse of $\left[A\right]$ exists if and only if $\det \left[A\right]\ne 0$.  A matrix which has no inverse is said to be singular.  The inverse of a matrix may be computed explicitly, by forming the cofactor matrix $\left[C\right]$ with components $c_{ij}$ as defined in the preceding section.  Then

${\left[A\right]}^{-1}=\frac{1}{\det \left[A\right]}{\left[C\right]}^T$

In practice, it is faster to compute the inverse of a matrix using methods such as Gaussian elimination. 

 

Note that

${\left(\left[A\right]\left[B\right]\right)}^{-1}={\left[B\right]}^{-1}{\left[A\right]}^{-1}$

For a diagonal matrix, the inverse is

$\left[A\right]=\left[\begin{array}{l}{a}_{11}00\cdots 0\\ 0a_{22}0\cdots 0\\ ⋮⋮⋮\ddots ⋮\\ 000\cdots a_{nn}\end{array}\right]=\left[\begin{array}{l}1/a_{11}00\cdots 0\\ 01/a_{22}0\cdots 0\\ ⋮⋮⋮\ddots ⋮\\ 000\cdots 1/a_{nn}\end{array}\right]$

For a $\left(2\times 2\right)$ matrix, the inverse is

$\left[\begin{array}{l}{a}_{11}{a}_{12}\\ a_{21}{a}_{22}\end{array}\right]=\frac{1}{a_{11}{a}_{22}-a_{12}{a}_{21}}\left[\begin{array}{l}{a}_{22}-a_{12}\\ -a_{21}{a}_{11}\end{array}\right]$

 

 Eigenvalues and eigenvectors. Let $\left[A\right]$ be an $\left(n\times n\right)$ matrix, with coefficients $a_{ij}$.  Consider the vector equation

$\left[A\right]x=\lambda x$                                                (1)

where x is a vector with n components, and $\lambda$ is a scalar (which may be complex).  The n nonzero vectors x and corresponding scalars $\lambda$ which satisfy this equation are the eigenvectors and eigenvalues of $\left[A\right]$.

 

Formally, eighenvalues and eigenvectors may be computed as follows.  Rearrange the preceding equation to

$\left(\left[A\right]-\lambda \left[I\right]\right)x=0$                                     (2)

This has nontrivial solutions for x only if the determinant of the matrix $\left(\left[A\right]-\lambda \left[I\right]\right)$ vanishes.  The equation

$\det \left(\left[A\right]-\lambda \left[I\right]\right)=0$

is an nth order polynomial which may be solved for $\lambda$.  In general the polynomial will have n roots, which may be complex.  The eigenvectors may then be computed using equation (2).  For example, a $(2\times 2)$ matrix generally has two eigenvectors, which satisfy

$\left|A-\lambda I\right|=\left|\begin{array}{l}{a}_{11}-\lambda a_{12}\\ a_{21}{a}_{22}-\lambda \end{array}\right|=(a_{11}-\lambda )(a_{22}-\lambda )-a_{12}{a}_{21}=0$

Solve the quadratic equation to see that

$\begin{array}{l}{\lambda }_1=\frac{1}{2}\left(a_{11}+a_{22}\right)-\frac{1}{2}{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\\ {\lambda }_2=\frac{1}{2}\left(a_{11}+a_{22}\right)+\frac{1}{2}{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\end{array}$

The two corresponding eigenvectors may be computed from (2), which shows that

$\left[\begin{array}{l}{a}_{11}-{\lambda }_i{a}_{12}\\ a_{21}{a}_{22}-{\lambda }_i\end{array}\right]\left[\begin{array}{l}{x}_1^{(i)}\\ x_2^{(i)}\end{array}\right]=0$

so that, multiplying out the first row of the matrix (you can use the second row too, if you wish $–$ since we chose $\lambda$ to make the determinant of the matrix vanish, the two equations have the same solutions.  In fact, if $a_{12}=0$, you will need to do this, because the first equation will simply give 0=0 when trying to solve for one of the eigenvectors)

$\begin{array}{l}\left(\frac{1}{2}\left(a_{11}-a_{22}\right)+\frac{1}{2}{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\right)x_1^{(1)}+a_{12}{x}_2^{(1)}=0\\ \left(\frac{1}{2}\left(a_{11}-a_{22}\right)-\frac{1}{2}{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\right)x_1^{(2)}+a_{12}{x}_2^{(2)}=0\end{array}$

which are satisfied by any vector of the form

$\begin{array}{l}{x}^{(1)}=\left[\begin{array}{l}2a_{12}\\ \left(a_{11}-a_{22}\right)+{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\end{array}\right]p\\ x^{(2)}=\left[\begin{array}{l}2a_{12}\\ \left(a_{11}-a_{22}\right)-{\left\{{\left(a_{11}+a_{22}\right)}^2-4\left(a_{11}{a}_{22}-a_{12}{a}_{21}\right)\right\}}^{1/2}\end{array}\right]q\end{array}$

where p and q are arbitrary real numbers.

 

It is often convenient to normalize eigenvectors so that they have unit `length’.  For this purpose, choose p and q so that $x^{(i)}\cdot x^{(i)}=1$.  (For vectors of dimension n, the generalized dot product is defined such that $x\cdot x={\displaystyle {\sum }_{i=1}^n{x}_i{x}_i}$ )

 

One may calculate explicit expressions for eigenvalues and eigenvectors for any matrix up to order $\left(4\times 4\right)$, but the results are so cumbersome that, except for the $\left(2\times 2\right)$ results, they are virtually useless.  In practice, numerical values may be computed using several iterative techniques.  Packages like Mathematica, Maple or Matlab make calculations like this easy.

 

The eigenvalues of a real symmetric matrix are always real, and its eigenvectors are orthogonal, i.e. the ith and jth eigenvectors (with $i\ne j$ ) satisfy $x^{(i)}\cdot x^{(j)}=0$.

 

The eigenvalues of a skew symmetric matrix are pure imaginary.

 

 Spectral and singular value decomposition.  Let $\left[A\right]$ be a real symmetric  $\left(n\times n\right)$ matrix. Denote the n (real) eigenvalues of $\left[A\right]$ by ${\lambda }_i$, and let $w^{(i)}$ be the corresponding normalized eigenvectors, such that $w^{(i)}\cdot w^{(i)}=1$.  Then, for any arbitrary vector b,

$\left[A\right]b={\displaystyle \sum _{i=1}^n{\lambda }_i\left(w^{(i)}\cdot b\right)w^{(i)}}$

Let $\left[\Lambda \right]$ be a diagonal matrix which contains the n eigenvalues of $\left[A\right]$ as elements of the diagonal, and let $\left[Q\right]$ be a matrix consisting of the n eigenvectors as columns, i.e.

$\left[\Lambda \right]=\left[\begin{array}{l}{\lambda }_{1}00\cdots 0\\ 0{\lambda }_{2}0\cdots 0\\ ⋮⋮⋮\ddots ⋮\\ 000\cdots {\lambda }_n\end{array}\right]\left[Q\right]=\left[\begin{array}{l}{w}_1^{(1)}{w}_1^{(2)}{w}_1^{(3)}\cdots w_1^{(n)}\\ w_2^{(1)}{w}_2^{(2)}{w}_2^{(3)}\cdots w_2^{(n)}\\ ⋮⋮⋮\ddots ⋮\\ w_n^{(1)}{w}_n^{(2)}{w}_n^{(3)}\cdots w_n^{(n)}\end{array}\right]$

Then

$\left[A\right]=\left[Q\right]\left[\Lambda \right]{\left[Q\right]}^T{\left[Q\right]}^T\left[Q\right]=\left[Q\right]{\left[Q\right]}^T=\left[I\right]{\left[Q\right]}^T\left[A\right]\left[Q\right]=\left[\Lambda \right]$

Note that this gives another (generally quite useless) way to invert $\left[A\right]$

${\left[A\right]}^{-1}=\left[Q\right]{\left[\Lambda \right]}^{-1}{\left[Q\right]}^T$

where ${\left[\Lambda \right]}^{-1}$ is easy to compute since $\left[\Lambda \right]$ is diagonal.

 

 Square root of a matrix.   Let $\left[A\right]$ be a real symmetric  $\left(n\times n\right)$ matrix.  Denote the singular value decomposition of $\left[A\right]$ by $\left[A\right]=\left[Q\right]\left[\Lambda \right]{\left[Q\right]}^T$ as defined above.  Suppose that $\left[S\right]={\left[A\right]}^{1/2}$ denotes the square root of $\left[A\right]$, defined so that

$\left[S\right]\left[S\right]=\left[A\right]$

One way to compute $\left[S\right]$ is through the singular value decomposition of $\left[A\right]$

$\left[S\right]=\left[Q\right]{\left[\Lambda \right]}^{1/2}{\left[Q\right]}^T$

where

${\left[\Lambda \right]}^{1/2}=\left[\begin{array}{l}\sqrt{{\lambda }_1}00\cdots 0\\ 0\sqrt{{\lambda }_2}0\cdots 0\\ ⋮⋮⋮\ddots ⋮\\ 000\cdots \sqrt{{\lambda }_n}\end{array}\right]$