{"id":377,"date":"2024-12-05T07:03:11","date_gmt":"2024-12-05T07:03:11","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2024\/12\/05\/how-to-interpret-matrix-expressions-transformations-a5e6871cd224\/"},"modified":"2024-12-05T07:03:11","modified_gmt":"2024-12-05T07:03:11","slug":"how-to-interpret-matrix-expressions-transformations-a5e6871cd224","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2024\/12\/05\/how-to-interpret-matrix-expressions-transformations-a5e6871cd224\/","title":{"rendered":"How to Interpret Matrix Expressions\u200a\u2014\u200aTransformations"},"content":{"rendered":"

How to Interpret Matrix Expressions\u200a\u2014\u200aTransformations
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Matrix algebra for a data scientist<\/h4>\n
\"\"
Photo by Ben Allan<\/a> on\u00a0Unsplash<\/a><\/figcaption><\/figure>\n

This article begins a series for anyone who finds matrix algebra overwhelming. My goal is to turn what you\u2019re afraid of<\/em> into what you\u2019re fascinated by<\/em>. You\u2019ll find it especially helpful if you want to understand machine learning concepts and\u00a0methods<\/strong>.<\/p>\n

Table of contents:<\/strong><\/h4>\n
    \n
  1. Introduction<\/li>\n
  2. Prerequisites<\/li>\n
  3. Matrix-vector multiplication<\/li>\n
  4. Transposition<\/li>\n
  5. Composition of transformations<\/li>\n
  6. Inverse transformation<\/li>\n
  7. Non-invertible transformations<\/li>\n
  8. Determinant<\/li>\n
  9. Non-square matrices<\/li>\n
  10. Inverse and Transpose: similarities and differences<\/li>\n
  11. Translation by a\u00a0vector<\/li>\n
  12. Final words<\/li>\n<\/ol>\n

    1. Introduction<\/h3>\n

    You\u2019ve probably noticed that while it\u2019s easy to find materials explaining matrix computation algorithms, it\u2019s harder to find ones that teach how to interpret complex matrix expressions<\/strong>. I\u2019m addressing this gap with my series, focused on the part of matrix algebra that is most commonly used by data scientists<\/strong>.<\/p>\n

    We\u2019ll focus more on concrete examples rather than general formulas. I\u2019d rather sacrifice generality for the sake of clarity and readability. I\u2019ll often appeal to your imagination and intuition, hoping my materials will inspire you to explore more formal resources on these topics. For precise definitions and general formulas, I\u2019d recommend you look at some good textbooks: the classic one on linear algebra\u00b9 and the other focused on machine learning\u00b2.<\/p>\n

    This part will teach\u00a0you<\/p>\n

    to see a matrix as a representation of the transformation applied to\u00a0data.<\/p><\/blockquote>\n

    Let\u2019s get started then\u200a\u2014\u200alet me take the lead through the world of matrices.<\/p>\n

    2. Prerequisites<\/h3>\n

    I\u2019m guessing you can handle the expressions that\u00a0follow.<\/p>\n

    This is the dot product<\/strong> written using a row vector and a column\u00a0vector:<\/p>\n

    \"\"<\/figure>\n

    A matrix<\/strong> is a rectangular array of symbols arranged in rows and columns. Here is an example of a matrix with two rows and three\u00a0columns:<\/p>\n

    \"\"<\/figure>\n

    You can view it as a sequence of\u00a0columns<\/strong><\/p>\n

    \"\"<\/figure>\n
    \"\"<\/figure>\n

    or a sequence of rows<\/strong> stacked one on top of\u00a0another:<\/p>\n

    \"\"<\/figure>\n

    As you can see, I used superscripts for rows and subscripts for columns. In machine learning, it\u2019s important to clearly distinguish between observations, represented as vectors, and features, which are arranged in\u00a0rows.<\/p>\n

    Other interesting ways to represent this matrix are A<\/strong>\u2082<\/em>\u2093\u2083 <\/em>and A<\/strong>[a\u1d62<\/em>\u207d\u02b2\u00a0<\/em>\u207e].<\/p>\n

    Multiplying<\/strong> two matrices A <\/strong>and B <\/strong>results in a third matrix C <\/strong>= AB<\/strong> containing the scalar products of each row of A <\/strong>with each column of B<\/strong>, arranged accordingly. Below is an example for C<\/strong>\u2082<\/em>\u2093\u2082<\/em> <\/strong>= A<\/strong>\u2082<\/em>\u2093\u2083<\/em>B<\/strong>\u2083<\/em>\u2093\u2082.<\/em><\/p>\n

    \"\"<\/figure>\n

    where c\u1d62<\/em>\u207d\u02b2 <\/em>\u207e is the scalar product of the i<\/em>-th column of the matrix B<\/strong> and the j<\/em>-th row of matrix\u00a0A<\/strong>:<\/p>\n

    \"\"<\/figure>\n

    Note that this definition of multiplication requires the number of rows of the left matrix<\/em> to match the number of columns of the right matrix<\/em>. In other words, the inner dimensions of the matrices must\u00a0match<\/strong>.<\/p>\n

    Make sure you can manually multiply matrices with arbitrary entries. You can use the following code to check the result or to practice multiplying matrices.<\/p>\n

    import numpy as np

    # Matrices to be multiplied
    A = [
        [ 1, 0, 2],
        [-2, 1, 1]
    ]

    B = [
        [ 0, 3, 1],
        [-3, 1, 1],
        [-2, 2, 1]
    ]

    # Convert to numpy array
    A = np.array(A)
    B = np.array(B)

    # Multiply A by B (if possible)
    try:
        C = A @ B
        print(f'A B = n{C}n')
    except:
        print(\"\"\"ValueError:
    The number of rows in matrix A does not match 
    the number of columns in matrix B
    \"\"\")

    #  and in the reverse order, B by A (if possible)
    try:
        D = B @ A
        print(f'B A =n{D}')
    except:
        print(\"\"\"ValueError:
    The number of rows in matrix B does not match 
    the number of columns in matrix A
    \"\"\")<\/pre>\n
    A B = 
    [[-4  7]
     [-5 -3]]

    B A =
    [[-6  3  3]
     [-5  1 -5]
     [-6  2 -2]]<\/pre>\n
    3. Matrix-vector multiplication<\/h3>\n
    In this section, I will explain the effect of matrix multiplication on vectors. The vector x<\/strong> is multiplied by the matrix A<\/strong>, producing a new vector\u00a0y<\/strong>:<\/p>\n
    \"\"<\/figure>\n
    This is a common operation in data science, as it enables a linear transformation of data<\/strong>. The use of matrices to represent linear transformations is highly advantageous, as you will soon see in the following examples.<\/p>\n
    Below, you can see your grid space and your standard basis vectors: blue for the x<\/em>\u207d\u00b9\u207e direction and magenta for the x<\/em>\u207d\u00b2\u207e direction.<\/p>\n
    \"\"
    Standard basis in a Grid\u00a0Space<\/figcaption><\/figure>\n
    A good starting point is to work with transformations that map two-dimensional vectors x<\/strong> into two-dimensional vectors y<\/strong> in the same grid\u00a0space.<\/p>\n
    Describing the desired transformation is a simple trick. You just need to say how the coordinates of the basis vectors change after the transformation and use these new coordinates as the columns of the matrix\u00a0A.<\/strong>\n<\/p><\/blockquote>\n
    As an example, consider a linear transformation that produces the effect illustrated below. The standard basis vectors are drawn lightly, while the transformed vectors are shown more\u00a0clearly.<\/p>\n
    \"\"
    Standard basis transformed by matrix\u00a0A<\/strong><\/figcaption><\/figure>\n
    From the comparison of the basis vectors before and after the transformation, you can observe that the transformation involves a 45-degree counterclockwise rotation about the origin, along with an elongation of the\u00a0vectors.<\/p>\n
    This effect can be achieved using the matrix A<\/strong>, composed as\u00a0follows:<\/p>\n
    \"\"<\/figure>\n
    The first column of the matrix contains the coordinates of the first basis vector after the transformation, and the second column contains those of the second basis\u00a0vector.<\/p>\n
    The equation (1) then takes the\u00a0form<\/p>\n
    \"\"<\/figure>\n
    Let\u2019s take two example points x<\/strong>\u2081and x<\/strong>\u2082\u00a0:<\/p>\n
    \"\"<\/figure>\n
    and transform them into the vectors y<\/strong>\u2081\u200b and y<\/strong>\u2082\u00a0:<\/p>\n
    \"\"<\/figure>\n
    I encourage you to do these calculations by hand first, and then switch to using a program like\u00a0this:<\/p>\n
    import numpy as np

    # Transformation matrix
    A = np.array([
        [1, -1],
        [1,  1]
    ])

    # Points (vectors) to be transformed using matrix A
    points = [
        np.array([1, 1\/2]),
        np.array([-1\/4, 5\/4])
    ]

    # Print out the transformed points (vectors)
    for i, x in enumerate(points):
        y = A @ x
        print(f'y_{i} = {y}')<\/pre>\n
    y_0 = [0.5 1.5]
    y_1 = [-1.5  1. ]<\/pre>\n
    The plot below shows the\u00a0results.<\/p>\n
    \"\"
    Points transformed by matrix\u00a0A<\/strong><\/figcaption><\/figure>\n
    The x<\/strong> points are gray\u00a0and\u00a0smaller, while their transformed counterparts y<\/strong> have black edges\u00a0and\u00a0are\u00a0bigger. If you\u2019d prefer to think of these points as arrowheads, here\u2019s the corresponding illustration:<\/p>\n
    \"\"
    Vectors transformed by matrix\u00a0A<\/strong><\/figcaption><\/figure>\n
    Now you can see more clearly that the points have been rotated around the origin and pushed a little\u00a0away.<\/p>\n
    Let\u2019s examine another\u00a0matrix:<\/p>\n
    \"\"<\/figure>\n
    and see how the transformation<\/p>\n
    \"\"<\/figure>\n
    affects the points on the grid\u00a0lines:<\/p>\n
    \"\"
    Grid lines transformed by matrix\u00a0B<\/strong><\/figcaption><\/figure>\n
    Compare the result with that obtained using B<\/strong>\/2, which corresponds to dividing all elements of the matrix B<\/strong> by\u00a02:<\/p>\n
    \"\"
    Grid lines transformed by matrix\u00a0B<\/strong>\/2<\/figcaption><\/figure>\n
    In general, a linear transformation:<\/p>\n