least-squares.org
author Santosh G. Vattam <vattam.santosh@gmail.com>
Fri, 16 Apr 2010 12:25:39 +0530
changeset 76 6dfdb6fc9d80
parent 75 3a94917224e9
child 81 2eff0ebac2dc
permissions -rw-r--r--
Minor edits.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     1
* Least Squares Fit
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     2
*** Outline
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     3
***** Introduction
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     4
******* What do we want to do? Why?
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     5
********* What's a least square fit?
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     6
********* Why is it useful?
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     7
******* How are we doing it?
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     8
******* Arsenal Required
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     9
********* working knowledge of arrays
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    10
********* plotting
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    11
********* file reading
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    12
***** Procedure
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    13
******* The equation (for a single point)
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    14
******* It's matrix form
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    15
******* Getting the required matrices
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    16
******* getting the solution
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    17
******* plotting
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    18
*** Script
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    19
    Welcome. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    20
    
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    21
    In this tutorial we shall look at obtaining the least squares fit
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    22
    of a given data-set. For this purpose, we shall use the same
75
3a94917224e9 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 2
diff changeset
    23
    pendulum data that we used in the tutorial on plotting from files.
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    24
    Hope you have the file with you. 
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    25
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    26
    To be able to follow this tutorial comfortably, you should have a
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    27
    working knowledge of arrays, plotting and file reading. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    28
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    29
    A least squares fit curve is the curve for which the sum of the
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    30
    squares of it's distance from the given set of points is
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    31
    minimum. We shall use the lstsq function to obtain the least
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    32
    squares fit curve. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    33
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    34
    In our example, we know that the length of the pendulum is
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    35
    proportional to the square of the time-period. Therefore, we
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    36
    expect the least squares fit curve to be a straight line. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    37
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    38
    The equation of the line is of the form T^2 = mL+c. We have a set
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    39
    of values for L and the corresponding T^2 values. Using this, we
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    40
    wish to obtain the equation of the straight line. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    41
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    42
    In matrix form the equation is represented as shown, 
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    43
    Tsq = A.p where Tsq is an NX1 matrix, and A is an NX2 as shown.
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    44
    And p is a 2X1 matrix of the slope and Y-intercept. In order to 
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    45
    obtain the least square fit curve we need to find the matrix p
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    46
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    47
    Let's get started. As you can see, the file pendulum.txt
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    48
    is on our Desktop and hence we navigate to the Desktop by typing 
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    49
    cd Desktop. Let's now fire up IPython: ipython -pylab
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    50
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    51
    We have already seen (in a previous tutorial), how to read a file
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    52
    and obtain the data set using loadtxt(). Let's quickly get the required data
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    53
    from our file. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    54
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    55
    l, t = loadtxt('pendulum.txt', unpack=True)
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    56
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    57
    loadtxt() directly stores the values in the pendulum.txt into arrays l and t
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    58
    Let's now calculate the values of square of the time-period. 
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    59
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    60
    tsq = t*t
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    61
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    62
    Now we shall obtain A, in the desired form using some simple array
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    63
    manipulation 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    64
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    65
    A = array([l, ones_like(l)])
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    66
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    67
    As we have seen in a previous tutorial, ones_like() gives an array similar
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    68
    in shape to the given array, in this case l, with all the elements as 1.
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    69
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    70
    A = A.T to get the transpose of the given array.
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    71
    
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    72
    Type A, to confirm that we have obtained the desired array. 
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    73
    A
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    74
    Also note the shape of A. 
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    75
    A.shape
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    76
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    77
    We shall now use the lstsq function, to obtain the coefficients m
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    78
    and c. lstsq returns a lot of things along with these
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    79
    coefficients. We may look at the documentation of lstsq, for more
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    80
    information by typing lstsq? 
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    81
    result = lstsq(A,tsq)
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    82
75
3a94917224e9 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 2
diff changeset
    83
    We extract the required coefficients, which is the first element
3a94917224e9 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 2
diff changeset
    84
    in the list of things that lstsq returns, and store them into the variable coef. 
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    85
    coef = result[0]
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    86
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    87
    To obtain the plot of the line, we simply use the equation of the
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    88
    line, we have noted before. T^2 = mL + c. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    89
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    90
    Tline = coef[0]*l + coef[1]
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    91
    plot(l, Tline)
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    92
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    93
    Also, it would be nice to have a plot of the points. So, 
76
6dfdb6fc9d80 Minor edits.
Santosh G. Vattam <vattam.santosh@gmail.com>
parents: 75
diff changeset
    94
    plot(l, tsq, 'o')
2
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    95
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    96
    This brings us to the end of this tutorial. In this tutorial,
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    97
    you've learnt how to obtain a least squares fit curve for a given
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    98
    set of points. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    99
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   100
    Hope you enjoyed it. Thanks. 
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   101
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   102
*** Notes
008c0edc6eac Added scripts for session-4 and session-6 of day-1.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   103