loading-data-from-files.rst
author Puneeth Chaganti <punchagan@gmail.com>
Wed, 22 Sep 2010 15:22:21 +0530
changeset 195 e8a251048213
parent 144 476ea1730aee
child 192 3a839dca2bdb
permissions -rw-r--r--
Comments and changes for lstsq script.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
144
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     1
========
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     2
 Script
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     3
========
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     4
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     5
Welcome to this tutorial on loading data from files. 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     6
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     7
{{{ Screen shows welcome slide }}}
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     8
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
     9
Until now, all the plots we have made use analytic functions. We have
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    10
been using analytic functions to generate a sequence of points and
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    11
plotting them, against another sequence of points. But, this is not
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    12
what we do most often. We often require to plot points obtained from
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    13
experimental observations.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    14
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    15
#[punch: the initial part of the paragraph may be removed, to make
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    16
this a more generic LO?]
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    17
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    18
In this tutorial we shall learn to read data from files and save it
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    19
into sequences that can later be used to plot.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    20
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    21
{{{ Show the outline for this tutorial }}} 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    22
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    23
We shall use the ``loadtxt`` command to load data from files. We will
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    24
be looking at how to get multiple columns of data into multiple
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    25
sequences.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    26
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    27
{{{ switch back to the terminal }}}
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    28
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    29
As usual, let us start IPython, using 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    30
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    31
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    32
  ipython -pylab 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    33
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    34
Now, Let us begin with reading the file primes.txt, which contains
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    35
just a list of primes listed in a column, using the loadtxt command.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    36
The file, in our case, is present in ``/home/fossee/primes.txt``.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    37
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    38
#[punch: do we need a slide for showing the path?]
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    39
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    40
We use the ``cat`` command to see the contents of this file. 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    41
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    42
#[punch: should we show the cat command here? seems like a good place
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    43
to do it] ::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    44
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    45
  cat /home/fossee/primes.txt
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    46
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    47
Now let us read this list into the variable ``primes``.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    48
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    49
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    50
  primes = loadtxt('/home/fossee/primes.txt')
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    51
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    52
``primes`` is now a sequence of primes, that was listed in the file,
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    53
``primes.txt``.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    54
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    55
We now type, ``print primes`` to see the sequence printed.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    56
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    57
We observe that all of the numbers end with a period. This is so,
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    58
because these numbers are actually read as ``floats``. We shall learn
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    59
about them, later.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    60
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    61
Now, let us use the ``loadtxt`` command to read a file that contains
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    62
two columns of data, ``pendulum.txt``. This file contains the length
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    63
of the pendulum in the first column and the corresponding time period
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    64
in the second.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    65
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    66
%%1%% Pause the video here, and use the ``cat`` command to view the
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    67
contents of this file and then resume the video.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    68
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    69
This is how we look at the contents of the file, ``pendulum.txt``
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    70
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    71
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    72
  cat /home/fossee/pendulum.txt
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    73
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    74
Let us, now, read the data into the variable ``pend``. Again, it is
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    75
assumed that the file is in ``/home/fossee/``
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    76
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    77
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    78
  pend = loadtxt('/home/fossee/pendulum.txt')
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    79
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    80
Let us now print the variable ``pend`` and see what's in it. 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    81
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    82
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    83
  print pend
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    84
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    85
Notice that ``pend`` is not a simple sequence like ``primes``. It has
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    86
two sequences, containing both the columns of the data file. Let us
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    87
use an additional argument of the ``loadtxt`` command, to read it into
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    88
two separate, simple sequences.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    89
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    90
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    91
  L, T = loadtxt('/home/fossee/pendulum.txt', unpack=True)
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    92
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    93
Let us now, print the variables L and T, to see what they contain.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    94
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    95
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    96
  print L
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    97
  print T
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    98
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
    99
Notice, that L and T now contain the first and second columns of data
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   100
from the data file, ``pendulum.txt``, and they are both simple
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   101
sequences.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   102
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   103
{{{ show the slide with loadtxt --- other features }}}
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   104
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   105
In this tutorial, we have learnt the basic use of the ``loadtxt``
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   106
command, which is capable of doing a lot more than we have used it for
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   107
until now, for example
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   108
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   109
%%2%% Pause the video here, and read the file
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   110
``pendulum_semicolon.txt`` which contains the same data as
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   111
``pendulum.txt``, but the columns are separated by semi-colons instead
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   112
of spaces. Use the IPython help to see how to do this. Once you have
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   113
finished, resume the video to look at the solution.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   114
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   115
{{{ switch back to the terminal }}}
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   116
::
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   117
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   118
  L, T = loadtxt('/home/fossee/pendulum.txt', unpack``True, delimiter``';')
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   119
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   120
  print L
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   121
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   122
  print T
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   123
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   124
This brings us to the end of this tutorial. 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   125
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   126
{{{ show the summary slide }}}
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   127
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   128
You should now be able to do the following, comfortably. 
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   129
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   130
  + Read data from files, containing a single column of data using the
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   131
    ``loadtxt`` command.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   132
  + Read multiple columns of data, separated by spaces or other
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   133
    delimiters.
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   134
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   135
Thank you!   
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   136
476ea1730aee Added rst files for scripts.
Puneeth Chaganti <punchagan@gmail.com>
parents:
diff changeset
   137