loading-data-from-files/script.rst
author Nishanth <nishanth@fossee.in>
Wed, 06 Oct 2010 16:15:42 +0530
changeset 221 7cd975ff5f0d
parent 217 b595f90016c5
child 247 81bb9a6d7eeb
permissions -rw-r--r--
Added questions
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
217
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     1
========
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     2
 Script
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     3
========
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     4
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     5
Welcome to this tutorial on loading data from files. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     6
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     7
{{{ Screen shows welcome slide }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     8
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
     9
We often require to plot points obtained from experimental
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    10
observations. In this tutorial we shall learn to read data from files
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    11
and save it into sequences that can later be used to plot.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    12
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    13
{{{ Show the outline for this tutorial }}} 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    14
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    15
We shall use the ``loadtxt`` command to load data from files. We will
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    16
be looking at how to read a file with multiple columns of data and
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    17
load each column of data into a sequence. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    18
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    19
{{{ switch back to the terminal }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    20
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    21
As usual, let us start IPython, using 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    22
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    23
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    24
  ipython -pylab 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    25
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    26
Now, Let us begin with reading the file primes.txt, which contains
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    27
just a list of primes listed in a column, using the loadtxt command.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    28
The file, in our case, is present in ``/home/fossee/primes.txt``. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    29
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    30
{{{ Navigate to the path in the OS, open the file and show it }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    31
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    32
.. #[punch: do we need a slide for showing the path?]
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    33
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    34
.. We use the ``cat`` command to see the contents of this file. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    35
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    36
.. #[punch: should we show the cat command here? seems like a good place
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    37
   to do it] ::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    38
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    39
     cat /home/fossee/primes.txt
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    40
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    41
.. #[Nishanth]: A problem for windows users.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    42
                Should we simply open the file and show them the data
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    43
                so that we can be fine with GNU/Linux ;) and windows?
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    44
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    45
Now let us read this list into the variable ``primes``.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    46
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    47
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    48
  primes = loadtxt('/home/fossee/primes.txt')
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    49
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    50
``primes`` is now a sequence of primes, that was listed in the file,
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    51
``primes.txt``.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    52
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    53
We now type, ``print primes`` to see the sequence printed.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    54
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    55
We observe that all of the numbers end with a period. This is so,
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    56
because these numbers are actually read as ``floats``. We shall learn
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    57
about them, later.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    58
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    59
Now, let us use the ``loadtxt`` command to read a file that contains
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    60
two columns of data, ``pendulum.txt``. This file contains the length
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    61
of the pendulum in the first column and the corresponding time period
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    62
in the second.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    63
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    64
%%1%% Pause the video here, and use the ``cat`` command to view the
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    65
contents of this file and then resume the video.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    66
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    67
This is how we look at the contents of the file, ``pendulum.txt``
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    68
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    69
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    70
  cat /home/fossee/pendulum.txt
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    71
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    72
.. #[Nishanth]: The first column is L values and second is T values
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    73
                from a simle pelculum experiment.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    74
                Since you are using the variable names later in the
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    75
                script.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    76
                Not necessary but can be included also.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    77
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    78
Let us, now, read the data into the variable ``pend``. Again, it is
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    79
assumed that the file is in ``/home/fossee/``
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    80
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    81
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    82
  pend = loadtxt('/home/fossee/pendulum.txt')
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    83
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    84
Let us now print the variable ``pend`` and see what's in it. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    85
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    86
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    87
  print pend
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    88
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    89
Notice that ``pend`` is not a simple sequence like ``primes``. It has
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    90
two sequences, containing both the columns of the data file. Let us
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    91
use an additional argument of the ``loadtxt`` command, to read it into
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    92
two separate, simple sequences.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    93
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    94
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    95
  L, T = loadtxt('/home/fossee/pendulum.txt', unpack=True)
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    96
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    97
.. #[Nishanth]: It has a sequence of items in which each item contains
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    98
                two values. first is l and second is t
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
    99
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   100
Let us now, print the variables L and T, to see what they contain.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   101
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   102
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   103
  print L
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   104
  print T
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   105
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   106
.. #[Nishanth]: Stress on ``unpack=True`` ??
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   107
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   108
Notice, that L and T now contain the first and second columns of data
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   109
from the data file, ``pendulum.txt``, and they are both simple
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   110
sequences. ``unpack=True`` has given us the two columns in to two
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   111
separate sequences instead of one complex sequence. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   112
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   113
{{{ show the slide with loadtxt --- other features }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   114
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   115
In this tutorial, we have learnt the basic use of the ``loadtxt``
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   116
command, which is capable of doing a lot more than we have used it for
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   117
until now, for example
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   118
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   119
%%2%% Pause the video here, and read the file
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   120
``pendulum_semicolon.txt`` which contains the same data as
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   121
``pendulum.txt``, but the columns are separated by semi-colons instead
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   122
of spaces. Use the IPython help to see how to do this. Once you have
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   123
finished, resume the video to look at the solution.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   124
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   125
{{{ switch back to the terminal }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   126
::
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   127
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   128
  L, T = loadtxt('/home/fossee/pendulum_semicolon.txt', unpack=True, delimiter=';')
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   129
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   130
  print L
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   131
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   132
  print T
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   133
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   134
This brings us to the end of this tutorial. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   135
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   136
{{{ show the summary slide }}}
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   137
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   138
You should now be able to do the following, comfortably. 
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   139
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   140
  + Read data from files, containing a single column of data using the
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   141
    ``loadtxt`` command.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   142
  + Read multiple columns of data, separated by spaces or other
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   143
    delimiters.
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   144
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   145
Thank you!   
b595f90016c5 Changed structure of my scripts.
Puneeth Chaganti <punchagan@fossee.in>
parents:
diff changeset
   146