No description has been provided for this image

[NP1] - A short introduction to Numpy¶

Numpy is an essential tool for the Scientific Python.

Objectives :¶

  • Understand the main principles of Numpy and its potential

Note : This notebook is strongly inspired by the UGA Python Introduction Course
See : https://gricad-gitlab.univ-grenoble-alpes.fr/python-uga/py-training-2017

Step 1 - Numpy the beginning¶

Code using numpy usually starts with the import statement

In [1]:
import numpy as np

NumPy provides the type np.ndarray. Such array are multidimensionnal sequences of homogeneous elements. They can be created for example with the commands:

In [2]:
# from a list
l = [10.0, 12.5, 15.0, 17.5, 20.0]
np.array(l)
Out[2]:
array([10. , 12.5, 15. , 17.5, 20. ])
In [3]:
# fast but the values can be anything
np.empty(4)
Out[3]:
array([4.67030424e-310, 0.00000000e+000, 4.67101322e-310, 4.67101319e-310])
In [4]:
# slower than np.empty but the values are all 0.
np.zeros([2, 6])
Out[4]:
array([[0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0.]])
In [5]:
# multidimensional array
a = np.ones([2, 3, 4])
print(a.shape, a.size, a.dtype)
a

(2, 3, 4) 24 float64
Out[5]:
array([[[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]],

       [[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]]])
In [6]:
# like range but produce 1D numpy array
np.arange(4)
Out[6]:
array([0, 1, 2, 3])
In [7]:
# np.arange can produce arrays of floats
np.arange(4.)
Out[7]:
array([0., 1., 2., 3.])
In [8]:
# another convenient function to generate 1D arrays
np.linspace(10, 20, 5)
Out[8]:
array([10. , 12.5, 15. , 17.5, 20. ])

A NumPy array can be easily converted to a Python list.

In [9]:
a = np.linspace(10, 20 ,5)
list(a)
Out[9]:
[np.float64(10.0),
 np.float64(12.5),
 np.float64(15.0),
 np.float64(17.5),
 np.float64(20.0)]
In [10]:
# Or even better
a.tolist()
Out[10]:
[10.0, 12.5, 15.0, 17.5, 20.0]

Step 2 - Access elements¶

Elements in a numpy array can be accessed using indexing and slicing in any dimension. It also offers the same functionalities available in Fortan or Matlab.

2.1 - Indexes and slices¶

For example, we can create an array A and perform any kind of selection operations on it.

In [11]:
A = np.random.random([4, 5])
A
Out[11]:
array([[0.20518501, 0.58875353, 0.45166626, 0.09836971, 0.07648559],
       [0.39507445, 0.49513016, 0.23119691, 0.70292165, 0.36082793],
       [0.23642025, 0.15254409, 0.93467091, 0.60955493, 0.04942166],
       [0.66744393, 0.18057559, 0.67351066, 0.24477554, 0.83200592]])
In [12]:
# Get the element from second line, first column
A[1, 0]
Out[12]:
np.float64(0.39507445278829356)
In [13]:
# Get the first two lines
A[:2]
Out[13]:
array([[0.20518501, 0.58875353, 0.45166626, 0.09836971, 0.07648559],
       [0.39507445, 0.49513016, 0.23119691, 0.70292165, 0.36082793]])
In [14]:
# Get the last column
A[:, -1]
Out[14]:
array([0.07648559, 0.36082793, 0.04942166, 0.83200592])
In [15]:
# Get the first two lines and the columns with an even index
A[:2, ::2]
Out[15]:
array([[0.20518501, 0.45166626, 0.07648559],
       [0.39507445, 0.23119691, 0.36082793]])

2.2 - Using a mask to select elements validating a condition:¶

In [16]:
cond = A > 0.5
print(cond)
print(A[cond])

[[False  True False False False]
 [False False False  True False]
 [False False  True  True False]
 [ True False  True False  True]]
[0.58875353 0.70292165 0.93467091 0.60955493 0.66744393 0.67351066
 0.83200592]

The mask is in fact a particular case of the advanced indexing capabilities provided by NumPy. For example, it is even possible to use lists for indexing:

In [17]:
# Selecting only particular columns
print(A)
A[:, [0, 1, 4]]

[[0.20518501 0.58875353 0.45166626 0.09836971 0.07648559]
 [0.39507445 0.49513016 0.23119691 0.70292165 0.36082793]
 [0.23642025 0.15254409 0.93467091 0.60955493 0.04942166]
 [0.66744393 0.18057559 0.67351066 0.24477554 0.83200592]]
Out[17]:
array([[0.20518501, 0.58875353, 0.07648559],
       [0.39507445, 0.49513016, 0.36082793],
       [0.23642025, 0.15254409, 0.04942166],
       [0.66744393, 0.18057559, 0.83200592]])

Step 3 - Perform array manipulations¶

3.1 - Apply arithmetic operations to whole arrays (element-wise):¶

In [18]:
(A+5)**2
Out[18]:
array([[27.09395103, 31.23416606, 29.72066504, 25.99337373, 25.77070593],
       [29.10682835, 30.19645549, 27.36542108, 32.52331535, 28.73847609],
       [27.42009709, 26.54871055, 35.22031879, 31.46710656, 25.49665909],
       [32.11992075, 26.83836348, 32.1887232 , 27.50767046, 34.01229307]])

3.2 - Apply functions element-wise:¶

In [19]:
np.exp(A) # With numpy arrays, use the functions from numpy !
Out[19]:
array([[1.2277522 , 1.8017412 , 1.57092758, 1.10337064, 1.07948663],
       [1.48449471, 1.64071178, 1.26010734, 2.01964479, 1.4345166 ],
       [1.26670654, 1.16479381, 2.54637533, 1.83961247, 1.05066328],
       [1.94924854, 1.19790667, 1.96111004, 1.27733457, 2.29792358]])

3.3 - Setting parts of arrays¶

In [20]:
A[:, 0] = 0.
print(A)

[[0.         0.58875353 0.45166626 0.09836971 0.07648559]
 [0.         0.49513016 0.23119691 0.70292165 0.36082793]
 [0.         0.15254409 0.93467091 0.60955493 0.04942166]
 [0.         0.18057559 0.67351066 0.24477554 0.83200592]]
In [21]:
# BONUS: Safe element-wise inverse with masks
cond = (A != 0)
A[cond] = 1./A[cond]
print(A)

[[ 0.          1.69850361  2.21402412 10.16573055 13.07435843]
 [ 0.          2.01967094  4.32531738  1.42263366  2.77140409]
 [ 0.          6.55548197  1.06989529  1.64054123 20.23404359]
 [ 0.          5.53784695  1.4847575   4.08537554  1.20191452]]

Step 4 - Attributes and methods of np.ndarray (see the doc)¶

In [22]:
for i,v in enumerate([s for s in dir(A) if not s.startswith('__')]):
    print(f'{v:16}', end='')
    if (i+1) % 6 == 0 :print('')

T               all             any             argmax          argmin          argpartition    
argsort         astype          base            byteswap        choose          clip            
compress        conj            conjugate       copy            ctypes          cumprod         
cumsum          data            device          diagonal        dot             dtype           
dump            dumps           fill            flags           flat            flatten         
getfield        imag            item            itemset         itemsize        mT              
max             mean            min             nbytes          ndim            newbyteorder    
nonzero         partition       prod            ptp             put             ravel           
real            repeat          reshape         resize          round           searchsorted    
setfield        setflags        shape           size            sort            squeeze         
std             strides         sum             swapaxes        take            to_device       
tobytes         tofile          tolist          tostring        trace           transpose       
var             view
In [23]:
# Ex1: Get the mean through different dimensions

print(A)
print('Mean value',  A.mean())
print('Mean line',   A.mean(axis=0))
print('Mean column', A.mean(axis=1))

[[ 0.          1.69850361  2.21402412 10.16573055 13.07435843]
 [ 0.          2.01967094  4.32531738  1.42263366  2.77140409]
 [ 0.          6.55548197  1.06989529  1.64054123 20.23404359]
 [ 0.          5.53784695  1.4847575   4.08537554  1.20191452]]
Mean value 3.9750749687241567
Mean line [0.         3.95287587 2.27349857 4.32857024 9.32043016]
Mean column [5.43052334 2.10780522 5.89999241 2.4619789 ]
In [24]:
# Ex2: Convert a 2D array in 1D keeping all elements

print(A)
print(A.shape)
A_flat = A.flatten()
print(A_flat, A_flat.shape)

[[ 0.          1.69850361  2.21402412 10.16573055 13.07435843]
 [ 0.          2.01967094  4.32531738  1.42263366  2.77140409]
 [ 0.          6.55548197  1.06989529  1.64054123 20.23404359]
 [ 0.          5.53784695  1.4847575   4.08537554  1.20191452]]
(4, 5)
[ 0.          1.69850361  2.21402412 10.16573055 13.07435843  0.
  2.01967094  4.32531738  1.42263366  2.77140409  0.          6.55548197
  1.06989529  1.64054123 20.23404359  0.          5.53784695  1.4847575
  4.08537554  1.20191452] (20,)

4.1 - Remark: dot product¶

In [25]:
b = np.linspace(0, 10, 11)
c = b @ b
# before 3.5:
# c = b.dot(b)
print(b)
print(c)

[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
385.0

4.2 - For Matlab users¶

Matlab Numpy
element wise .* *
dot product * @

numpy arrays can also be sorted, even when they are composed of complex data if the type of the columns are explicitly stated with dtypes.

4.3 - NumPy and SciPy sub-packages:¶

We already saw numpy.random to generate numpy arrays filled with random values. This submodule also provides functions related to distributions (Poisson, gaussian, etc.) and permutations.

To perform linear algebra with dense matrices, we can use the submodule numpy.linalg. For instance, in order to compute the determinant of a random matrix, we use the method det

In [26]:
A = np.random.random([5,5])
print(A)
np.linalg.det(A)

[[0.32600516 0.28206657 0.26282316 0.770393   0.3431083 ]
 [0.16533586 0.44123925 0.19849497 0.73062533 0.68488145]
 [0.33187007 0.00982615 0.8793322  0.67263088 0.86575286]
 [0.22878297 0.18961378 0.39589715 0.20554336 0.37801677]
 [0.29081857 0.31814718 0.22892718 0.08828156 0.43990676]]
Out[26]:
np.float64(-0.008387013996040981)
In [27]:
squared_subA = A[1:3, 1:3]
print(squared_subA)
np.linalg.inv(squared_subA)

[[0.44123925 0.19849497]
 [0.00982615 0.8793322 ]]
Out[27]:
array([[ 2.27779454, -0.51417516],
       [-0.02545335,  1.14297232]])

4.4 - Introduction to Pandas: Python Data Analysis Library¶

Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for Python.

Pandas tutorial Grenoble Python Working Session Pandas for SQL Users Pandas Introduction Training HPC Python@UGA

No description has been provided for this image