[NP1] - A short introduction to Numpy¶
Numpy is an essential tool for the Scientific Python.Objectives :¶
- Understand the main principles of Numpy and its potential
Note : This notebook is strongly inspired by the UGA Python Introduction Course
See : https://gricad-gitlab.univ-grenoble-alpes.fr/python-uga/py-training-2017
Step 1 - Numpy the beginning¶
Code using numpy usually starts with the import statement
In [1]:
import numpy as np
NumPy provides the type np.ndarray. Such array are multidimensionnal sequences of homogeneous elements. They can be created for example with the commands:
In [2]:
# from a list
l = [10.0, 12.5, 15.0, 17.5, 20.0]
np.array(l)
Out[2]:
array([10. , 12.5, 15. , 17.5, 20. ])
In [3]:
# fast but the values can be anything
np.empty(4)
Out[3]:
array([4.66336529e-310, 0.00000000e+000, 4.66336497e-310, 4.66336497e-310])
In [4]:
# slower than np.empty but the values are all 0.
np.zeros([2, 6])
Out[4]:
array([[0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0.]])
In [5]:
# multidimensional array
a = np.ones([2, 3, 4])
print(a.shape, a.size, a.dtype)
a
(2, 3, 4) 24 float64
Out[5]:
array([[[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]],
[[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]]])
In [6]:
# like range but produce 1D numpy array
np.arange(4)
Out[6]:
array([0, 1, 2, 3])
In [7]:
# np.arange can produce arrays of floats
np.arange(4.)
Out[7]:
array([0., 1., 2., 3.])
In [8]:
# another convenient function to generate 1D arrays
np.linspace(10, 20, 5)
Out[8]:
array([10. , 12.5, 15. , 17.5, 20. ])
A NumPy array can be easily converted to a Python list.
In [9]:
a = np.linspace(10, 20 ,5)
list(a)
Out[9]:
[10.0, 12.5, 15.0, 17.5, 20.0]
In [10]:
# Or even better
a.tolist()
Out[10]:
[10.0, 12.5, 15.0, 17.5, 20.0]
In [11]:
A = np.random.random([4, 5])
A
Out[11]:
array([[0.17802562, 0.70291548, 0.3972505 , 0.60066953, 0.32432842],
[0.0420063 , 0.48224088, 0.52356364, 0.44738872, 0.76419371],
[0.32352191, 0.44416695, 0.33902905, 0.43158284, 0.10374585],
[0.00147743, 0.46039124, 0.53495914, 0.94692235, 0.87335917]])
In [12]:
# Get the element from second line, first column
A[1, 0]
Out[12]:
0.0420062976455734
In [13]:
# Get the first two lines
A[:2]
Out[13]:
array([[0.17802562, 0.70291548, 0.3972505 , 0.60066953, 0.32432842],
[0.0420063 , 0.48224088, 0.52356364, 0.44738872, 0.76419371]])
In [14]:
# Get the last column
A[:, -1]
Out[14]:
array([0.32432842, 0.76419371, 0.10374585, 0.87335917])
In [15]:
# Get the first two lines and the columns with an even index
A[:2, ::2]
Out[15]:
array([[0.17802562, 0.3972505 , 0.32432842],
[0.0420063 , 0.52356364, 0.76419371]])
2.2 - Using a mask to select elements validating a condition:¶
In [16]:
cond = A > 0.5
print(cond)
print(A[cond])
[[False True False True False] [False False True False True] [False False False False False] [False False True True True]] [0.70291548 0.60066953 0.52356364 0.76419371 0.53495914 0.94692235 0.87335917]
The mask is in fact a particular case of the advanced indexing capabilities provided by NumPy. For example, it is even possible to use lists for indexing:
In [17]:
# Selecting only particular columns
print(A)
A[:, [0, 1, 4]]
[[0.17802562 0.70291548 0.3972505 0.60066953 0.32432842] [0.0420063 0.48224088 0.52356364 0.44738872 0.76419371] [0.32352191 0.44416695 0.33902905 0.43158284 0.10374585] [0.00147743 0.46039124 0.53495914 0.94692235 0.87335917]]
Out[17]:
array([[0.17802562, 0.70291548, 0.32432842],
[0.0420063 , 0.48224088, 0.76419371],
[0.32352191, 0.44416695, 0.10374585],
[0.00147743, 0.46039124, 0.87335917]])
In [18]:
(A+5)**2
Out[18]:
array([[26.81194936, 32.52324502, 29.13031293, 31.36749914, 28.34847309],
[25.42182751, 30.05496505, 30.50975529, 29.67404386, 33.2259291 ],
[28.33988552, 29.63895375, 28.50523119, 29.50209211, 26.04822174],
[25.01477653, 29.81587251, 30.6357727 , 35.36588547, 34.49634793]])
3.2 - Apply functions element-wise:¶
In [19]:
np.exp(A) # With numpy arrays, use the functions from numpy !
Out[19]:
array([[1.19485594, 2.01963234, 1.48772856, 1.82333916, 1.38310147],
[1.04290105, 1.61969989, 1.68803248, 1.56422223, 2.14726235],
[1.38198643, 1.55919077, 1.40358412, 1.53969268, 1.10931849],
[1.00147853, 1.58469386, 1.70737848, 2.57776399, 2.39494237]])
3.3 - Setting parts of arrays¶
In [20]:
A[:, 0] = 0.
print(A)
[[0. 0.70291548 0.3972505 0.60066953 0.32432842] [0. 0.48224088 0.52356364 0.44738872 0.76419371] [0. 0.44416695 0.33902905 0.43158284 0.10374585] [0. 0.46039124 0.53495914 0.94692235 0.87335917]]
In [21]:
# BONUS: Safe element-wise inverse with masks
cond = (A != 0)
A[cond] = 1./A[cond]
print(A)
[[0. 1.42264614 2.51730333 1.66480895 3.08329443] [0. 2.07365249 1.90998748 2.2351927 1.30856874] [0. 2.25140571 2.94959975 2.31705229 9.63893941] [0. 2.17206565 1.86930164 1.0560528 1.1450043 ]]
In [22]:
for i,v in enumerate([s for s in dir(A) if not s.startswith('__')]):
print(f'{v:16}', end='')
if (i+1) % 6 == 0 :print('')
T all any argmax argmin argpartition argsort astype base byteswap choose clip compress conj conjugate copy ctypes cumprod cumsum data diagonal dot dtype dump dumps fill flags flat flatten getfield imag item itemset itemsize max mean min nbytes ndim newbyteorder nonzero partition prod ptp put ravel real repeat reshape resize round searchsorted setfield setflags shape size sort squeeze std strides sum swapaxes take tobytes tofile tolist tostring trace transpose var view
In [23]:
# Ex1: Get the mean through different dimensions
print(A)
print('Mean value', A.mean())
print('Mean line', A.mean(axis=0))
print('Mean column', A.mean(axis=1))
[[0. 1.42264614 2.51730333 1.66480895 3.08329443] [0. 2.07365249 1.90998748 2.2351927 1.30856874] [0. 2.25140571 2.94959975 2.31705229 9.63893941] [0. 2.17206565 1.86930164 1.0560528 1.1450043 ]] Mean value 1.9807437896735376 Mean line [0. 1.9799425 2.31154805 1.81827668 3.79395172] Mean column [1.73761057 1.50548028 3.43139943 1.24848488]
In [24]:
# Ex2: Convert a 2D array in 1D keeping all elements
print(A)
print(A.shape)
A_flat = A.flatten()
print(A_flat, A_flat.shape)
[[0. 1.42264614 2.51730333 1.66480895 3.08329443] [0. 2.07365249 1.90998748 2.2351927 1.30856874] [0. 2.25140571 2.94959975 2.31705229 9.63893941] [0. 2.17206565 1.86930164 1.0560528 1.1450043 ]] (4, 5) [0. 1.42264614 2.51730333 1.66480895 3.08329443 0. 2.07365249 1.90998748 2.2351927 1.30856874 0. 2.25140571 2.94959975 2.31705229 9.63893941 0. 2.17206565 1.86930164 1.0560528 1.1450043 ] (20,)
4.1 - Remark: dot product¶
In [25]:
b = np.linspace(0, 10, 11)
c = b @ b
# before 3.5:
# c = b.dot(b)
print(b)
print(c)
[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] 385.0
4.2 - For Matlab users¶
|
Matlab | Numpy |
|---|---|---|
| element wise | .* |
* |
| dot product | * |
@ |
numpy arrays can also be sorted, even when they are composed of complex data if the type of the columns are explicitly stated with dtypes.
4.3 - NumPy and SciPy sub-packages:¶
We already saw numpy.random to generate numpy arrays filled with random values. This submodule also provides functions related to distributions (Poisson, gaussian, etc.) and permutations.
To perform linear algebra with dense matrices, we can use the submodule numpy.linalg. For instance, in order to compute the determinant of a random matrix, we use the method det
In [26]:
A = np.random.random([5,5])
print(A)
np.linalg.det(A)
[[0.79771765 0.8894402 0.01987013 0.26252542 0.97374149] [0.82654597 0.20429293 0.96590728 0.02768128 0.41583749] [0.1072818 0.75375777 0.71860969 0.10434004 0.34100039] [0.01659443 0.87460735 0.73796852 0.39153514 0.96417177] [0.21007846 0.35430059 0.61584624 0.00640809 0.87444986]]
Out[26]:
-0.17918698076450318
In [27]:
squared_subA = A[1:3, 1:3]
print(squared_subA)
np.linalg.inv(squared_subA)
[[0.20429293 0.96590728] [0.75375777 0.71860969]]
Out[27]:
array([[-1.23631087, 1.66176671],
[ 1.29678034, -0.35146975]])
4.4 - Introduction to Pandas: Python Data Analysis Library¶
Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for Python.
Pandas tutorial Grenoble Python Working Session Pandas for SQL Users Pandas Introduction Training HPC Python@UGA
