Integrated Development Environments

Python can be used in batch mode, but there are a number of excellent integrated development environments (IDEs) for Python, among the most used:

• JupyterLab / Jupyter Notebook
• Spyder
• PyCharm
• RStudio

JupyterLab installation (Windows)

• Download latest python release from Microsoft Store

• Install it the usual way

• Run system symbol

  • Install the python package manager pip:

python -m ensurepip –upgrade

• Install JupyterLab with pip:

pip install jupyterlab

• Set a new System environment variable to the path (adapt it to fit your particular installation):

C:\Users\dmori\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts

JupyterLab installation (Windows)

• Run jupyterlab from the system symbol:

jupyter lab

A new instance of the default browser will launch. If it returns an error, go to the system symbol and copy and paste the address that looks similar to:

http://localhost:8888/lab?token=e1c50f3b897c86536a6ecd7b7dcb148b5de74d007e22d83f

JupyterLab installation (MacOS)

• Install homebrew https://brew.sh/

• Install python and JupyterLab from brew using the Terminal app:

brew install python
python -m ensurepip --upgrade
brew install jupyterlab

JupyterLab installation (Debian based Linux)

• Install python and JupyterLab:

sudo apt install python3
sudo snap install jupyterlab-desktop --classic

JupyterLab

A Jupyter notebook is divided into individual, vertically arranged cells, which can be executed separately:

JupyterLab screenshot

Python evolution

Python timeline

Getting help

Information on Python objects can be obtained quickly in an interactive environment:

help(len)

Help on built-in function len in module builtins:

len(obj, /)
    Return the number of items in a container.

Basic programming

Programs can be implemented very quickly – this is a pretty minimal example. You can write this command to a text file of your choice and run it directly on your system:

print("Hello there!")

Hello there!

• Only one function print() (shown here as a keyword),

• Function displays argument (a string) on screen,

• Arguments are passed to the function in parentheses,

• A string must be wrapped in ” ” or ’ ’,

• No semicolon at the end.

Basic programming

Types of objects

• Numbers (integers, floating-point numbers, and complex numbers)

• Booleans

• The “null” type (NA in some other languages)

• Strings

• Lists

Basic programming

Operations

• Sum: x + y

• Difference: x - y

• Product: x * y

• Quotient: x / y

• Remainder: x % y

• Power: x ** y

• Absolute value: abs(x)

Basic programming

Comparison

• x == y

• x != y

• x > y / x >= y

• x < y / x <= y

• And: &

• Or: |

Basic programming

Importing additional packages

import pandas as pd

Importing only a function from a package

from datetime import datetime

Basic programming

Operations (extension)

import math

math.factorial(6)

720

Basic programming

Operations (extension)

import math

math.log(1)

0.0

Basic programming

Operations (extension)

import math

math.exp(1)

2.718281828459045

Basic programming

Comments in python (jupyterLab)

# This is
# a multiline comment

Basic programming

Declaring new variables

a = 2
vec = [] * 1000 # Array of size 1000

Basic programming

Changing the working directory

import os

os.getcwd()

'/home/dmorina/Insync/dmorina@ub.edu/OneDrive Biz/Docència/UAB/2024-2025/Aprenentatge Automàtic 1 (GEA)/Pràctiques/Pràctica 1'

import os

os.chdir('/home/dmorina/')
os.getcwd()

'/home/dmorina'

Basic programming

Defining (and using) new functions:

def newFunction(x, y):
  return x % y

newFunction(3, 2)

1

Basic programming

Defining (and using) new functions:

def newFunction2(x):
  if x > 5:
    return x+5
  elif x == 5:
    return x+10
  else:
    return x+100

newFunction2(3)

103

newFunction2(5)

15

newFunction2(20)

25

Basic programming

Defining (and using) new functions:

def newFunction3():
  for x in range(6):
    if x == 3: continue
    print(x)
  else:
    print("Finally finished!")

newFunction3()

0
1
2
4
5
Finally finished!

Basic programming

Defining (and using) new functions:

def newFunction4():
  for x in range(6):
    if x == 3: break
    print(x)
  else:
    print("Finally finished!")

newFunction4()

0
1
2

Basic programming

pandas import and export

Basic programming

Reading and basic work with data (pandas!)

import pandas as pd
import os

os.chdir("/home/dmorina/Insync/dmorina@ub.edu/OneDrive Biz/Docència/UAB/2024-2025/Aprenentatge Automàtic 1 (GEA)/Pràctiques/Pràctica 1/examples/")
newData = pd.read_csv("titanic.csv")

Basic programming

Reading and basic work with data (pandas!)

newData.head(2)

Basic programming

Reading and basic work with data (pandas!)

newData.tail(5)

Basic programming

Reading and basic work with data (pandas!)

newData.shape

(891, 12)

len(newData)

891

newData.size

10692

newData.ndim

2

newData.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   PassengerId  891 non-null    int64  
 1   Survived     891 non-null    int64  
 2   Pclass       891 non-null    int64  
 3   Name         891 non-null    object 
 4   Sex          891 non-null    object 
 5   Age          714 non-null    float64
 6   SibSp        891 non-null    int64  
 7   Parch        891 non-null    int64  
 8   Ticket       891 non-null    object 
 9   Fare         891 non-null    float64
 10  Cabin        204 non-null    object 
 11  Embarked     889 non-null    object 
dtypes: float64(2), int64(5), object(5)
memory usage: 83.7+ KB

Basic programming

Reading and basic work with data (pandas!)

newData.count()

PassengerId    891
Survived       891
Pclass         891
Name           891
Sex            891
Age            714
SibSp          891
Parch          891
Ticket         891
Fare           891
Cabin          204
Embarked       889
dtype: int64

newData['Age'].count()

np.int64(714)

Basic programming

Reading and basic work with data (pandas!)

newData.describe()

Basic programming

Reading and basic work with data (pandas!)

newData.groupby(["Sex", "Pclass"])["Fare"].describe()

Basic programming

Selecting rows

newData.iloc[:3]

Basic programming

Selecting rows (conditionally)

newData.query('Age>40 & Sex=="female"').head(2)

Basic programming

Selecting rows (conditionally)

newData[(newData.Age > 40) & (newData.Sex == "female")].head(2)

Basic programming

Selecting rows (randomly)

newData.sample(n=2)

newData.sample(frac=0.001)

Basic programming

Selecting columns

newData[['Age', 'Sex']].head(2)

Basic programming

Selecting columns

newData.loc[:, 'Age':'Ticket'].head(2)

Basic programming

Selecting columns

newData[['Age', 'Sex']].head(2)

Basic programming

Rename columns

newData.rename(columns={'Age': 'age'}).head(3)

Basic programming

Drop columns

newData.drop(['Age', 'Sex'], axis=1).head(2)

Basic programming

Drop duplicates

newData.drop_duplicates().head(3)

Basic programming

Create a new column

newData["AgeGroup"] = pd.cut(newData.Age, range(0, 105, 10), right=False)
newData[['Age', 'AgeGroup']].head(8)

Basic programming

Join DataFrames vertically

less50 = newData[newData.Age <= 50]
over50 = newData[newData.Age > 50]
total = pd.concat([less50, over50])
total.head(2)

Basic programming

Join DataFrames horizontally

df1 = pd.DataFrame({
    'A': [1,2,3,4,5],
    'B': [1,2,3,4,5]
})

df2 = pd.DataFrame({
    'C': [1,2,3,4,5],
    'D': [1,2,3,4,5]
})

df_concat = pd.concat([df1, df2], axis=1)
df_concat.head(2)

Basic programming

Merge DataFrames

df1 = pd.DataFrame({
    'id': [1,2,3,4,5],
    'col1': [1,2,3,4,5]
})

df2 = pd.DataFrame({
    'id': [1,2,3,4,5],
    'col2': [6,7,8,9,10]
})

df_merge = df1.merge(df2, on='id')
df_merge.head(2)

Basic programming

Exporting data (pandas!)

import pandas as pd
newData.to_excel("titanic.xlsx", sheet_name="passengers", index=False)

Basic programming

Generating basic graphs with pandas

import matplotlib.pyplot as plt

newData.plot()
plt.show()

Basic programming

newData.plot.scatter(x="Age", y="Fare", alpha=0.5)
plt.show()

Basic programming

newData.plot.box(y="Fare")
plt.show()

Basic programming

newData.plot.hist(y="Fare")
plt.show()

Basic programming

newData2 = newData.groupby(['Sex','AgeGroup']).size()
newData2 = newData2.unstack()
newData2.plot.bar()
plt.show()

Basic programming

newData3 = newData.groupby(['AgeGroup']).size()
newData3.plot.pie(y="AgeGroup", title="Age group", legend=False,
                   autopct='%1.1f%%', 
                   shadow=True, startangle=0)

<Axes: title={'center': 'Age group'}>

Basic programming

Remove objects

del [[newData, newData2, newData3]]

Basic programming

More information on pandas: https://pandas.pydata.org

Exercise 1

• Download all csv files in the practice/data folder and define the proper working directory

• Import all the files to the JupyterLab session

Exercise 2

• Combine all the files in a single DataFrame

• Sort the resulting file by date (‘fecha Siniestro Acto’)

• Rename the column fecha Siniestro Acto to Date

Exercise 3

• Group by week number (Hint: Use first to_datetime function from pandas library to make the column datetime and then isocalendar().week from datatable library) and sum the values of ‘Unidades Acto’.

Exercise 4

• Plot the evolution of the variable ‘Unidades Acto’.

Exercise 5

• Export the final DataFrame to a csv file