Pandas is a powerful open-source data manipulation and analysis library for Python. It provides easy-to-use data structures and data analysis tools for handling and analyzing structured data. Pandas is built on top of NumPy, another popular library for scientific computing with Python.<\/p>\n
In this tutorial, we will learn how to use Pandas for data analysis in Python. We will cover the following topics:<\/p>\n
By the end of this tutorial, you will have a good understanding of how to use Pandas to analyze and manipulate data in Python.<\/p>\n
Before we can start using Pandas, we need to install it. Fortunately, installing Pandas is easy using either To install Pandas using If you are using Anaconda, you can install Pandas using Make sure you have a working Python installation before installing Pandas.<\/p>\n Once you have installed Pandas, you can import it into your Python script or Jupyter Notebook by adding the following line at the beginning:<\/p>\n This line imports the Pandas library and assigns it the alias Now we are ready to start using Pandas!<\/p>\n DataFrames are the central data structure in Pandas. They are similar to the tables in a relational database or a spreadsheet in Excel. A DataFrame is a two-dimensional labeled data structure with columns of potentially different types.<\/p>\n There are several ways to create a DataFrame in Pandas.<\/p>\n You can create a DataFrame from a list of lists or a list of dictionaries. Each inner list or dictionary represents a row in the DataFrame, and the columns are inferred from the data.<\/p>\n For example, let’s create a DataFrame from a list of lists:<\/p>\n In this example, our list of lists contains three rows, and each row has three values representing the name, age, and country of a person. We also specify the column names explicitly by passing a list of strings to the You can also create a DataFrame from a dictionary where each key-value pair represents a column in the DataFrame.<\/p>\n For example, let’s create a DataFrame from a dictionary:<\/p>\n In this example, our dictionary has three keys corresponding to the column names, and each value is a list representing the data in that column. The column names are inferred from the keys of the dictionary.<\/p>\n You can also create an empty DataFrame and then populate it with data later.<\/p>\n For example, let’s create an empty DataFrame and add data to it:<\/p>\n In this example, we first create an empty DataFrame with the specified column names. Then, we use the Pandas provides various methods for loading data from different file formats into a DataFrame, such as CSV, Excel, SQL databases, JSON, and more.<\/p>\n To load data from a CSV file into a DataFrame, you can use the For example, let’s load a CSV file named “data.csv” into a DataFrame:<\/p>\n In this example, we assume that the CSV file is in the same directory as our Python script or Jupyter Notebook. If the file is in a different directory, you need to provide the full path to the file.<\/p>\n By default, You can also specify additional parameters for handling missing values, converting data types, skipping rows or columns, selecting specific columns, and more. Refer to the Pandas documentation for more details on the available parameters.<\/p>\n To load data from an Excel file into a DataFrame, you can use the For example, let’s load an Excel file named “data.xlsx” into a DataFrame:<\/p>\n By default, You can also specify additional parameters for handling missing values, converting data types, selecting specific rows or columns, and more. Refer to the Pandas documentation for more details on the available parameters.<\/p>\n Once you have loaded your data into a DataFrame, you can start exploring and analyzing it using various methods and properties provided by Pandas.<\/p>\n To view the first few rows of a DataFrame, you can use the To view the last few rows of a DataFrame, you can use the You can access individual columns of a DataFrame using the column names as attributes. This returns a Pandas Series object, which is a one-dimensional labeled array.<\/p>\n Alternatively, you can use the You can access individual rows of a DataFrame using the You can also access multiple rows by specifying a range of labels or indices:<\/p>\n You can access subsets of data in a DataFrame by specifying both the rows and columns using the Pandas provides a variety of methods to calculate summary statistics of numerical columns in a DataFrame.<\/p>\n For example, you can use the You can use the You can use the You can use the You can use the Pandas provides various methods and functions for manipulating data in a DataFrame.<\/p>\n To add a new column to a DataFrame, you can assign a new Series object to a new column name:<\/p>\n In this example, we assign a new Series to the column ‘Salary’ with three values.<\/p>\n Alternatively, you can use the In this example, we insert a new column named ‘Salary’ at position 1 (after the first column).<\/p>\n To update values in a DataFrame, you can use boolean indexing to select the rows and columns, and then assign new values to them.<\/p>\n For example, let’s update the ‘Salary’ of the first row:<\/p>\n In this example, we use To filter rows based on a condition, you can use boolean indexing.<\/p>\n For example, let’s filter the rows where the ‘Age’ is greater than or equal to 30:<\/p>\n In this example, we use boolean indexing to select the rows where the ‘Age’ is greater than or equal to 30. The resulting DataFrame contains only the selected rows.<\/p>\n To sort a DataFrame by one or more columns, you can use the For example, let’s sort the DataFrame by the ‘Age’ column in descending order:<\/p>\n In this example, we use To remove rows or columns from a DataFrame, you can use the For example, let’s remove the ‘Salary’ column:<\/p>\n In this example, we use To remove rows, you can specify the row index(es) instead of the column name.<\/p>\n Pandas provides various methods and functions for aggregating data in a DataFrame.<\/p>\n To group data in a DataFrame by one or more columns and calculate aggregate functions for each group, you can use the For example, let’s group the data by the ‘Country’ column and calculate the average ‘Age’ for each country:<\/p>\n In this example, we use To create a pivot table from a DataFrame, you can use the For example, let’s create a pivot table that shows the average ‘Age’ for each combination of ‘Country’ and ‘Gender’:<\/p>\n In this example, we specify the DataFrame, the values to aggregate (‘Age’), the index (‘Country’), the columns (‘Gender’), and the aggregate function (‘mean’).<\/p>\n To reshape data in a DataFrame, you can use various methods such as For example, let’s melt the DataFrame to convert it from wide to long format:<\/p>\n In this example, we specify the DataFrame, the identifier variable (‘Name’), the variables to melt (‘Age’ and ‘Salary’), the variable name (‘Variable’), and the value name (‘Value’).<\/p>\n Pandas provides basic data visualization capabilities that are built on top of the Matplotlib library.<\/p>\n To create a plot in Pandas, you can use the To create a line plot, you can call the For example, let’s create a line plot of the ‘Age’ column:<\/p>\n To create a bar plot, you can call the For example, let’s create a bar plot of the average ‘Salary’ for each ‘Country’:<\/p>\n In this example, we first group the data by ‘Country’ and calculate the average ‘Salary’ for each group. Then, we create a bar plot from the resulting Series.<\/p>\n To create a histogram, you can call the For example, let’s create a histogram of the ‘Age’ column:<\/p>\n To create a scatter plot, you can call the For example, let’s create a scatter plot of the ‘Age’ versus ‘Salary’ columns:<\/p>\n In this example, we specify the ‘Age’ column as the x-axis and the ‘Salary’ column as the y-axis.<\/p>\npip<\/code> or conda<\/code>.<\/p>\npip<\/code>, run the following command in your terminal:<\/p>\npip install pandas\n<\/code><\/pre>\nconda<\/code>. Run the following command in your terminal:<\/p>\nconda install pandas\n<\/code><\/pre>\n2. Importing Pandas<\/h2>\n
import pandas as pd\n<\/code><\/pre>\npd<\/code>, which is a common convention in the Python data science community.<\/p>\n3. Creating Pandas DataFrames<\/h2>\n
3.1 Creating a DataFrame from a List<\/h3>\n
data = [\n ['John', 25, 'USA'],\n ['Alice', 28, 'Canada'],\n ['Bob', 32, 'UK']\n]\n\ndf = pd.DataFrame(data, columns=['Name', 'Age', 'Country'])\n<\/code><\/pre>\ncolumns<\/code> parameter of the pd.DataFrame()<\/code> function.<\/p>\n3.2 Creating a DataFrame from a Dictionary<\/h3>\n
data = {\n 'Name': ['John', 'Alice', 'Bob'],\n 'Age': [25, 28, 32],\n 'Country': ['USA', 'Canada', 'UK']\n}\n\ndf = pd.DataFrame(data)\n<\/code><\/pre>\n3.3 Creating an Empty DataFrame<\/h3>\n
df = pd.DataFrame(columns=['Name', 'Age', 'Country'])\n\ndf = df.append({'Name': 'John', 'Age': 25, 'Country': 'USA'}, ignore_index=True)\ndf = df.append({'Name': 'Alice', 'Age': 28, 'Country': 'Canada'}, ignore_index=True)\ndf = df.append({'Name': 'Bob', 'Age': 32, 'Country': 'UK'}, ignore_index=True)\n<\/code><\/pre>\nappend()<\/code> method to add rows to the DataFrame. The ignore_index=True<\/code> parameter ensures that the index of the added rows is reset.<\/p>\n4. Loading Data into a DataFrame<\/h2>\n
4.1 Loading Data from a CSV File<\/h3>\n
pd.read_csv()<\/code> function.<\/p>\ndf = pd.read_csv('data.csv')\n<\/code><\/pre>\npd.read_csv()<\/code> assumes that the CSV file has a header row containing the column names. If your CSV file does not have a header row, you can specify it using the header<\/code> parameter:<\/p>\ndf = pd.read_csv('data.csv', header=None)\n<\/code><\/pre>\n4.2 Loading Data from an Excel File<\/h3>\n
pd.read_excel()<\/code> function.<\/p>\ndf = pd.read_excel('data.xlsx')\n<\/code><\/pre>\npd.read_excel()<\/code> assumes that the first sheet of the Excel file contains the data. If your data is in a different sheet, you can specify it using the sheet_name<\/code> parameter:<\/p>\ndf = pd.read_excel('data.xlsx', sheet_name='Sheet2')\n<\/code><\/pre>\n5. Exploring Data in a DataFrame<\/h2>\n
5.1 Viewing Data<\/h3>\n
head()<\/code> method. By default, it returns the first 5 rows, but you can specify a different number of rows as the argument:<\/p>\ndf.head()\n\ndf.head(10)\n<\/code><\/pre>\ntail()<\/code> method. By default, it returns the last 5 rows, but you can specify a different number of rows as the argument:<\/p>\ndf.tail()\n\ndf.tail(10)\n<\/code><\/pre>\n5.2 Accessing Columns<\/h3>\n
df['Name']\n\ndf['Age']\n\ndf['Country']\n<\/code><\/pre>\nloc[]<\/code> or iloc[]<\/code> methods to access columns by label or index, respectively:<\/p>\ndf.loc[:, 'Name']\n\ndf.iloc[:, 1]\n<\/code><\/pre>\n5.3 Accessing Rows<\/h3>\n
loc[]<\/code> or iloc[]<\/code> methods and specify the row label or index, respectively:<\/p>\ndf.loc[0]\n\ndf.iloc[0]\n<\/code><\/pre>\ndf.loc[0:5]\n\ndf.iloc[0:5]\n<\/code><\/pre>\n5.4 Accessing Subsets of Data<\/h3>\n
loc[]<\/code> or iloc[]<\/code> methods:<\/p>\ndf.loc[0:5, ['Name', 'Age']]\n\ndf.iloc[0:5, [0, 1]]\n<\/code><\/pre>\n5.5 Summary Statistics<\/h3>\n
mean()<\/code> method to calculate the mean of a numerical column:<\/p>\ndf['Age'].mean()\n<\/code><\/pre>\nstd()<\/code> method to calculate the standard deviation:<\/p>\ndf['Age'].std()\n<\/code><\/pre>\nmin()<\/code> and max()<\/code> methods to calculate the minimum and maximum values, respectively:<\/p>\ndf['Age'].min()\n\ndf['Age'].max()\n<\/code><\/pre>\ncount()<\/code> method to count the number of non-missing values:<\/p>\ndf['Age'].count()\n<\/code><\/pre>\ndescribe()<\/code> method to calculate various summary statistics at once:<\/p>\ndf.describe()\n<\/code><\/pre>\n6. Manipulating Data in a DataFrame<\/h2>\n
6.1 Adding a Column<\/h3>\n
df['Salary'] = [50000, 60000, 70000]\n<\/code><\/pre>\ninsert()<\/code> method to insert a new column at a specific position:<\/p>\ndf.insert(1, 'Salary', [50000, 60000, 70000])\n<\/code><\/pre>\n6.2 Updating Values<\/h3>\n
df.loc[0, 'Salary'] = 55000\n<\/code><\/pre>\nloc[]<\/code> to select the first row and the ‘Salary’ column, and assign a new value to it.<\/p>\n6.3 Filtering Data<\/h3>\n
df_filtered = df[df['Age'] >= 30]\n<\/code><\/pre>\n6.4 Sorting Data<\/h3>\n
sort_values()<\/code> method.<\/p>\ndf_sorted = df.sort_values(by='Age', ascending=False)\n<\/code><\/pre>\nsort_values()<\/code> to sort the DataFrame by the ‘Age’ column in descending order. The resulting DataFrame is sorted based on the specified column(s).<\/p>\n6.5 Removing Rows or Columns<\/h3>\n
drop()<\/code> method.<\/p>\ndf = df.drop('Salary', axis=1)\n<\/code><\/pre>\ndrop()<\/code> to remove the ‘Salary’ column by specifying the column name and the axis=1<\/code> parameter.<\/p>\n7. Aggregating Data in a DataFrame<\/h2>\n
7.1 Grouping Data<\/h3>\n
groupby()<\/code> method.<\/p>\ndf_grouped = df.groupby('Country')['Age'].mean()\n<\/code><\/pre>\ngroupby()<\/code> to group the data by the ‘Country’ column. Then, we select the ‘Age’ column and calculate the mean value using the mean()<\/code> method.<\/p>\n7.2 Pivot Tables<\/h3>\n
pivot_table()<\/code> function.<\/p>\ndf_pivot = pd.pivot_table(df, values='Age', index='Country', columns='Gender', aggfunc='mean')\n<\/code><\/pre>\n7.3 Reshaping Data<\/h3>\n
melt()<\/code>, stack()<\/code>, unstack()<\/code>, and pivot()<\/code>.<\/p>\ndf_melted = pd.melt(df, id_vars='Name', value_vars=['Age', 'Salary'], var_name='Variable', value_name='Value')\n<\/code><\/pre>\n8. Visualizing Data with Pandas<\/h2>\n
plot()<\/code> method on a DataFrame or a Series.<\/p>\n8.1 Line Plot<\/h3>\n
plot()<\/code> method with the kind='line'<\/code> parameter.<\/p>\ndf['Age'].plot(kind='line')\n<\/code><\/pre>\n8.2 Bar Plot<\/h3>\n
plot()<\/code> method with the kind='bar'<\/code> parameter.<\/p>\ndf.groupby('Country')['Salary'].mean().plot(kind='bar')\n<\/code><\/pre>\n8.3 Histogram<\/h3>\n
plot()<\/code> method with the kind='hist'<\/code> parameter.<\/p>\ndf['Age'].plot(kind='hist')\n<\/code><\/pre>\n8.4 Scatter Plot<\/h3>\n
plot()<\/code> method with the kind='scatter'<\/code> parameter.<\/p>\ndf.plot(kind='scatter', x='Age', y='Salary')\n<\/code><\/pre>\n8.5 Box Plot<\/h3>\n