Series and DataFrames

What you will be able to do

After this lesson you can

• explain what a Series and a DataFrame are,
• build a small table from a dictionary, and
• look at its shape, its columns, its types and its summary.

Why pandas?

In chapter 1, you worked with NumPy arrays: columns of numbers of one type. Real data is a table. Each column has a name (distance, period, mass), the columns have different types (names are text, distances are decimals), and some values are missing. A spreadsheet holds such a table. The library for tables in Python is pandas. It is built on NumPy, so everything that you know about arrays works inside it, and it adds names, labels, missing values and the tools to read, clean, join and summarise data. Almost every data analysis in science and business is done with it.

On this site, pandas is ready. The first time that your code uses it, the site needs a few seconds to load it. By custom we import it as pd:

A Series

A Series is a one-dimensional array in which every value has a label.

Predict first. Which line will print 1.0? What will period * 365.25 give? Then run it.

The printed Series shows the labels on the left, the values on the right, and the type of the values at the bottom: float64.

A DataFrame

A DataFrame is a table made of Series that share the same index: columns of the same length, each with a name.

The bars are drawn to fit: the distances are stretched three times. In both columns Saturn, the sixth planet, is the largest of these six.

Picking from a table

describe is a quick way to check a table for mistakes. A minimum of $-999$ for a distance, or a count smaller than the number of rows, shows a problem.

Common mistakes

• Using one pair of brackets for two columns. Write df[["a", "b"]].
• Making a table from lists of different lengths. pandas raises a ValueError.
• Mixing up df["a"] and df.a. The first always works. The second fails if the name has a space.
• Taking iloc for a label. iloc is by place, and loc is by label.
• Expecting a table to be a NumPy array. It has labels, and its columns have different types.

Practice

Start with two quick questions. Each New numbers gives you a fresh one, so you can practise as long as you like.

Then the labs and the problems:

1. Warm-up. Open the lab Build a table. You make a table from lists and find a row in it.
2. Build. Open the problem Make the table and the problem What does describe say?

Remember these

• A Series is an array with labels. A DataFrame is a table of Series with a common index.
• df.shape, df.columns, df.dtypes, df.head() and df.describe() give a first look.
• df["a"] is a column. df[["a", "b"]] is a smaller table. df.iloc[i] is a row by place.
• Each column has its own type.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. What is the shape of a table with 8 rows and 5 columns?

2. How do you pick the two columns name and mass of the table df?

3. What does period.idxmax() give?

4. Why is a DataFrame more useful than a NumPy array for a table of planets?