Files and Data Formats

Name: Files and Data Formats
Author: StemPy Lab

Everything that a program knows disappears when it ends. A file keeps data after the program is gone, and lets one program hand its data to another. In this course you read and write text files with the with statement, choose between the read, write and append modes, handle a missing file with try and except, treat the lines of a file as records, read and write tables in the CSV format, and save whole lists and dictionaries in the JSON format. A marks book that remembers its marks between runs is the project at the end. The practice runs your code on real files and tells you which results are wrong.

5 lessons, about 2 h 35 min5 simulations11 problems10 practice questionsPython in every lesson

Created by StemPy Lab

Course outline (5 lessons)

Lesson 1about 40 min
Reading and writing text files
A file keeps text after the program ends. open with a name and a mode gives a file object, and the with statement closes it for you. The mode r reads and needs the file to exist, w writes and empties a file that exists, and a adds to the end and keeps the old text. read gives the whole text, readlines a list of lines, and a loop over the file gives one line at a time, each ending with a newline that strip removes. write does not add a newline, so you add it. A missing file raises a FileNotFoundError, which try and except can handle, and os.path.exists asks first.
1 simulation2 practice questions5 Python programs3 problems1 Python lab1 question set
Read the lesson
Lesson 2about 30 min
Lines as records
A line of a file can hold a record: for example name colon mark. Read each line, strip the newline, split it at the separator, and change the text of the number into a number. Keep the records in a dictionary or a list, and write the whole file again when a record changes, with a newline after each line. Real files have bad lines: an empty line, a missing part, a mark that is not a number. Catch the ValueError for each line, count the bad lines with their line numbers, and keep going, so that one bad line does not stop the whole file. Counting lines, words and different words in a file is the same loop with a dictionary.
1 simulation2 practice questions3 Python programs3 problems1 Python lab
Read the lesson
Lesson 3about 30 min
CSV: tables of data
CSV, comma separated values, is a table written as text, one row on each line with commas between the values, and often a header row of names. The csv module reads it: reader gives each row as a list of texts, and DictReader uses the header to give each row as a dictionary. Every value is text, so numbers must be converted. writer and writerow write rows, and the module protects values that hold commas or quotation marks, which split on a comma cannot do. Open the file with newline equal to an empty text for csv. io.StringIO lets a text behave like a file, which is useful for tests.
1 simulation2 practice questions3 Python programs2 problems1 Python lab
Read the lesson
Lesson 4about 30 min
JSON: saving whole structures
JSON is a text format for whole structures: dictionaries, lists, texts, numbers, True, False and None, nested as deep as you like. json.dumps turns a structure into a text and json.loads turns a text back, and json.dump and json.load do the same with a file. indent makes the text neat for people. JSON cannot keep everything exactly: a tuple comes back as a list, a key that is a number comes back as a text, and a set cannot be saved at all, which is a TypeError. Text that is not valid JSON raises a JSONDecodeError. A good pattern is to load the data, change it, and save it, with a default value when the file is missing or damaged.
1 simulation2 practice questions3 Python programs2 problems1 Python lab
Read the lesson
Lesson 5about 25 min
Project: a marks book that remembers
In this project we build a marks book that remembers its marks between runs. The data is a dictionary of names and lists of marks, kept in a JSON file. Functions load and save, add_mark, average and report do the work. A mark must be a whole number from 0 to 100, and a refused mark must not change the file. The load survives a missing or damaged file. We plan the expected states by hand, build the functions, test the refused marks, the new student, the missing file and the damaged file, and extend the book with removal of a mark, a CSV export and a backup.
1 simulation2 practice questions1 Python program1 problem1 Python lab
Read the lesson

Test Your Learning

Files and Data Formats: Predict the Output

10 questions · 15 min

Not started

Your teacher

StemPy Lab

← Data In and Out

← PreviousComprehensions, Iterators and Generators →

More onPython Going Further › Files and Data Formats

Discussion (0)

Loading…

Files and Data Formats

Course outline (5 lessons)

Reading and writing text files

Lines as records

CSV: tables of data

JSON: saving whole structures

Project: a marks book that remembers

Test Your Learning

Your teacher

Discussion (0)