← All weeks and tools

Code along: text

Cultural Data Analysis · Week 4: Cultural Data Forms: Text

Interactive tools from this lecture: Term frequency · From words to embeddings

This week's workshop: Text and language models

Eleven exercises from today's slides, in the order they come up in the lecture: from counting words to language models, with the State of the Union speeches from 1790 to 2022. The exercises that build text representations by hand run in Python Tutor, as in Week 1. The ones with scikit-learn, pandas and language models run in Google Colab. Use one Colab notebook for all of them, because later exercises reuse the speeches loaded in exercise 1.

Before the last summary, a Let's try slide sends you to the lecture notebook. Open it once now and choose File → Save a copy in Drive, so that your changes are kept. The box below shows where it comes. The slides show the number of each exercise in their top corner.

There are two kinds of exercise:

• Type and trace (Python Tutor): short pieces of plain Python. Predict what will happen, type the code by hand (the typos are part of the lesson), then click Visualize Execution and press Next to step through it.
• Copy and run (Google Colab): longer code with libraries such as pandas and scikit-learn. Predict the result, then copy the code into Colab and run it. Read the New note: it explains the lines you have not seen before. You don't need to be able to write the rest yet.

Then check your result against the expected output, and try the follow-up under Then.

Contents

---

The State of the Union data (236 speeches, 1790–2022) is at github.com/goto4711/cdai. Python Tutor: pythontutor.com · Google Colab: colab.research.google.com