Getting, scraping, cleaning data

To download for class

A note about data cleaning and scraping

I know this can all seem overwhelming. These are introductions to new tools every week.

Keep in mind:

Don't get lost in the bog of tools and techniques to master.
What problem does this tool solve?
Data is dirty and finicky and difficult
75 percent of working with data is finding it, transforming it, before you can analyze or visualize it.
It's not about the tools or the cool gadgets.
It's about the story.

Notes on the Midterm due October 21

Obtain and analyze federal government data and write a summary of the analysis.

Not the actual story.

Instead, the memo should detail the process it will take to get to the final story or visualization.

For an example of the level detail I'll be looking for, check out my list under Homework 2 from Class 6

Explain what it took to get the data, who explained it to you, point out the nuances and limitations of it and how it was collected or processed.

Discuss the gaps in tha data and what could be improved. Then, analyze the agency that provided the data set.

Detail the scope, quality, and accesibility of the rest of the department's data.

How journalists might have used it so far or how it could be used in the future.

Length: 1,000 words.