Python Web Scraping

March 7, 2022, 10:00am to 1:00pm

Trying to register, but not affiliated with the UCB campus? If you are from Berkeley Lab (LBL), UCSF, or CZ Biohub, please register via our partner portals here.

If you are from the UCB campus there's no more waitlist! But after registering above, please do fill out the affiliations form if you have not done so at least once before: https://dlab.berkeley.edu/affiliations

Location: Remote via Zoom. Link will be sent on the morning of the event.

Date & Time: This workshop runs from 10am-1pm on Monday, March 7.

Start Time: D-Lab workshops start 10 minutes after the scheduled start time (“Berkeley Time”). We will admit all participants from the waiting room at that time.

Description

In this workshop, we cover how to extract data from the web using Python. We focus on two approaches to extracting data from the web: leveraging application programming interfaces (APIs) and web scraping.

APIs are often official services offered by companies and other entites, which allow one to directly query their servers in order to retrieve their data. When APIs are not available, one can turn to web scraping, which requires downloading a webpage's source code and sifting through the material to extract the desired data. This workshop demonstrates both approaches, their advantages and disadvantages, and how to use both responsibly.

Basic familiarity with Python is assumed. Understanding the material in the Python Fundamentals and Python Data Wrangling workshops highly recommended. We additionally recommend a basic understanding of HTML and CSS.

Topics Covered

  • How the web works
  • Accessing databases via RESTful APIs
  • Using a 3rd party Python package to query data from a API
  • HTML / CSS
  • Webscraping with Beautiful Soup

Requirements

We will assume a basic knowledge of Python. If you've taken the D-Lab's Python Intensive, that should be sufficient.

Prerequisites: D-Lab’s Python Fundamental introductory series or equivalent knowledge.

GitHub Repository: https://github.com/dlab-berkeley/Python-Web-Scraping

Software Requirements:Installation Instructions for Python Anaconda

Is Python Not working on your laptop?

Attend the workshop anyway, we can provide you with a cloud-based solution until you figure out the problems with your local installation.

Feedback: After completing the workshop, please provide us feedback using this form

Questions? Email: dlab-frontdesk@berkeley.edu