Data Science

PoliPy: A Python Library for Scraping and Analyzing Privacy Policies

February 8, 2022

In light of recent scandals involving the misuse and improper handling of personal data by large corporations, advocacy groups and regulators alike have given increased attention to the issue of consumer privacy [e.g., 1, 2, 3, 4, 5]. National and local governments have been enacting privacy legislation that requires companies to minimize the amount of data they collect, deters the collection of sensitive data, limits the purposes for which the data are used, and critically, gives users more transparency into data collection and use.

As part...

Where the Streets Have No Name: Spatial Data in Informal Settlements

February 1, 2022

In our era, with Google Maps on every smartphone, it may feel like spatial data is easy to come by. However, this is not the case for many communities in the world. In particular, for informal settlements, developed “outside state control over urban design, planning, and construction,” accurate maps can be hard to come by. You may open up Google Maps to find a few streets with no names, or sometimes, nothing at all. Informal settlements are...

Is your Random Sample Really Random?

January 20, 2022

One of the frequent ways people can run into random numbers is through their research. We often hear the term “random sample,” or a “randomized” assignment to control. Or, sometimes, we can randomly select a certain number of rows or columns from data to perform an analysis on a representative snapshot of the data. Additionally, for many of us from a natural science or engineering background, random numbers are often used in simulations or optimization models. Given the wide variety of uses for random numbers in Data Science, I thought it would be interesting to take an...

Working with spatial networks using NetworkX

December 7, 2021

I have always been interested in working with spatial networks. My first introduction to spatial network modeling was in Prof. John Radke’s Geographic Information Systems class when I learned about building and analyzing spatial networks using the Network Analyst extension in ArcMap. This extension provides powerful tools to solve common network problems, such as finding the best route across a city, finding the closest...

Resisting our Data Doppelgangers: A Proposal for Unpacking the Dangers of Data-Driven Fertility Advertising With Data Science Tools

December 7, 2021

Introduction

When Janet Vertasi, a sociology professor of technology at Princeton, learned of her pregnancy, she decided to conduct a personal experiment. She hid her pregnancy from the internet for nine months. This meant only sharing her pregnancy with close friends and family, using her own personal server while making purchases on Amazon and even opting to use cash For many of her transactions. During this time Amazon mistook her as a “suspicious customer” (Vertasi 2014, Gray 2014). Recall another incident of how Target found out about a...

A Beginner’s Guide to the Bootstrap

November 22, 2021

What is the bootstrap method?

If you take a quantitative methods course here at Berkeley, chances are that you will learn how to perform a bootstrap. As an introductory data science instructor, it’s one of my favorite topics to teach, not just because it’s a powerful and useful tool, but also because it’s incredibly intuitive. In short, the bootstrap -- also known as resampling with replacement -- allows us to generate a distribution of sample statistics given only a single sample, estimating sampling error.The name of this method...

Stumbling Upon Data Sonification When I Fused My Passion for Music with Coding

November 16, 2021

Like many graduate students from the MIDS program who are also full-time working professionals, I return to campus to seek knowledge and satisfy my intellectual curiosity in information and data science. It has become a part of a lifelong learning pursuit that enables me to constantly apply what I learn back into the real world. Along the way, I never forget that it is also important to have fun with science by combining new knowledge with my own passions in arts and music in whatever ways possible. For nearly a decade, I have been helping clients in...

Rural vs. Urban: Using Python to Explore Legislative Data

November 8, 2021

Before COVID-19, becoming a data scientist was never on my radar. As a policy analyst for the California Research Bureau, a legislative research and reference section of the California State Library, I’ve worked on a variety of projects and requests. For the last 8 years, my work has focused on producing timely, confidential ...

Assessing the Effectiveness of a Social Norms-Based Sexual Violence Prevention Digital Campaign on the UC Berkeley Campus

August 31, 2021
In collaboration with the prevention team at the PATH To Care Center (PTC) at the University of California, Berkeley, we experimentally assess the effectiveness of a sexual violence & sexual harassment (SVSH) prevention social media campaign on perceived social norms. Content Warning: This blog post mentions sexual violence & sexual harassment (SVSH)

Project HOME: Modeling and Mapping Eviction Rates in California

August 18, 2021

6 months ago, the D-Lab community made possible a connection between the UC Berkeley School of Information, D-Lab Data Science Fellows, and the Urban Displacement Project (UDP). A summer of brainstorming, collaboration, and multiple Zoom sessions later, the team at Project HOME is excited to present our 5th Year Master of Information and Data...