Web Scraping

Teng-Jui (Owen) Lin

Consulting Drop-In Hours: By appointment only

Consulting Areas: Bionanotechnology, Chemistry, Data Curation, Data Sources, Data Visualization, Databases and SQL, HTML / CSS, Javascript, LaTeX, Machine Learning, MATLAB, Meta-Analysis, Python, Regression Analysis, SQL, Web Scraping

Quick-tip: the fastest way to speak to a consultant is to first ...

Aidan Lee

Consulting Drop-In Hours: By appointment only

Consulting Areas: ArcGIS Desktop - Online or Pro, Bayesian Methods, Causal Inference, Cluster Analysis, Data Sources, Data Visualization, Databases and SQL, Digital Health, Excel, Experimental Design, Geospatial Data: Maps and Spatial Analysis, Git or GitHub, LaTeX, Machine Learning, Means Tests, Mixed Methods, Natural Language Processing (NLP), OCR, Python, Qualtrics, R, Regression Analysis, Research Design, Research Planning, RStudio, RStudio Cloud, SAS, Software Output Interpretation, SPSS, SQL,...

Vy Ngo Thai

Consulting Drop-In Hours: By appointment only

Consulting Areas: Python, SQL, Javascript, HTML / CSS, APIs, Data Visualization, Databases and SQL, Digital Humanities, Web Scraping, Software Development, Git or GitHub, Tableau

Quick-tip: the fastest way to speak to a consultant is to first submit a request...

Jose Aguilar

Data Science & AI Fellow 2025-2026
Berkeley Graduate School of Education

Jose R. Aguilar is currently a PhD student in the Policy, Politics, and Leadership program at UC Berkeley’s School of Education. His research utilizes natural language processing, machine learning, and social network analysis to investigate how institutional discourse, algorithmic decision-making, and education policy influence postsecondary access and equity for marginalized students. Before Berkeley, Jose earned his M.A. in Urban Education from Loyola Marymount University and dual B.A./B.S.A. degrees in Government, Latina/o Studies, and Computer Science from the University of...

Jiayu Lai

Data Science & AI Fellow 2025-2026
Political Science

Jiayu Lai is a PhD student in Political Science at the University of California, Berkeley. Her research interests cover trade politics, labor politics, and the political economy of industrial transfers and global production. Prior to UC Berkeley, she received a Bachelor's degree from Sun Yat-sen University and a Master's degree from the University of Chicago.

Frances Leung

Data Science Fellow 2021-2022
School of Information

Frances Leung is a master’s student at UC Berkeley School of Information where she focuses her studies in information and data science. She has a keen interest in leveraging data-driven insights to better understand consumer behaviors and the world around us. In her professional work as a management consultant, she advises retailers and consumer businesses on digital transformation and creating web/mobile experiences that delight consumers through a human-centered approach. Frances holds a Master in Business Administration from York University, Schulich School...

Sahiba Chopra

Data Science Fellow 2024-2025
Haas School of Business

I'm a PhD student in the Management and Organizations (Macro) group at Berkeley Haas. I have a diverse professional background, primarily as a data scientist across numerous industries, including fintech, cleantech, and media. I hold a BA in Economics from the University of Maryland, an MS in Applied Economics from the University of San Francisco, and an MS in Business Administration from UC Berkeley.

My research focuses on the intersection of inequality, technology, and the labor market. I am particularly interested in understanding how to reduce inequality in...

The Evolving Landscape of Web Scraping on Social Media Platforms

March 11, 2025
Nanqin Ying
by Nanqin Ying. As social media platforms enforce stricter policies against unauthorized data collection, businesses and researchers must adapt to new API-based access models. This shift limits large-scale web scraping, impacting industries reliant on social media insights. The transition to paid API access and stringent compliance measures raises concerns about accessibility, cost, and ethical data collection. This article explores the evolving regulatory landscape, the enforcement of API restrictions, and how organizations can legally and ethically navigate data access in a world where scraping is becoming increasingly difficult. Understanding these changes is crucial for staying compliant while maintaining valuable insights from social media data.

Suraj Nair

Data Science Fellow 2023-2024
School of Information

I am a PhD Student at the School of Information. My research interests lie at the intersection of development economics and machine learning, with a focus on the use of large scale digital data and new computational tools to study pressing issues in global development.

Lauren Chambers

Consultant
School of Information

Lauren Chambers is a Ph.D. student at the Berkeley School of Information, where she studies the intersection of data, technology, and sociopolitical advocacy with Prof. Deirdre Mulligan. Previously Lauren was the staff technologist at the ACLU of Massachusetts, where she explored government data in order to inform citizens and lawmakers about the effects of legislation and political leadership on our civil liberties. Lauren received her Bachelor's from Yale in 2017, where she double-majored in astrophysics and African American studies, and she spent two years after graduation in...