Software Tools

Tactics for Text Mining non-Roman Scripts

April 15, 2024
Hilary Faxon, Ph.D.
by Hilary Faxon, Ph.D. & Win Moe. Non-Roman scripts pose particular challenges for text mining. Here, we reflect on a project that used text mining alongside qualitative coding to understand the politicization of online content following Myanmar’s 2021 military coup.

Thomas Lai

Consultant
School of Information

I am a Product Engineer passionate about applying engineering, data science, machine learning, and problem-solving principles to improve device performance and solve complex challenges. With experience in statistical analysis, lab bench automation, and Python scripting, I have developed a strong technical skill set that allows me to make meaningful contributions to any project. Beyond my work, I am also passionate about exploring new topics and ideas, from the latest technology trends to how to improve the overall well-being of humans. I enjoy applying the first principle to any...

Introduction to Propensity Score Matching with MatchIt

April 1, 2024
Alex Ramiller
by Alex Ramiller. When working with observational (i.e. non-experimental) data, it is often challenging to establish the existence of causal relationships between interventions and outcomes. Propensity Score Matching (PSM) provides a powerful tool for causal inference with observational data, enabling the creation of comparable groups that allow us to directly measure the impact of an intervention. This blog post introduces MatchIt – a software package that provides all of the necessary tools for conducting Propensity Score Matching in R – and provides step-by-step instructions on how to conduct and evaluate matches.

Tracking Urban Expansion Through Satellite Imagery

December 12, 2023
Leïla Njee Bugha
by Leïla Njee Bugha. Among its many uses, remote sensing can prove especially useful to document changes and trends from eras or settings, where traditional sources are either inexistent or infrequently collected. This is the case when one wants to study urban expansion in sub-Saharan countries over the past 20 years. To further remedy the lack of data on land cover uses from earlier time periods, classification methods can be used as well. Using easily accessible satellite imagery from Google Earth Engine, I provide here an example combining remote sensing with classification to detect changes in the land cover in Nigeria since 2000 due to urban expansion.

Processing Videos in Python with OpenCV

November 28, 2023
Leah Lee
by Leah Lee. Videos and images are quickly becoming the most common type of data we store and interact with. Computer vision technologies derive useful information from these forms of data and are now commonly used in health care, agriculture, transportation, and security. OpenCV is a powerful tool for image processing and computer vision tasks. In this blog post, we will explore how we can use OpenCV in Python to carry out basic computer vision tasks. Specifically, we’ll focus on the simple task of identifying an object from a video and labeling a frame with a box around the object.

Mapping Census Data with tidycensus

November 6, 2023
Alex Ramiller
by Alex Ramiller. The U.S. Census Bureau provides a rich source of publicly available data for a wide variety of research applications. However, the traditional process of downloading these data from the census website is slow, cumbersome, and inefficient. The R package “tidycensus” provides researchers with a tool to overcome these challenges, enabling a streamlined process to quickly downloading numerous datasets directly from the census API (Application Programming Interface). This blog post provides a basic workflow for the use of the tidycensus package, from installing the package and identifying variables to efficiently downloading and mapping census data.

Introduction to Item Response Theory

October 24, 2023
Mingfeng Xue
by Mingfeng Xue. Measurements (e.g., tests, surveys, questionnaires) are inevitably involved with various sources of errors. Among many psychometric theories, item response theory stands out for its capability of detailed analyses at the item level and its potential to reduce some of the measurement errors. This post first discussed the limitations of conventional summation and average, which give rise to the IRT models, and then introduced a basic form of the Rasch model, including expressions of the model, the assumptions underlying it, some of its advantages, and software packages. Some codes are also provided.

From paper to vector: converting maps into GIS shapefiles

April 11, 2023
Madeleine Parker
by Madeleine Parker. GIS is incredibly powerful: you can transform, overlay, and analyze data with a few clicks. But sometimes the challenge is getting your data into a form to be able to use with GIS. Have you ever found a PDF or even paper map of what you needed? Or googled your topic with “shapefile” after it to no avail? The process of transforming a PDF, paper, or even hand-drawn map with boundaries into a shapefile for analysis is straightforward but involves a few steps. I walk through the stages of digitization, georeferencing, and drawing, from an image to a vector shapefile ready to be used for visualization and spatial analysis.

Mapping Time-Series Satellite Images with Google Earth Engine API

July 17, 2023
Meiqing Li
by Meiqing Li. Remote sensing imagery has the potential to reveal land use patterns and human activities at a planetary scale. For example, nighttime light intensity extracted from can shed light on spatial patterns of human activities and settlements, especially in places where traditional data are scarce. This blog post introduces Google Earth Engine (GEE) as a general purpose tool to extract time-series remote sensing data from GEE data catalog. I walk through using GEE to obtain data, filter by time and geographic region, and visualize it on static and interactive maps.

Unlock the Joy and Power of Reading in Language Learning

August 21, 2023
Bowen Wang-Kildegaard
by Bowen Wang-Kildegaard. I share my story of how reading for pleasure transformed my English speaking and writing skills. This experience inspired my passion to promote the joy and power of reading to all language learners. Using natural language processing techniques, I dive into the Language Learning subreddit, revealing a trend: Learners are often highly anxious about output practices, but are generally positive about input methods like reading and listening. I then distill complex language learning theories into actionable language learning tips, emphasizing the value of extensive reading for pleasure, pointing to potential methods like using ChatGPT for customization of reading materials, and advocating for joy in the learning journey.