Survey Design

Causal Effect Estimation in Observational Field Studies of Thermal Comfort

April 1, 2025
Ruiji Sun
by Ruiji Sun. We introduce and apply regression discontinuity to thermal comfort field studies, which are typically observational. The method utilizes policy thresholds in China, where the winter district heating policy is based on cities' geographical locations relative to the Huai River. Using the regression discontinuity method, we quantify the causal effects of the experiment treatment (district heating) on the physical indoor environments and subjective responses of building occupants. In contrast, using conventional correlational analysis, we demonstrate that the correlation between indoor operative temperature and thermal sensation votes does not accurately reflect the causal relationship between the two. This highlights the importance of causal inference methods in thermal comfort field studies and other observational studies in building science where the regression discontinuity method might apply.

Claudia von Vacano, Ph.D.

Founding Executive Director, P.I., Research Director, FSRDC

Dr. Claudia von Vacano is the Founding Executive Director and Senior Research Associate of D-Lab and Digital Humanities at Berkeley and is on the boards of the Social Science Matrix and Berkeley Center for New Media. She has worked in policy and educational administration since 2000, and at the UC Office of the President and UC Berkeley since 2008. She received a Master’s degree from Stanford University in Learning, Design, and Technology. Her doctorate is in Policy, Organizations, Measurement, and Evaluation from UC Berkeley. Her expertise is in organizational theory and...

Field Experiments in Corporations

January 28, 2025
Yue Lin
by Yue Lin. How do social science researchers conduct field experiments with private actors? Yue Lin provides a brief overview of the recent developments in political economy and management strategy, with a focus on filing field experiments within private corporations. Unlike conventional targets like individuals and government agencies, private companies are an emergent sweet spot for scholars to test for important theories, such as sustainability, censorship, and market behavior. After comparing the strengths and weaknesses of this powerful yet nascent method, Lin brainstorms some practical solutions to improve the success rate of field experimental studies. She aims to introduce a new methodological tool in a nascent research field and shed some light on improving experimental quality while adhering to ethical standards.

Causal Inference in International Political Economy: Hurdles and Advancements

September 9, 2024
Yue Lin
by Yue Lin. What are the key challenges and opportunities of applying experiments in the International Political Economy (IPE) research? In this blog, I reviewed an enduring methodological battle between statistics and experiments, and pointed out that the difficulties of randomization and locating credible counterfactuals have served as main hurdles for IPE scholars to widely adopt experimental tools. However, I further demonstrated some new progress in applying survey, field, and lab experiments in the recent IPE scholarship. I concluded that it is crucial for future researchers to think innovatively about how to combine different research methods to make causal claims in IPE studies.

Survey Fundamentals

February 21, 2024, 1:00pm
This two-hour workshop offers a comprehensive introduction to designing and conducting survey studies. Tailored for beginners, it provides clear, step-by-step guidance complemented by concise examples, practical considerations, and useful support materials. Participants will learn the entire process, from formulating a research question to creating, administering, and analyzing surveys, as well as interpreting results and communicating their findings.

Survey Fundamentals

April 11, 2024, 3:00pm
This two-hour workshop offers a comprehensive introduction to designing and conducting survey studies. Tailored for beginners, it provides clear, step-by-step guidance complemented by concise examples, practical considerations, and useful support materials. Participants will learn the entire process, from formulating a research question to creating, administering, and analyzing surveys, as well as interpreting results and communicating their findings.

Transparency in Experimental Political Science Research

April 9, 2024
Kamya Yadav
by Kamya Yadav. With the increase in studies with experiments in political science research, there are concerns about research transparency, particularly around reporting results from studies that contradict or do not find evidence for proposed theories (commonly called “null results”). To encourage publication of results with null results, political scientists have turned to pre-registering their experiments, be it online survey experiments or large-scale experiments conducted in the field. What does pre-registration look like and how can it help during data analysis and publication?

Design Your Observational Study with the Joint Variable Importance Plot

March 12, 2024
Lauren Liao
by Lauren Liao. When evaluating causal inference in observational studies, there often is a natural imbalance in the data. Luckily, variables are often measured alongside that can be helpful for adjustment. However, deciding which variables should be prioritized for adjustment is not trivial – since not all variables are equally important to the intervention or the outcome. I recommend using the joint variable importance plot during the observational study design phase to visualize which variables should be prioritized. This post provides a gentle guide on how to do so and why it is important.

From Ideas to Streamlined Research: The Benefits of Full-Cycle Methodology

December 5, 2023
Farnam Mohebi
by Farnam Mohebi. As an aspiring leading researcher, I find the full-cycle research methodology crucial for transforming initial curiosities into organized studies and research products. This approach begins with thorough observation, leads to theory and hypothesis development and experimentation, and concludes with synthesizing findings into coherent narratives. It's beneficial for researchers of all backgrounds, enhancing the depth and impact of their work. By embracing this method, researchers comprehensively understand each stage and its contribution to the broader research context and can lead the process of converting an initial unspecified research idea to a streamlined research study and product. This systematic approach is particularly effective in complex studies, fostering thorough, investigative, and innovative research processes.

Introduction to Item Response Theory

October 24, 2023
Mingfeng Xue
by Mingfeng Xue. Measurements (e.g., tests, surveys, questionnaires) are inevitably involved with various sources of errors. Among many psychometric theories, item response theory stands out for its capability of detailed analyses at the item level and its potential to reduce some of the measurement errors. This post first discussed the limitations of conventional summation and average, which give rise to the IRT models, and then introduced a basic form of the Rasch model, including expressions of the model, the assumptions underlying it, some of its advantages, and software packages. Some codes are also provided.