R

A Brief Introduction to Cloud Native Approaches for Big Data Analysis

March 20, 2023
Millie Chapman
by Millie Chapman. Satellites, smart phones, and other monitoring technologies are creating vast amounts of data about our earth every day. These data hold promise to provide global insights on everything from biodiversity patterns to human activity at increasingly fine spatial and temporal resolution. But leveraging this information often requires us to work with data that is too big to fit in our computer's "working memory" (RAM) or even to download to our computer's hard drive. In this post, I walk through tools, terms, and examples to get started with cloud native workflows. These workflows allow us to remotely access and query large data from online resources or web services, all while skipping the download step!

James Hall

Consultant
Department of Statistics

James Hall is a graduate student in the Statistics MA program at University of California, Berkeley. He is a husband and father to three awesome kids. Originally from Baltimore, MD, James earned his bachelors in Mathematics at the United States Military Academy at West Point, NY in 2011, and served as a U.S. Army officer. He’s served as a leader at multiple levels within large organizations with a professional focus on visualizing and communicating complex analysis to decision makers. James’ experience and coursework give him expertise in navigating different statistical methods,...

Michael Ruiz

IUSE Research Team
Psychology

Michael earned his B.A.in Psychology from UC Berkeley and currently works as the manager of Professor Okonofua's Equity, Diversity, and Empathy Navigation Sciences Lab in the UC Berkeley Psychology department.

Cheng Ren

Senior Data Science Fellow
School of Social Welfare

Cheng Ren is a D-Lab Senior Data Science Fellow and a Ph.D. student at the School of Social Welfare. His research interests are community engagement and assessment, nonprofit development, community database, computational social welfare, and data for social goods.

Wadzanai Makomva

Discovery Graduate Fellow
School of Information

Wadzanai is a graduate student at the School of Information and she is a part of the MIMS program. She has a vested interest in the integration between data science, technology and developmental surveillance techniques. She has prior experience working as a quantitative analyst in project management consulting within a professional services firm, public health, and most recently in sustainable construction materials. Wadzanai is particularly interested in increasing access of STEM subjects and fields to under-privileged women of color in the African continent, particularly her home...

Aniket Gupta

Discovery Fellow
School of Information

I am a first year masters student at UC Berkeley school of Information majoring in Information Management and Systems with a focus on Data Science and ML. I like to build optimized yet simple and scalable solutions powered by data using emerging AI technologies.

Daniel Lobo

Computational Social Science Fellow
Sociology

Daniel Lobo is a PhD student in the Department of Sociology with an emphasis in Political Economy at UC Berkeley. He is broadly interested in how culture, or the unspoken “rules of the game,” reproduces inequality within a system of racial capitalism. At the individual level, he is interested in documenting and measuring the extent to which cultural capital and social capital enable or constrain opportunities for intergenerational mobility. At the organizational level, he is interested in documenting and measuring the extent to which culturally-based selection and promotion processes...

Peter Amerkhanian

Graduate Student Researcher (GSR), Instructor
Goldman School of Public Policy (GSPP)

I’m a D-Lab GSR and a graduate student in The Goldman School’s Master of Public Policy/The I School’s Graduate Certificate in Applied Data Science. I have 5 years of experience working on data problems in government and nonprofits. I’m interested in social policy, program evaluation, and computational methods. Python is my principal language, but I’ve developed experience using and teaching a variety of other tools, including R, Excel, Tableau, and JavaScript. I deeply enjoy teaching data science methods and am excited to be a part of the D-Lab.

Aniket Kesari, Ph.D.

Former D-Lab Postdoc and Senior Data Science Fellow
Berkeley Law

Aniket Kesari was a postdoc and data science fellow at D-Lab. He is currently a research fellow at NYU’s Information Law Institute, and will join the faculty of Fordham Law School in 2023. His research focuses on law and data science, with particular interests in privacy, cybersecurity, and consumer protection.

Featured D-Lab Blog Post: Introducing “A Three-Step Guide to Training Computational Social Science Ph.D. Students for...

Avery Richards

Senior Data Science Fellow
School of Public Health

Avery is an MPH graduate at the School of Public Health. With a background in literature and behavioral health, his current research focuses on innovations in applied epidemiology, including multidisciplinary approaches to health and social science data. Avery's general interests include public health surveillance, data quality assurance, and geospatial analysis.