COVID Tracking Project Records and Resources Now Available

This announcement is authored by COVID Tracking Project Archive Lead, Alex Duryee

The UCSF Library Archives and Special Collections is pleased to announce that the COVID Tracking Project (CTP) records are available online for research.  The CTP is a crowdsourced digital archive that was managed by a group of journalists at The Atlantic and approximately 500 volunteers who gathered, cataloged, and published state-level COVID-19 data over the first fifteen months of the pandemic. “The COVID Tracking Project was a remarkable and influential initiative — part citizen science, part journalism, part crisis response. I’m thrilled that UCSF Archives has acquired, processed, and made available the digital records of this unique organization,” said Amanda French, a digital archivist and key leader of the CTP at The Atlantic.

In addition to the CTP’s data products, this collection includes its data creation and quality records, organizational records, correspondence, and code repositories. Over 2,100 academic articles have cited data from the collection and federal agencies like the Centers for Disease Control and Prevention.

Open records available

The finding aid on the Online Archive of California describes the entirety of the collection and includes all of the CTP records held by UCSF. Records range from data processing infrastructure and documentation, correspondence with state and territorial health departments, original COVID-19 data captures, and Slack discussions like #gratitude and #emoji-march-madness.  A significant portion of the collection is restricted until 2102 to protect the privacy of CTP members. However, the open records are available for digitally and on-site by appointment within the UCSF Library Archives and Special Collections reading room. 

The final data products from the CTP are available on Dryad, in accordance with FAIR principles:

In addition to the final data sets, UCSF developed a tool for viewing the data as it changed over time.  COVID-19 data was never static. Often reporting schedules were inconsistent around weekends and holidays, and data was either reported late or updated long after the initial release. Another factor was that states continuously changed their data definitions throughout the pandemic. UCSF’s Data Explorer lets researchers view CTP’s data as it was updated, providing a more profound view of the topline numbers. Data Explorer includes references to original data sources (generally screenshots of websites and data files) and daily Slack discussions for each reporting source (available on-site at UCSF).

Oral histories and open source tools

Along with the collection’s files and data, the CTP records include oral histories created by the CTP as it came to a close in 2021.  These oral histories provide a human-centered perspective on the data, the organization, and the pandemic in the United States.  With permission from the interviewees, the oral histories are available via Calisphere.

The UCSF Archives and Special Collections also developed several open-source tools to aid in acquisition, preservation, and access to the CTP materials. CTP used platforms like GitHub, Instagram, and Twitter for public and internal communication.  These platforms do not always provide accessible tools for preserving data; thus, UCSF created tools to download posts and private messages and generate access versions in PDF.  These tools are available on GitHub for use in and development of digital archives.

Inspiring future research and education

This collection was designed in adherence to UCSF Library’s Archives as Data initiative and the broader Collections as Data movement. UCSF Archives and Special Collections developed multiple platforms and pathways to approach the collection.

This way researchers across disciplines can discover and use the records in their work. Whether it is from an epidemiological, social science, or data science lens, CTP archive lead Alexander Duryee acknowledges the powerful insights this collection affords, “We believe that this collection will provide key context for the story of the pandemic and that researchers across disciplines will find it illuminating.” By cross-linking between the archival collection, oral histories, and data sets, the collection encourages deep exploration of the “whats” and “hows” of the CTP and its data.

The collection serves as the foundation of the Data Journalism Course In A Box (DJCB) project, which is building a data science curriculum around the CTP records to support journalism education.  The collection includes a comprehensive view of the data, from its initial publication on agency web pages through quality control and publication. Investigative reporter Tyler Dukes is developing the DJCB with the help of the UCSF team. The curriculum uses CTP data to illustrate to journalists how to work with and analyze real-world public health data and how to communicate complex topics to a broad audience.

Project team members

  • Tyler Dukes, data journalism consultant
  • Alexander Duryee, Covid Tracking Project archive lead
  • Edith Escobedo, UCSF project archivist
  • Polina Ilieva, UCSF Associate University Librarian for Collections and archivist
  • Charlie Macquarie, former UCSF digital archivist
  • Kevin Miller, former Covid Tracking Project archive lead

In addition, the team would like to thank the many collaborators across the University of California system and advisory board members for their contributions to this project.

Funding for The COVID Tracking Project Archive was provided by the Alfred P. Sloan Foundation (Sloan grant G-2022-17133).


Alex Duryee Named New COVID Tracking Project Archive Lead

The UCSF Archives & Special Collections is delighted to welcome our new colleague, Alex Duryee who took over from Kevin Miller as the COVID Tracking Project Archive Lead. The project team continues the work of preserving, providing online access, and building educational resources for the organizational records and datasets of the COVID Tracking Project at The Atlantic (CTP).

Alex Duryee

Alex brings a background in metadata, digital archives, and archival access to the COVID Tracking Project Archive team.  He holds a BA from The College of New Jersey and a MLIS from Rutgers University, and also serves as the Manager for Archival Metadata at the New York Public Library.  In this position, he manages the Library’s archival metadata platforms and develops metadata policy for the Library’s archival collections.  He also collaborates with staff across the organization to improve systems integrations and develop new methods for accessing and using archival materials.  Alex also serves on the National Finding Aid Network (NAFAN) Technical Advisory Working Group, SAA’s Technical Subcommittee for Encoded Archival Standards, and as the chair of the SNAC (Social Networks and Archival Context) Technology & Infrastructure Working Group.  He contributes to open-source projects such as ArchivesSpace, as well as developing open-source metadata tools.  In 2019, his team was awarded the C. F. W. Coker Award for Archival Description by the Society of American Archivists.

Alex’s background also includes experience as a freelance ArchivesSpace developer, a consultant with AVP, and a digital archives fellow with Rhizome.

Alex enjoys puzzles of all sorts (including metadata), board games, baking, and dancing.

Continuing Conversations

This week’s story comes from Isabella Durgin, third-year English and geography student at the University of California, Los Angeles (UCLA).

Post by Isabella Durgin:

Human connection can break the space-time continuum. 

This summer, I worked with the UCSF Archives & Special Collections department to help the digital archiving of the COVID Tracking Project (CTP), a cohort of volunteers that worked under The Atlantic to document and report on the first year of the COVID-19 pandemic. As the Project came to a close in 2021, they also began to tell their story – and that was where I came in. Along with the team, my work at UCSF ultimately spun a layered tale of interconnectedness.

As a Digital Archive Intern, I was primarily responsible for the organization and curation of the Oral History Archive component of the broader CTP Archive. These files were the product of CTP’s internal interviews to break down and reflect on the work of volunteers and staff. Interviews occurred across a large period of time, spanning from the winddown of the Project in spring 2021 to several months later, when the gears had pretty much ground to a halt. 

Many interviews allocated a significant portion of time to discuss the community they had found during the first 365 days of the pandemic – March 7, 2020 to March 7, 2021 – during which CTP was producing publicly available data and information.

The people at CTP built an organization to handle an extremely unfamiliar crisis with fairly unfamiliar tools – now virtual workplace darlings like Zoom and Slack. Nevertheless, a thriving hive was built up bit by bit (and byte by byte), founded on prioritizing volunteers as humans and crafting a new language of custom emoji.

And somehow, by the end of the summer, I felt like I had a place in it as well. 

Without speaking to any of these people or bonding by synchronistically living through the same pandemic moments, I still had the same experience of feeling like part of something bigger. Which, in some ways, was the hope – the longstanding one, anyways – for the Oral Histories. One of the intentions was for others to learn from CTP’s approach and use the model in other contexts in order to build more compassionate and sustainable communities. 

Around a year after CTP’s 2021 conversations about the year prior, I was able to tap into the web of people and relationships. The same can be done with the thousands of other files in the CTP Archive; there is the same potential for different fields to learn from the stories told by the outreach efforts or the data journalism. Even the numbers tell a story. We just have to be ready to preserve and to listen – actions I learned are far more similar than one may think.

Open-source tools from The COVID Tracking Project Archive.

The COVID Tracking Project Archive has several unique challenges, namely how to preserve unique, born-digital materials in formats that will be easily accessible to researchers far in the future. Tools like Twitter, Instagram, and Slack are constantly changing their interfaces, making preservation difficult. 

To make the job of the archive and other archivists easier, the COVID Tracking Project is releasing several tools we have developed to preserve these digital formats on our Github Organization. These include: 

  • Twitter Preserver – A tool to convert the downloaded Zip file a user gets from Twitter into stable HTML files. This includes Direct Messages as well as public Tweets and Favorites. View a preview of the output of this tool.
  • XLSX Bulk Converted – A python script that will bulk-convert Excel files into folders of CSV files, one file per worksheet. 
  • Instagram Preserver — A tool that logs into Instagram and downloads all the feed data and images from another account. Instagram is particularly difficult to access without logging in, so this tool uses an internal API to access the user’s feed. 

Our XLSX bulk converter was written by our amazing Tech intern Tracy Lee

We believe these kind of tools provide a model of how to preserve and protect information from proprietary and sometimes fleeting platforms for future researchers. 

Support for development came from the Alfred P. Sloan Foundation.

Welcome COVID Tracking Project Archive Interns

We are pleased two welcome two new interns, Tracy and Isabella, to the COVID Tracking Project Archives. Tracy will be build tools to manage our extensive digital collection, while Isabella will help assess and categorize oral histories and documents. We are happy to have the extra help!

Tracy Lee

Tracy

Tracy first became interested in libraries and archives as a child exploring the History Center at the San Francisco Public Library; this interest evolved into a passion when she took a history of information course as an elective for her BA in Cognitive Science at UC Berkeley and was allowed to handle some of the oldest items in the Bancroft Library collection as part of a class assignment. After a short stint in software development and logistics roles for a couple of seed-stage startups in San Francisco, she returned to the world of libraries by completing a Certificate of Achievement in Library Information Technology at City College of San Francisco and interning at the Asian Art Museum C. Laan Chun Library. She is a current MLIS Candidate at San José State University hoping to focus on digital asset management, government information resources, and library systems.

Isabella Durgin

Isabella

 Isabella is currently studying English and Geography at the University of California, Los Angeles (UCLA). In addition to her studies, she is also the Music | Fine Arts Editor at the Daily Bruin, and aims to obtain her Masters in Library Science. She loves hiking, being outdoors, and improving her climbing skills. 

UCSF Library and San Francisco poets create space for the San Francisco community to “Pause, Breathe, and Re-Connect” during the COVID-19 pandemic

This is a guest post by Dr. Michelle-Linh (Michelle) Nguyen, a primary care doctor and researcher at UCSF and the Zuckerberg San Francisco General Hospital. 

As social distancing rules and regulations begin to relax, many of us are feeling the strain of prolonged social isolation and re-learning how to reach out to others.

On April 29th, 2021, 48 San Francisco and UCSF community members gathered virtually during the lunch hour on Zoom for a series of poetry readings and discussion centered around the human experience of medicine. Farah Hamade, the inaugural UCSF Library Artist-in-Residence, took visual notes and created an art piece that represents the event and experience (featured below).

Pause, Breath, Re-Connect artwork.
© Farah Hamade 2021. All rights reserved.
© Farah Hamade 2021. All rights reserved

Three poets—Kathleen McClung, Sharon Pretti, and Peggy Tahir—were selected through a submissions process from the San Francisco community to read their work. Sharon Pretti read a series of poems written during and after her brother’s pancreatic cancer diagnosis, treatment, and eventual death. Kathleen McClung read a sequence of sonnets inspired by her partner and her experiences navigating his treatment and surgery for a pituitary mass.

© Farah Hamade 2021. All rights reserved.

Peggy Tahir read a series of poems written for each radiation treatment she underwent for breast cancer. The readings were followed by a 10-second pause to create space for reflection and a rich discussion.

Michelle-Linh (Michelle) Nguyen closed the event with a reading from The Book of Delights by Ross Gay, which can be accessed here: https://www.theparisreview.org/blog/2019/02/14/sharing-love/.

© Farah Hamade 2021. All rights reserved.

The introduction of the event and poetry readings were recorded with the poets’ permission. The recording was turned off for the discussion and closing to create a more comfortable, intimate space. After the event, the poetry reading recording, Farah Hamade’s art piece and a poem by Michelle-Linh (Michelle) Nguyen was shared with event registrants and the public.

© Farah Hamade 2021. All rights reserved.

The public can access the recording at the following link: https://archive.org/details/ucsf-pause-breathe-re-connect-poetry-and-discussion-2021-1.

The event was organized by Michelle-Linh (Michelle) Nguyen, Farah Hamade, Polina Ilieva, and Joanna Kang with support from the UCSF Library.

“Unmasking History: Who Was Behind the Anti-Mask League Protests During the 1918 Influenza Epidemic in San Francisco?” an article by Dr. Brian Dolan

In his recent article Dr. Brian Dolan looks at the politics of protests during the 1918 influenza epidemic in San Francisco.

“On April 17, 2020, San Francisco Mayor London Breed did something that
had not been done for 101 years. She issued an order that face masks be
worn in public as a measure to help prevent the spread of infectious disease
in the midst of a pandemic. This act promptly raised questions about
how things were handled a century ago. The media soon picked up on the
antics of an “Anti-Mask League” that was formed in San Francisco to protest
this inconvenience, noting some historical parallels with current public
complaints about government overreach. This essay dives deeper into the
historical context of the anti-mask league to uncover more information
about the identity and possible motivations of those who organized these
protests. In particular it shines light on the fascinating presence of the leading
woman in the campaign—lawyer, suffragette, and civil rights activist, Mrs. E.C.
Harrington.” Read the full story in Perspectives in Medical Humanities (UC Medical Humanities Consortium, May 19, 2020)