Evaluating Automated Transcription Accuracy: A Data Science Fellowship Report

Posted on September 22, 2023 by Kate Tasker

Guest post by Noel Salmeron, 2023 Senior Data Science Fellow for the Industry Documents Library and Data Science Initiative.

Hi everyone! I had the opportunity of interning for the Industry Documents Library in coordination with the Data Science Initiative as the Senior Data Science Fellow for the Summer of 2023. I am working towards my Bachelor’s degree in Data Science and minoring in Education, and I plan to graduate in May 2024. I feel grateful that I could earn this position with UCSF and work with the fascinating Industry Documents Library as I realize how valuable archives and data are, especially when doing my own research. The Data Science Initiative was extremely helpful in teaching me Machine Learning and Natural Language Processing topics pertinent to the project and valuable for my future in data science.

Project Background

Currently, the Industry Documents Library contains more than 18 million documents relating to public health, as well as thousands of audiovisual materials, such as “recordings of internal focus groups and corporate meetings, depositions of tobacco industry employees, Congressional hearings, and radio and TV cigarette advertisements.” With this project, we wanted to evaluate the transcription accuracy of digital archives and its impact on documentation and the creation of subject words and descriptions for such archives.

Project Team

Kate Tasker, Industry Documents Library Managing Archivist
Rebecca Tang, Industry Documents Library Applications Programmer (and Junior Fellows Advisor)
Geoffrey Boushey, Head of Data Engineering (and Senior Fellow Advisor)
Rachel Taketa, Industry Documents Library Processing and Reference Archivist
Melissa Ignacio, Industry Documents Library Program Coordinator
Noel Salmeron, Senior Data Science Fellow
Adam Silva, Junior Data Science Fellow
Bryce Quintos, Junior Data Science Fellow

Project Terminology

Here are a few important terms to note!

Metadata: a set of data that describes other data (i.e., author, date published, file size, etc.)
Classification: categorizing objects (or text) into organized groups
Text cleaning: reducing complex text to simple text for more efficient use in Natural Language Processing

And a few terms were used interchangeably throughout this project!

Description / Summary
- A condensed version of some text
Subject / Tag / Keyword / Topic
- A single word that helps to define the text or frequently appears within the text

Project Objectives

Overall, the project had a couple of main objectives. The team wanted to train a Machine Learning model to extract subjects from Industry Documents Library video transcripts and evaluate the accuracy of the machine-generated subjects. We planned to utilize the junior interns’ datasheet they created with subjects and descriptions for over 300 videos to train the model for each tag we chose to analyze.

(The video transcripts were generated beforehand by Google AutoML with the help of Geoffrey Boushey).

Transcript Cleaning

Once the video transcripts were created from Google AutoML, I was able to clean the text using techniques I learned from previous Data Science Initiative workshops. The “Machine Learning NLP Prep” workshop techniques were especially helpful for this portion of the project. I began by setting all 324 transcripts in our dataframe to a lowercase format. This helps simplify text analysis in the long run, especially when avoiding case sensitivity complications. My next step was to remove stop words, which are common and redundant words such as articles, conjunctions, and prepositions. This was possible with the Natural Language Toolkit library for Python, which contains a list of stop words I could add to since I especially noticed ‘p.m.’ and ‘a.m.’ appearing in depositions. I continued by removing everything that isn’t alphabetic using a regular expression (or regex), a sequence of characters corresponding to a pattern to be matched. Any single characters or two character pairs were also removed. Finally, it was essential to stem words to be able to group common words without worrying about suffixes.

ML Model Creation using ID and subject/tag

After text cleaning, we set video IDs as their indices in our running dataframe to efficiently and consistently identify them. Our running dataframe consisted of a row for each of the 324 videos with columns that denoted their ID, subject words, transcript, and a category value of ‘0’ for ‘no’ or ‘1’ for ‘yes’ that corresponded to whether or not the video’s subjects words included the specific tag we were after in each single-tag analysis.

To provide a more concrete example, we will use the “lawsuit” tag, which means each video was denoted with a ‘1’ in the category column if it contained the “lawsuit” tag from the junior interns’ datasheet.

Continuing, we created training and test sets from the dataframe with a 50/50 split. This was followed by a pipeline of several operations in a sequence that included Count Vectorization and Random Forest Classification. Count Vectorization is a method in Natural Language Processing to convert text into numerical values primed for Machine Learning. This way, we can note word frequency in each word for each transcript. Furthermore, Random Forest Classification is a collection of decision trees that make binary decisions based on input and continually “bootstraps” (re-samples) from the training data set to make predictions about whether or not a video contained the “lawsuit” tag.

Features for Each Tag

We then gathered feature words and their importance values as to how they supported the model in determining if a video belonged to the “lawsuit” tag. These feature words included “exhibit,” “plaintiffs,” “counsel,” and “documents,” which change every time we run the model. It appears the less common words also slipped through, such as the company name “Mallinckrodt,” which may not appear as important in other transcript datasets relating to lawsuits.

Chart displaying a list of feature words such as “email,” “exhibit,” “court,” and other words, each labeled with numeric value representing the word’s importance value.

Cross Validation and Match Probability

Moving forward, we used Cross Validation to verify that the model’s performance was not drastically different with different training and test subsets from the running dataframe. Following this process, we were able to create a dataframe that included a column “y_adj” to indicate “Not” for the video not falling under the “lawsuit” tag and an indication of “Match” otherwise. Moreover, we included two columns, “prob_no_match” and “prob_match,” that denote the model’s assessment of the probability that a video doesn’t fit under “lawsuit” or does, respectively.

Chart displaying a list of video IDs and an associated numeric value representing the video’s probability match.

We also ran some code that narrowed down the dataframe to videos where the model incorrectly predicted a video’s match.

Chart displaying a list of video IDs and associated information, representing videos which had been incorrectly matched.

This is where we began to run into issues with this dataset since it contained a relatively small amount of videos and, therefore, a low number of videos where the “lawsuit” tag applied. The “lawsuit” tag was filed under only 26 of the 324 videos, a mere 8 percent of the dataset. It was also quite difficult to discern an appropriate threshold for whether or not a video transcript should be marked as a match to a tag because the videos that the model marked incorrectly usually appear to have significantly different probabilities for matching.

This caused our models for tags with counts under 25 or so to result in a non-existent F-score, as well as precision and recall, but a high accuracy which I will explain shortly. Meanwhile, an F-score is critical in providing an overall measure or metric for the performance of a Machine Learning model using its precision and recall.

Chart displaying a list of tags including “tobacco,” “marketing,” “lawsuit,” and other words, and a numeric value representing how many times the tag appears.

Precision & Recall

Diving into Precision and Recall, Precision can be defined as the proportion of correct positive predictions in the number of predicted positive values, while Recall is the proportion of correct positive predictions in the number of actual positive values.

In this project, the positive values would be video matches for a tag, so in terms of the project, precision is the proportion of correct match predictions out of the predicted matches, and recall is the proportion of correct match predictions out of the actual, true matches. In addition, Accuracy refers to the comprehensive correctness of all positive and negative predictions.

This image may also help visualize the precision/recall relationship:

Graphic illustrating the concept of precision (how many retrieved items are relevant) vs the concept of recall (how many relevant items are retrieved).

Thresholds

Another step we took in this project’s analysis was creating precision-recall curves for specified thresholds by the Scikit-learn library that allows for Machine Learning in Python. This way, we could recognize that as the threshold for the probability of a match increases, the precision slowly increases from about 90 percent to 100 percent. In comparison, the recall decreases from 100 percent to 0 percent.

Two charts illustrating precision-recall curves.

This can be explained by referring back to the definitions of precision and recall! Suppose the threshold for the probability of a match increases and becomes stricter. In that case, precision (the proportion of correct matches out of predicted matches) will only increase as the requirement for a video to be labeled as a match becomes stricter. Regarding the recall (the proportion of correct match predictions out of the actual matches), it becomes clear that since there is more precision, there will be no videos incorrectly marked as matches or any videos correctly marked as matches.

Opportunities for Further Research

There were a few concerns and curiosities that I ended the beginning of this project with since it was simply a pilot, and there is much more to be explored. This includes more text cleaning in subjects/tags and transcripts to make the Natural Language Processing as streamlined as possible. Additionally, it would be crucial to explore this same analysis of subjects/tags for descriptions/summaries that we could not get to. Having a fully-developed human-made datasheet for a larger dataset to explore would also be incredibly useful.

Conclusion

I am pleased to have been a part of this team with UCSF’s Industry Documents Library and Data Science Initiative this summer, as it provided me with extensive real-world experience in data analysis, machine learning, and natural language processing. It truly puts into perspective how much valuable data is out there and all of the fascinating analysis you can conduct.

Prior to this summer, I had worked with various datasets in classes, but I felt inspired by the IDL’s endeavor to enhance its vast collection and make it easier for users to search through documents with supplementary metadata. I can especially appreciate this as I have spent countless hours sifting through documents for research papers in the past. Once the subject and description generations are in full effect, I can only imagine the potential of this data and what it could lead to, as I hope it supports other people’s work.

I also tremendously appreciate the time and effort the junior interns, Adam and Bryce, put into populating their datasheet after watching hundreds of videos. Their work was foundational to getting this project running.

I also want to express my appreciation for Geoffrey and Rebecca throughout this summer for working closely with me, making me feel welcome, and addressing any concerns or questions I had during my fellowship. I am incredibly grateful for this work experience with exceptional communication, collaboration, and kindness.

Thank you to UCSF and everyone on this team for an enjoyable and fascinating fellowship experience!

Addendum: When Should We Apply a Subject Tag to an Uncategorized Document?

By Geoff Boushey, UCSF Library Head of Data Engineering

Overview

Noel described the process for creating a machine learning (ML) model, analyzing the features that go into classifying a document, and applying the model to estimate the probability that a transcript generated from the Tobacco or Opioid collection should be included in a subject tag, such as “marketing,” “legal,” or “health.”

Because most tags in the collection show up in less than 10% of the records in our training and testing set, we shouldn’t expect most tags to apply to most records. As a result, we’re looking for a relatively rare event. If we were only concerned with the overall accuracy or our model, we could achieve 90% effectiveness by never applying a specific tag to a record.

The output from our machine learning model reflects this low probability. By default, our machine learning model would only include a tag if it estimates that the probability of a match exceeds 50%. Because we’re trying to predict a relatively rare event (again, a specific tag would only apply to at most 10% of the records in a collection), it’s unlikely that we’ll have many predictions that exceed this threshold. In fact, when we test our model, we can see that records that clearly (based on human observation) belong to a specific category may have no more than a 30-40% estimated probability of belonging to this category according to the ML model. While this is below the default 50% threshold, it does represent a much higher probability than random chance, (30-40% vs 10%).

We don’t want to erroneously include a tag too often, or it will become clutter. We don’t want to erroneously exclude it too often, or researchers will miss out on relevant record matches. We may want to lower the threshold for determining when to apply a tag to a particular record, but the right threshold isn’t always clear, and can vary depending on the frequency of a tag, the accuracy of our model, and the scenario-dependent benefit or harm of false positives versus false negatives.

The harm of false positives or negatives depends heavily on the research or use scenario. For example, a researcher who wants to retrieve all reasonably likely matches and is not concerned with the inclusion of a few documents that are not related to litigation might want to set the threshold very low, even below 10%. Alternatively, a researcher might simply wish to sample a small number of litigation-related documents with a very high level of accuracy. In this case, a high threshold would be more beneficial.

Precision and Recall curves can help find an optimal threshold that strikes the right balance between false positives and false negatives.

Technical Considerations and Limitations

Because our initial dataset is small (only 300 human reviewed records are available for supervised classification), and many of the tags only show up in 10% of the records, we limit our initial analysis to a small set of metadata tags. Because these tags are human-generated and do not conform to a limited and controlled vocabulary, there is inconsistency in the training data as well. Some tags are redundant, showing up in clusters (legal and litigation, for instance, have a 95%+ overlap). Other times, two categories that might be better approached as a single category cause a split that may greatly reduce the effectiveness of an ML based classifier. Human ambiguity is often amplified when used to train ML models, and we see that effect at work here.

Precision-Recall Curves

Because there is a class imbalance between positive and negative categorization (including versus excluding a tag) and false positives are unlikely to be a serious problem (though, as discussed above, there may be some scenarios, such as sampling, where we would want to avoid them), we’ll take a look at precision-recall curves for a few of the more commonly occurring tags.

For quick reference, *Precision* refers to how often a positive classification was correct. For example, if our model predicted that a “Legal” tag should apply correctly 9 times and incorrectly 1 time, the Precision would be 90%. *Recall* refers to how often a positive classification was accurately detected. For example, if 10 records should have been classified as Legal, and our model detected 8 of them, our recall rate would be 80%. Ideally, we would like to strike some kind of balance between these two metrics, something we can achieve by raising or lowering the probability threshold for including a record in a tag. For example, if our model assigned a 30% chance that a particular record should be classified as “Legal”, we might or might not set that assignment based on whether we are trying to improve precision or recall.

For a more technical/mathematical discussion of Precision and Recall, please consult the scikit learn documentation at:

https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html

Workbook

The jupyter notebook implementing a Precision-Recall visualization for the “Legal” tag is

available at:

https://github.com/geoffswc/IDL-DSOS-2023/blob/main/Precision-Recall-Tag.ipynb

This workbook uses the scikit-plot module from scikit-learn to generate a precison-recall curve for a tag used in the classification model. Keep in mind that there isn’t much benefit to analyzing tags that show up in less than 10% of the records, and some tags may result in an error, as positive observations may be so rare (fewer than 1-2% of the records) that there is insufficient data to train or apply an ML model (a random test/train split may have *no* observations for a rare tag such a small dataset).

The visualizations generated by this workbook are available in the next section.

Visualization and Interpretation

This section displays the PR curve for “Legal”, a tag that shows up in approximately 10% of the training records. Keep in mind that common tags like “Tobacco”, which show up in 90% of the records, are auto-assigned based on the source of the collection, and do not represent the common use case. As a result, “Legal” will provide a better overview for a common tag that does not apply to most records, and performs relatively well in our predictive model.

Precision-Recall

The precision recall curve for Legal indicates a wide threshold range that preserves usable precision and recall levels. Very high or low thresholds cause degradation of model performance, but precision and recall above 80% are available with flexibility to optimize for one or the other.

Precision/Recall-Threshold

This chart plots both precision and recall curves on the Y axis with the threshold level on the X axis. We see a rapid improvement of precision with a gradual, near-linear decrease in recall, indicating an effective threshold range well below 50%.

Graph plotting the precision and recall curves (on the Y axis) with the threshold level (on the X axis).

Production Application

Although our current data set is small, these results suggest that there is some value in using a supervised classification model to extend metadata to uncategorized documents based on ML generated transcript, though there are a number of challenges, and integrating these techniques into production would involve a number of decisions that are outside the scope of this pilot.

Challenges

A production rollout of an ML based model similar to this pilot would likely run into a number of issues with scale, such as:

Training Data: our supervised machine learning model requires a set of categorized transcripts for training. This is a very time and labor intensive undertaking. We may not be able to create a sufficiently large and broad training dataset to create a meaningful model that covers even the most common tags.

Varying Thresholds: The ideal threshold will vary based on the model performance for each individual tag and the research objectives. This variance, combined with the scale of processing required, may make customizable searches based on tag probability unrealistic in a production system.

Availability of Transcripts: The tobacco, opioid, and other industry documents collections contain a large number of files (current estimate is 18 million), many are video or audio files without transcriptions. Without transcriptions available, it won’t be possible to apply the results of an ML model to make predictions for uncategorized documents.

Recommendations

This pilot does provide a template for an interesting and promising approach, and researchers may be interested in building their own ML models to analyze the transcripts in the collections.

We could provide some of this utility without a full production integration through the following:

Pre-Built Transcription Datasets: The Industry Documents Library website currently provides pre-built transcription datasets for many image record collections. A similar initiative to provide transcriptions for video and audio would provide substantial benefit for researchers, independent of the ML based classification model.

Classification Probability Estimates: Instead of integrating classification probabilities or tags into search, we could provide the ML output for each record in a pre-built dataset. This would leave the decision for setting a threshold up to researchers, but it would avoid the need to re-generate results based on model performance and researcher scenario for each tag. This approach might allow researchers to benefit from partial information.

Generalized ML Models: Several AI tools, such as Google AutoML AI, do provide pre-trained models that can provide categorization. Because these models wouldn’t be trained specifically on our metadata, they may not capture the kind of classification most relevant to researchers, but they would eliminate the need for the very labor intensive generation of a training data set.

Student Fellows Explore Machine Learning with UCSF Industry Documents Library and Data Science Initiative

Posted on July 18, 2023 by Kate Tasker

The UCSF Industry Documents Library (IDL) and Data Science Initiative (DSI) teams are excited to be working with three Data Science Fellows this summer. The Data Science Fellows are part of a joint IDL-DSI project to explore machine learning technologies to create and enhance descriptive metadata for thousands of audio and video recordings in IDL’s archival collections. This year’s summer program includes two junior fellows and one senior fellow.

Our junior fellows are tasked with manually assigning or improving metadata fields such as title, description, subject, and runtime for a selection of videos in IDL’s collection on the Internet Archive. This is a detailed and time-consuming task, which would be costly to perform for the entire collection. In contrast, our senior fellow is using transcriptions of the videos, which we have generated with Google’s AutoML tool, to explore different technologies to automatically extract the descriptive information. We’ll then compare the human-generated data with the machine-generated data to assess accuracy. The hope is that IDL can develop a workflow for using machine learning to create or improve metadata for many other videos in our collections.

Our Junior Data Science Fellows are Bryce Quintos and Adam Silva. Bryce and Adam are both participating in the San Francisco Unified School District (SFUSD) Career Pathway Summer Fellowship Program. This six-week program provides opportunities for high school students to gain work experience in a variety of industries and to expand their learning and skills outside of the classroom. Bryce and Adam are learning about programming and creating transcription for selected audiovisual materials. The IDL thanks SFUSD and its partners for running this program and providing sponsorship support for our fellows.

Noel Salmeron is our Senior Data Science Fellow participating in Life Science Cares Bay Area’s Project Onramp. Noel is using automated transcription tools to extract text from audiovisual files, run sentiment and topic analyses, and compare automated results to human transcription. Noel also provides guidance and mentoring to the Junior Fellows.

Our Fellows have shared a bit about themselves below. Please join us in recognizing Bryce, Adam, and Noel for their contributions to the UCSF Library this summer!

IDL-DSI Junior Data Science Fellow Bryce Quintos

Hi everyone! My name is Bryce Quintos and I am an incoming freshman at Boston University. I
hope to major in biochemistry and work in the biotechnology and pharmaceutical field. As someone who is interested in medical research and science, I am incredibly honored for the opportunity to help organize the Industry Documents Library at UCSF this summer and learn more about computer programming. I can’t wait to meet all of you!

IDL-DSI Junior Data Science Fellow Adam Silva

Hi, my name is Adam Silva and I am a Junior Intern for the UCSF Library. Currently, I am 17 years old and I am going into my senior year at Abraham Lincoln High School in San Francisco. I am part of Lincoln High School’s Dragon Boat team and I am also a part of Boy Scout Troop 15 in San Francisco. My favorite activities include cooking, camping, hiking, and backpacking. My favorite thing that I did in Boy Scouts was backpacking through Rae Lakes for a week. I am excited to work as a Junior Intern this year because working online rather than in person is new to me. I look forward to working with other employees and gaining the experience of working in a group.

IDL-DSI Senior Data Science Fellow Noel Salmeron

My name is Noel Salmeron and I am a third-year data science major and education minor at UC Berkeley. I’m excited to work with everyone this summer and looking forward to contributing to the Industry Documents Library!

Welcome Dick Fine Papers intern Lupe Samano!

Posted on May 31, 2023 by Peggy Tran-Le

Please join us in giving a warm welcome to our new intern, Lupe Samano!

Lupe will be working on processing the Dick Fine Papers (MSS 2022-02). UCSF physician Dr. Richard H. Fine (1940 – 2015) worked at the San Francisco General Hospital and Trauma Center, where he served as chief of the adult health center for 25 years and helped found a primary care residency program to train doctors to work with poor and vulnerable patients. The collection contains yearbooks, appointment books, correspondence, clippings, audio/visual material, photographs, and ephemera.

Lupe has provided her introduction below:

Hello, my name is Lupe and I recently completed the MLIS program at San Jose State University. I moved to San Francisco in 2013 to attend San Francisco State University where I earned my BA in Philosophy and Religion. The past 6-ish years I worked with kids but didn’t find it in me to pursue teaching. During the COVID pandemic, my program organized a small library cohort, and the librarians there convinced me to pursue a degree in MLIS. I enrolled in the program shortly after and discovered that my true passion lies in archives.

I had the privilege of interning for NPS at the Presidio Park Archives and Record Center where I had a great time learning and exploring about not just archives but the history of San Francisco. I am thrilled for this opportunity to further expand my knowledge of the city’s history and gain more hands-on experience in archival work.

In my free time, I enjoy thrifting, hiking, watching my dog frolic on the beach, and starting coloring pages that I never seem to finish.

Welcome to Summer Interns May Yuan and Lianne de Leon!

Posted on June 18, 2021 by Kate Tasker

Please join us in giving a warm welcome to our two newest summer interns, May Yuan and Lianne de Leon!

May and Lianne are both participating in the San Francisco Unified School District (SFUSD) Career Pathway Summer Fellowship Program. This six-week program provides opportunities for high school students to gain work experience in a variety of industries and to expand their learning and skills outside of the classroom. Lianne and May will be working (remotely) with the UCSF Industry Documents Library (IDL), and we are grateful to SFUSD and its partners for sponsoring these internships.

May and Lianne will be working on several collection description projects with IDL this summer, including correcting and enhancing document metadata, and creating descriptions for audio-visual materials. They have provided their introductions below.

My name is May Yuan and I’m a junior at Raoul Wallenberg Traditional High School. During my free time, I enjoy reading, learning and trying new things, and helping others academically. I’m super excited to work here at the UCSF IDL to help provide valuable information to the public as well as learn more about the various documents, lawsuits, etc. myself; I also hope to enhance my productivity and organization skills during my time working here as these skills are crucial to college and everyday life in general. The career paths I’m interested in are bioengineering (bioinformatics/biostatistics), law, and finance.

Hi, my name is Lianne R. de Leon. I am a part of the Class of 2023 at Phillip and Sala Burton High School. In the past, I have worked on VEX EDR Robotics competition in 2018-2019. In my spare time I enjoy trying new foods and yoga. I aspire to become a computer hardware engineer and to travel across the entirety of Asia. I look forward to meeting and working with you all.

Welcome to IDL Summer Intern, Khushi Bhat

Posted on June 7, 2021 by Kate Tasker

Please join us in giving a warm welcome to Khushi Bhat, who will be conducting a remote internship with the UCSF Industry Documents Library (IDL) this summer.

Khushi is currently a rising senior at Rutgers University where she is majoring in Biotechnology and minoring in Computer Science. This summer, she is working in the Industry Documents Library researching tools and methods to extract geographic locations from a collection of documents related to the tobacco industry’s influence in public policy.

Khushi will be conducting an independent course project to help the IDL team enhance descriptive metadata for our industry documents collections. We have long been aware of a research need to be able to filter documents by geographic location. Tobacco control researchers and other public health experts at UCSF and around the world use the documents in the Industry Documents Library to understand how corporations impact public health. This research is often used to inform policymakers who write laws and policies regulating the sale and use of products such as tobacco. Researchers and policymakers need information which relates to their local area such as their city, county, state, or country.

Geographic location is not currently included in IDL’s document-level metadata, and since IDL contains more than 15 million documents it is not feasible to manually catalog this information.

Khushi’s work will focus on researching Natural Language Processing (NLP) and Named Entity Recognition (NER) text analysis methods. She will investigate available tools which have the potential to automatically identify and label geographic information in text. Khushi’s research, recommendations, and pilot testing will help the IDL team outline workflows and strategies for enhancing our document metadata to include geographic information.

Khushi aspires to pursue a career in bioinformatics in the future and intends on pursuing higher education in this field upon graduation. In her spare time, Khushi enjoys dancing, baking, and hiking. Prior to joining Rutgers, she was an avid Taekwondo practitioner (and has a 2nd degree black belt to show for it!)

Image of IDL intern Khushi Bhat — IDL Summer Intern Khushi Bhat

Highlighting the work of Freeman Bradley, of UCSF’s Research Development Department and the Black Caucus

Posted on May 3, 2021 by Erin Hurley

By Shannon Foley, Archives & Special Collections Intern

Brought to Light wants to bring attention to remarkable former UCSF faculty member Freeman Bradley. His significant contributions to the medical community and the Black community at UCSF deserve to be recognized. Bradley grew up in Alabama, and after high school, he continued his education at Howard University in Washington D.C., where he received his bachelor’s degree in Biology. After his graduation, Bradley moved to Maryland and started working at the National Institute of Health, where he remained for four years before starting his career at UCSF. His position was with the Cardiovascular Research Institute, where he conducted research about respiratory changes associated with various anesthesias.

During his time at UCSF, Freeman Bradley worked as a technician to Dr. John Severinghaus and and Bradley’s contributions were fundamental to Severinghaus’s groundbreaking work. From 1957 to 1958, Dr. Severinghaus and Mr. Bradley combined technology created by Richard Stow and Leland Clark to create the first blood gas analysis system. Shortly after the first system was created, they were commercialized and proved revolutionary in health care. In Dr. Severinghaus’s written account of his research and the evolution of the invention of the blood gas analysis system, he emphasizes how his and Mr. Bradley’s invention changed medicine. By the 1960s they blood gas analysis systems were widely available, and and these tests provided essential information about a patient’s illness.. These systems are still used today, and in 1985 Dr. Severinghaus donated the first apparatus he and Mr. Bradley worked on at the Smithsonian Museum. In 1977 after his research with Dr. Severinghaus, Mr. Bradley was appointed Director of Development and Research. In this position, he helped progress the technology and development of medical tools. One of the other advancements he made at his time at UCSF was in the transportation technology of newborn babies or neonates. His contributions to medical advancements do not go unnoticed.

Image taken from SYNAPSE – THE UCSF STUDENT NEWSPAPER, VOLUME 27, NUMBER 19, 24 FEBRUARY 1983, https://synapse.library.ucsf.edu/?a=d&d=ucsf19830224-01.1.3.

Freeman Bradley was not only an incredible asset to advancing medical research, but he also was an active member in UCSF’s Black Caucus. The Black Caucus is a club at UCSF whose mission statement is “The Black Caucus is a forum open to all Black-identified individuals and allies on this campus. Here they may openly express themselves regarding matters of race as they affect life on the campus and in the community. The Black Caucus serves as an instrument for the formation of a Black consensus on those racial matters that affect every person on this campus. This consensus will then be presented to the Administration for appropriate action.” One of the founding members and President of the Black Caucus, David Johnson, worked to create this community where Black members of UCSF could have their needs and concerns met. During Freeman Bradley’s time at UCSF and as an active member of the Black Caucus and used his calm temperament to make sure issues could be addressed and changes made. Mr. Bradley is quoted saying that even though he was criticized for staying diplomatic, he knew that it was the way to be more successful in the long run. In a 1983 interview of Mr. Bradley printed by Synapse, he shared his concerns with the lack of Black role models for youths in the sciences. He believes that minorities would be more likely to become a part of the medical field with more role models. Freeman Bradley is the perfect example of a role model to the youth and can be seen as an inspiration to all.

Works Cited:

“David Johnson, Freeman Bradley – Black Caucus Leaders.” Synapse, Volume 27, Number 19, 24 February 1983, synapse.library.ucsf.edu/?a=d&d=ucsf19830224-01.1.3, accessed April 21, 2021.

Severinghaus, John W. “The Invention and Development of Blood Gas Analysis Apparatus.” Anesthesiology, vol. 97, no. 1, 2002, pp. 253–256., doi:10.1097/00000542-200207000-00031, accessed April 21, 2021.

New Archives Intern – Shannon Foley

Posted on February 12, 2021 by Erin Hurley

Please join us in welcoming our new intern, Shannon Foley. She’ll be working with us on writing and researching blog posts, as well as working on part 2 of our legacy finding aid project.

“Hello, My name is Shannon Foley. I am incredibly excited to be interning at UCSF’s
Archives & Special Collections this semester. I am currently in my last semester at the University of
San Francisco and will be graduating with a degree in Art History and Museum Studies.
Last semester I interned at the University of San Francisco’s Thacher Gallery. I helped
with their online exhibitions, a great experience learning how to navigate fully online
exhibitions.
At the University of San Francisco, the Art History and Museum Studies program offers
a diverse range of skills and experiences that I am excited to bring these skills to this
internship.
I grew up just an hour north of San Francisco in Sonoma County but have enjoyed
living in San Francisco for the past few years. When I am not in a class, I love to cook
and bake for my friends and family. I have also played tennis for the past ten years and
have enjoyed playing more during the pandemic. I am very passionate about traveling
and visiting museums abroad and locally.
On any given day, I can be seen walking my dog Louie and reading a good book.”

Sue Rochman Papers

Posted on August 20, 2019 by David Krah

This is a post from intern Harold Hardin, working on the NEH grant-funded project The San Francisco Bay Area’s Response to the AIDS Epidemic.

Sue Rochman papers, GLBTHS 2005-13 miscellaneous research papers

The Sue Rochman Papers (Collection 2005-13 at the GLBT Historical Society) contain critical information regarding the systematic oppression of incarcerated people living with HIV/AIDS in the first decade of the epidemic. The collection at just over 350 pages consists of interviews, newspaper clippings, and often most compellingly, correspondence from incarcerated people living with HIV/AIDS. Given the ongoing wave of HIV criminalization (a recent famous example being the case of Michael Johnson, who, incidentally, was released this month after spending five-years of a thirty-year sentence in Missouri, for allegedly seroconverting several partners with HIV without revealing his HIV-positive status) Micheal Johnson and Greg’s Smith’s cases among others were rallying cries for HIV/AIDS activists bringing to our collective attention the ongoing histories of HIV criminalization. It is particularly important to look back at the particular ways in which this stigmatization of people living with HIV/AIDS began within the prison system and consider an early case of which the Sue Rochman Papers document. In this way, we can further contextualize our current historical moment in regards to the continuing criminalization of people living with HIV/AIDS–particularly the ways in which black gay men are overwhelmingly impacted by this deleterious trend.
The correspondence between Ms. Rochman and various incarcerated people in several different prison locations (Attica prison in New York, Chino prison in California among others) echo similar findings. The correspondence notes the systematic way in which prison officials valued “security” to the detriment of the lives of incarcerated people living with HIV/AIDS. Confidentiality rights regarding seroconversion status were routinely trampled and ignored at the behest of prison officials. There was little to no basic health information regarding the spread of the disease. Incarcerated people with HIV/AIDS were often isolated in poor conditions, with little medical attention by qualified specialists in HIV/AIDS. The widespread abuse of incarcerated people with HIV/AIDS by prison guards themselves was also well documented. Having the disease in prison not only meant living in such conditions but additionally meant being socially ostracized through officially sanctioned segregation–barred from participation in vocational programs, college classes, and not allowed to have family visits. A jail in Fort Worth, Texas went as far as mandating LGB incarcerated populations wear colored wrist bands to identify their sexual orientation from afar. From such systematic forms of discrimination it is unsurprising then that HIV criminalization was birthed in such an environment.
The Rochman papers document the case of Greg Smith who in 1990 was convicted of attempted murder, assault and terroristic threats. Charges were filed after he allegedly bit and spat on a guard in a New Jersey jail in 1989. He maintained his innocence throughout the trial famously saying after his sentence was read, ‘I never bit an officer, and I’ll say that until the day I die. I may die in the next year or two, but I’ll die proud. I told the truth.” His case was taken up by ACLU via ACT UP prison-activist Judy Greenspan and a significant amount of Rochman papers covers Greenspan’s media campaign and legal filings. Smith, who ultimately died in prison in 2003, was an ACT UP activist, black and gay. His case is viewed as an early example of the compounding effects of race, class, sexual orientation and HIV status-indeed of HIV criminalization.

Not Sanitized for Your Protection: Diseased Pariah News and the Political Uses of Humor

Posted on June 26, 2019 by David Krah

This is a guest post by intern Harold Hardin, who is working on the NEH Grant-Funded Project The Bay Area’s Response to the AIDS Epidemic.

I came across recently a sardonic, humorously bizarre little zine in the Beowulf Thorne papers (GLBT Historical Society, 2003-10) called Diseased Pariah News (DPN). DPN was a zine created during the early 90’s that used gallows humor to humorously educate/entertain mostly gay (often white) cisgender men about HIV/AIDS among other gay men’s health issues. Humor is not something I would immediately associate with AIDS/HIV. Certainly, in the popular imagination AIDS and humor couldn’t be further apart. Queer white, cis, men living with HIV/AIDS in popular media depictions are generally akin to Tom Hanks in Philadelphia: a “noble, suffering AIDS victim”.

Further, many current LGBTQ media consumers tend to shy away from LGBTQ depictions that have overt internalized homophobia/transphobia, straying away from media depictions that might seem to make light of oppressive circumstances in ways that are ultimately self- cannibalizing. Rupaul was famously castigated for having content on her show that was deemed transphobic. Lisa Lampanelli, though not queer, is known for her gallows humor and recently left show business citing, “people in their 20s and 30s weren’t getting into that [insult comedy] tradition”. I spoke to a friend on Facebook about DPN and they echoed a popularly resonant sentiment, “I really don’t like to view historical media/works of art relating to our [queer] community. Because they always carry the hint of shame, of internalized homophobia and transphobia.”

Clearly, we are currently living through a shift in what we find humorous from particular groups of people based on their identities. And to be honest, it shouldn’t be ok for a white, cisgender, straight, man or woman to make jokes about communities that they historically (or contemporaneously, for that matter) oppress. But should queer people with HIV/AIDS be able to laugh at their own lived experiences? If observational comedy is about illuminating the mundane and often untintentionally humorous aspects of our everyday lives then DPN represents to me a group of queers with HIV/AIDS taking this to its’ logical conclusion: finding humor in the everyday lives of queer folx living with HIV/AIDS. Additionally, I think something is foreclosed when we as a queer community rush to quash inter-group humor that may on its surface appear aberrant. Queer people should be able to laugh at their own lived experiences if they so desire, especially, if by laughing, we find a form of resistance while skewering social and political realities that we ultimately find empowering.

New Archives Intern: Ganzolboo Ayurzana

Posted on June 5, 2019 by Charles Macquarie

Today’s post is an introduction from Ganzolboo Ayurzana, one of our current interns here in the Archives. Ganzolboo has actually been working with us for several months now, and he is helping us inventory born-digital collections materials which are currently stored with physical collections so that we can capture the data off of them before they become unreadable.

Hello, my name is Ganzolboo Ayurzana. I am a senior year student at San Francisco State University. I am currently pursuing a double major of computer and math. I am from Mongolia, and I came to this country in pursuit of greater knowledge and career. One thing interesting about me is that I am able to converse in 5 different languages — Mongolian, Korean, English, Japanese, and Russian. Ever since I was young I had a talent for picking up new languages faster than my peers. In my free time I like to play basketball, hang out with my friends, and write computer code. I am also a huge Marvel movie fan, and every movie that came out in theaters I would go watch at least twice; sometimes even thrice. I also love Harry Potter, and I have read the book and watched the movie enough times to know about it all inside out.

I am very much looking forward to getting to know this excellent group of people and learning more about what librarians do.

Brought to Light

Stories from UCSF Archives & Special Collections

Tag Archives: Interns