New Archives Intern: Harold Hardin

Harold Hardin is joining us in Archives & Special Collections this spring to work on finishing the NEH grant-funded project The San Francisco Bay Area’s Response to the AIDS Epidemic. Harold will be helping QA digital objects among other tasks related to the digitization workflow.

Harold Hardin is a current student in Cuesta Colleges’ Library/Information Technology program and San Francisco City College’s Paralegal Studies program. While pursuing a double major in Sociology/Critical Race Ethnic Studies at UC Santa Cruz Harold developed an academic interest in the often hidden and occluded histories of marginalized communities, particularly histories of oppression and resistance. Through their own experiences of political activism at UC Santa Cruz and beyond (#Blacklivesmatter Oakland/ Stockton, GaySHAME SF) Harold has insisted on moving iteratively between theory and praxis: centering an intersectional feminist analysis of power. 

These analytical lenses and political participation increased Harold’s consciousness regarding the fundamental ways in which access to information (particularly personal/community histories) profoundly shapes participation in our democracy (or lack thereof). Harold is interested in the nuances of political participation and uncovering the innumerable sites of quotidian resistance! Therefore, Harold sees their internship within UCSF’s AIDS History Project as not only a unique privilege to work toward increasing community access to Queer history, but also, and importantly,  an extension of the deeply personal (political) work of (re)understanding their multiple positions within (and outside) of the Archives.

UCSF Archives & Special Collections awarded grant to digitize historical California public health materials

UCSF Archives has been awarded a California Revealed grant to digitize historical reports, newspapers, yearbooks, and other publications that document the development of medicine and public health in California and the Bay Area and various activist and community roles in that history. The publications to be digitized include The Cap & Seal yearbook of the San Francisco General Hospital Nursing School, the Annual Reports of the San Francisco Nursery for Homeless Children, the Annual Reports of St. Mary’s Hospital, the Bay Area Health Liberation News Newspaper, the Annual Reports of the California Women’s Hospital, the Clarion journal of the SF Department of Public Health Tuberculosis division, and the Annual Reports of St. Luke’s Hospital.

These materials contain fascinating and valuable primary source documentation of the development of medicine and public health in California. Included are countless historical images of hospital spaces, technologies, and equipment; historical data on hospital patients, surgeries, and finances; historical patient voices through writings and illustrations; and evidence of the broad and diverse movement building which was a part of progressive public health development in the civil rights era.

The project will include 80 total volumes of the items outlined above. Having the digitization provided for free by California Revealed is equivalent to an estimated $5500 of actual digitization costs. The digitized materials will be published to Calisphere for public access and download.

The front page of the Bay Area Health Liberation News newspaper with an article about medical repression in prisons.

About California Revealed:
California Revealed is a State Library initiative to help California’s public libraries, in partnership with other local heritage groups, digitize, preserve, and provide online access to archival materials – books, newspapers, photographs, audiovisual recordings, and more – that tell the incredible stories of the Golden State.

Internship Opportunities

UCSF Library Archives and Special Collections has 2 new internship opportunities.

Archives Intern for AIDS History

The San Francisco Bay Area’s Response to the AIDS Epidemic: Digitizing, Reuniting and Providing Universal Access to Historical AIDS Records.

The Archives Intern for AIDS History will be assigned various tasks to assist in completion of the project including performing Quality Control checks on digitized papers, digital objects and metadata. Candidate should be a student or recent graduate from a library or information science program, preferably with a concentration or interest in archives and special collections. Students of public history, and history of health sciences are also encouraged to apply. This is a part time temporary appointment.
Department: Archives and Special Collections
Rank and Salary: Library Intern – $15/hr
Term: 150 hours Fall 2018 – Spring 2019

Project Description

The Archives and Special Collections department of the University of California, San Francisco (UCSF) Library, in collaboration with the San Francisco Public Library (SFPL) and the Gay, Lesbian, Bisexual, Transgender (GLBT) Historical Society, has been awarded a $315,000 implementation grant from the National Endowment for the Humanities. The collaborating institutions will digitize about 127,000 pages from 49 archival collections related to the early days of the AIDS epidemic in the San Francisco Bay Area and make them widely accessible to the public online. In the process, collections whose components had been placed in different archives for various reasons will be digitally reunited, facilitating access for researchers outside the Bay Area.
 The 127,000 pages from the three archives range from handwritten correspondence and notebooks to typed reports and agency records to printed magazines. Also included are photographic prints, negatives, transparencies, and posters. The materials will be digitized by the University of California, Merced Library’s Digital Assets Unit, which has established a reputation for digitizing information resources so that they can be made available to the world via the web. All items selected for digitization will be carefully examined to address any privacy concerns. The digital files generated by this project will be disseminated broadly through the California Digital Library, with the objects freely accessible to the public through both Calisphere, operated by the University of California, and the Digital Public Library of America, which will have an AIDS history primary sources set.

Skills and experience desired:

  • Strong candidates will be detail oriented and possess excellent organizational skills
  • Proficiency  with MS Excel and Google spreadsheets
  • Proficiency with document sharing and cloud computing services (Google drive, Box)
  • Experience with digital asset management systems
  • Ability to work independently
  • Ability to lift boxes weighing up to 40 pounds.

Hours and Location:

The timing of the internship is flexible, but should be carried out during the Fall of 2018 and ending early Spring 2019,  based on applicant and institutional commitments.  Up to two 8-hour days per week for 10-12 weeks. Work will be performed onsite at the library, though offsite work is possible.

Stipend:

A stipend of $15/hour is available for the internship. 

To Apply:

Applications for the UCSF Archives & Special Collections Internship, including a cover letter, resume, and names/contact info of two references should be sent to 
David Krah, Project Archivist 
UCSF Archives and Special Collections
University of California, San Francisco
530 Parnassus Avenue
San Francisco, CA 94143-0840
Apply for this position

Digital Processing and Implementation Intern

The Digital Processing and Implementation Intern will assist the UCSF Digital Archivist with various aspects of the Digital Archives program as they are implemented and brought online for the first time. Potential projects include:

  • Testing digital forensics and processing hardware and software being implemented in the digital forensics lab.
  • Compiling inventory of physical archival collections containing digital media, and pulling collections and identifying, counting, and cataloging digital media present.
  • Disk-imaging digital media removed from collections and transferring data to library storage systems.
  • Creating metadata about digital media being processed in digital forensics lab, editing metadata for various digitization or cataloging projects.
  • Operating scanning equipment to digitize archival collections for patron and researcher use.
  • Processing digital collections under the supervision of the Digital Archivist, including finding aid and container list creation and manipulation of access copies of born-digital content to create access-ready versions of collection.
  • Researching computer tools and systems for management and preservation of digital objects, and compiling and reporting on capabilities, requirements, dependencies, etc. of these utilities.
  • Participate in staff meetings, assist with writing blog posts, and help with reference/duplication requests.
Department: Archives and Special Collections
Rank and Salary: Library Intern – $15/hr
Term: 150 – 200 hours Fall 2018 – Spring 2019

Location

UCSF Library and Center for Knowledge Management,
530 Parnassus Avenue, San Francisco, CA 94143-0840

Work Type

Archival Processing, Information Technology, Computer Science

Work To Be Done

On site, with occasional opportunities to work from home or other location

Desired Qualifications

  • Experience with ArchivesSpace, Nuxeo or other archival collections management software
  • Experience with or interest in digital preservation, digital file formats and media, computer science, or history of computing technologies
  • Experience with or interest in digital forensics in archival collections and various digital forensics tools, such as FTK Imager and BitCurator
  • Familiarity with scripting, computer programming in any language, Unix.
  • Excellent analytical and writing skills
  • High level of accuracy and attention to detail
  • Ability to work independently
  • Ability to lift boxes weighing up to 40 pounds

Stipend

A stipend of $15/hour is available for the internship. The internship is intended for those who are currently enrolled in an undergraduate/graduate program.

Hours

Up to two 8-hour days per week for 10-12 weeks. Specific on-site hours are negotiable, but must be completed between 8:00 a.m.  and 5:00 pm Monday through Friday. Start and end dates are flexible.

Application Process

Please submit a letter of interest, a current resume and contact information for two professional references to:

Charles Macquarie
Digital Archivist
UCSF Archives and Special Collections
University of California, San Francisco
530 Parnassus Avenue
San Francisco, CA 94143-0840

Apply for this position

The UCSF Library is committed to a culture of inclusion and respect. We embrace diversity of thought, experience, and people as a source of strength which is critical to our success. We encourage candidates to apply who thrive in an environment which celebrates and serves our diverse communities.

Equal Employment Opportunity
The University of California San Francisco is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran or disabled status, or genetic information.

About UCSF
The University of California, San Francisco (UCSF) is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It is the only campus in the 10-campus UC system dedicated exclusively to the health sciences.

About UCSF Archives and Special Collections
UCSF Archives & Special Collections is a dynamic health sciences research center that contributes to innovative scholarship, actively engages users through educational activities, preserves past knowledge, enables collaborative research experiences to address contemporary challenges, and translates scientific research into patient care.

The San Francisco Bay Area’s Response to the AIDS Epidemic: 1 year update on the National Endowment for the Humanities Implementation Grant

Archives and Special Collections has just submitted its annual report to the National Endowment for the Humanities on the collaborative mass digitization grant The San Francisco Bay Area’s Response to the AIDS Epidemic. 

At the one year point of The Bay Area’s Response to the AIDS Epidemic, the consortium of UCSF Library Archives & Special Collections, San Francisco Public Library History Center/Hormel LGBTQIA Center, Gay, Lesbian, Bisexual, Transgender Historical Society, and UC Merced have made significant headway towards our goal of digitizing and publishing 127,000 pages from our various AIDS History collections.

To date we have published seven complete collections on Calisphere, and we have scanned and published the poster component of UCSF’s AIDS History Project Ephemera collection. Thirteen other collections have nearly completed the digitization process and are undergoing quality control checks before being harvested into Calisphere.

The Ultimate Point: Shooting Up and Sharing Needles Puts You at Risk for AIDS. This Fact May Save Your Life!

The Ultimate Point (SF AIDS Foundation). AIDS History Project Ephemera Collection, MSS 2000-31

40,518 pages of materials have to date been uploaded to the Nuxeo Digital Asset Management System we use for managing and publishing to Calisphere.  Some of these have gone to active publication, some are still undergoing quality assurance (QA) procedures. An additional 35,061 pages have been scanned, but have yet to be ingested into the DAMS.

We have also given talks at four library and archives conferences in the past year to share details about our project.

In the coming year we will continue digitizing and publishing collection materials to Calisphere.org and begin planning online exhibits for Calisphere and Digital Public Library of America that will serve to unite and interpret the collections across our partnering institutions.

AIDS Legal Referral Panel's AIDSLaw Conference. Friday November 11, 1988.

AIDSLaw Conference 1988. AIDS Legal Referral Panel Records, 2000-46, Box 2, Folder 8 (GLBTHS)

Digitized collections currently online:

  1. ACT-UP Golden Gate Records, 1988-1993, MSS 98-47 https://calisphere.org/collections/308/ 
  2. Barbara Cameron Papers, 1968-2003 (SFPL GLC 63) https://calisphere.org/collections/27002/
  3. Shanti Project Records, 1982-1994, MSS 98-48 https://calisphere.org/collections/19989/
  4. AIDS Legal Referral Panel (ALRP) records, 1984-2000, (#2000-46) GLBT HS  https://calisphere.org/collections/469/
  5. Bobbi Campbell Diary, 1983-1984, MSS 96-33 https://calisphere.org/collections/3684/
  6. Mobilization Against AIDS Records, 1984-1995,
    MSS 95-03 https://calisphere.org/collections/14922/
  7. People vs. Owen Bathhouse Closure Litigation, 1984-1987
    (SFPL SFH 31) https://calisphere.org/collections/26990/
  8. AIDS History Project — Ephemera Collection, 1981-2002, MSS 2000-31 (posters) https://calisphere.org/collections/466/

National Endowment for the Humanities

The San Francisco Bay Area’s Response to the AIDS Epidemic has been made possible in part by a major grant from the National Endowment for the Humanities: Exploring the human endeavor.

 

Experiments with Digital Tools in the Archives — OCR

Working on digital “stuff” in the archives is always fascinating, because it blurs the borders between digital and physical. Most of the work the takes up my time is at these borders. Physical stuff requires lots of human touches to transition to “digital,” and digital stuff similarly requires lots tending by humans to ensure that it is preserved physically. After all, the 1s and 0s are stored physically somewhere, even if on the cloud or in DNA.

We’re currently working on several projects to convert physical materials to digital text. The huge quantities of rich and complicated textual material in archival collections is full of potential for use as data in both computational health research and also digital medical humanities work, but to be usable for these kinds of projects it needs to be converted to digital text or data, so that it can be interpreted by computers. To get to this point the documents must be scanned, and the scanned documents must either be transcribed, which can be immensely labor intensive, or converted directly by computers using a software that can perform Optical Character Recognition, or OCR. One of our projects using OCR to extract text from a document provides a fascinating look into the world of computer vision.

A pen and ink illustration of the lungs and a lymph gland from the Ralph Sweet Collection of Medical Illustrations

An example of the illustrations in the Ralph Sweet Collection

The Ralph Sweet Collection of Medical Illustration contains extraordinary examples of the work of one of the most renowned medical illustrators in the United States, so we’re working on digitizing the collection and putting it online. To do this we need to have detailed metadata — the kind of information you might expect to find in a catalog record, title, date, author — about each illustration. Currently this metadata for the Sweet Collection exists only in the form of printed index that was written on a typewriter. We can scan the index, but we do not have the labor to transcribe each of the 2500 or so entries. This is a job for OCR.

The image below shows what a page of the Ralph Sweet index looks like. This is the metadata that we want to be able to extract and turn into digital text so that it can be understood by a computer and used as data.

A page of an type-written index of the Ralph Sweet Collection, showing metadata about each illustration in the colleciton.

A page of the index for the Ralph Sweet Collection.

One of the first problems we encountered in attempting to extract text from this document is a classic difficulty of computer vision. As English-speaking humans, we know by looking at this document that it contains three columns, and that the order in which to read the page is top to bottom by column, starting on the left and moving right. To a computer however, it is simply a page full of text, and there is no way to know whether or not the text is broken into columns or whether each line should be read all the way across the page. This simple task presented a difficulty for most of the software that we tested, but we found one software which could identify these columns easily. The software is called Tesseract, and it was actually developed in the 1980’s but continues to be a good open-source tool to perform OCR.

If we plug the above page into Tesseract, we get mostly recognizable text, which in itself is pretty miraculous when you think about it. Looking at the text though, it quickly becomes clear that it is not an exact transcription of what’s on the page. There are misspellings (“Iivev”), and some chunks of text have been separated from the entry in which they should appear (“horizontal”).

An image of the text-output of the software tesseract showing errors in transciption.

An example of the text extracted from the Ralph Sweet Collection Index by Tesseract.

Digging into the way that Tesseract (and OCR software more generally) works can help us begin to understand why these errors are cropping up. Plus, it looks really cool.

OCR programs have to go through a set of image manipulation processes to help them decide which marks on the page are text — and hence should be interpreted — and which are other marks that can be ignored. This all happen behind the scenes, and usually this involves deciding what the background parts of the image are and blurring them out, increasing the image contrast, and making the image bi-tonal so that everything on the page is only black or white. Then, the computer can trace the black pixels on the image and get a series of shapes which it can use to begin attempting to interpret as text. The image below shows the shapes that Tesseract has identified as letters and traced out for interpretation. Each new color indicates that the computer believes it has moved on to a new letter.

A page of colorful text on a black background illustrating the text that has been automatically traced from the Ralph Sweet Index by the computer program Tessearact.

The result of Tesseract tracing the letters it has interpreted. Each new color is something that’s been identified as a new letter.

Interestingly, when comparing the computer tracing of the letters to the original image you can see that Tesseract has already made the assumption that the black spaces from the three-hole punch in the top of the page are not letters, and thus it should not bother to trace them. Lucky for us, that’s correct.

Next the computer has to take all these letters and turn them into words. In actual practice it’s not quite this simple, but basically the computer iterates on each letter identification that it believes it has made by testing whether or not that word is in its dictionary, and thus whether or not it is likely to be a word. If the combination of letters that the computer thinks it sees are not a word, then it will go back and make a new guess about the letters and test whether or not that’s a word, and so on. Part of this whole process is to chunk the letters into words using their actual spacing on the page. Below you can see an image of how Tesseract has begun to identify words using the spaces between lines and letters.

A view of a page of the Ralph Sweet Index showing each word as a blue rectangle encompassing the space taken up by that block of text against a black background -- the "word" output of the OCR program Tesseract.

The “words” that the OCR software has identified on the page. Each blue rectangle represents a space that Tesseract has marked as containing a word.

In addition to checking the word against the dictionary though, most OCR programs also use the context of the surrounding words to attempt to make a better guess about the text. In most cases this is helpful — if the computer has read a sentence that says “the plant requires wader” it should be a relatively easy task to decide that the last word is actually “water.” In our case though, this approach breaks down. The text we want the computer to extract in this case is not sentences, but rather (meta)data. The meaning of the language has little influence on how each individual word should be read. One of the next steps for us will be trying to figure out how to better instruct Tesseract about the importance of context in making word-identification decisions (i.e., that it’s not important).

Finally, as the OCR software interprets the text it also identifies blocks of words that it believes should be grouped together, like a paragraph. Below you can see the results of this process with Tesseract.

A view of the different elements of tesseract's text identification showing letters traced in primary colors and contained in yellow bounding boxes, words set against blue rectangles outlining the space they encompass, and blocks of text outlined in larger bounding boxes and numbered -- all of this set against a black background.

This view shows all of the elements of Tesseract’s word identification combined. Text has been traced in color, separate letters are contained in bounding boxes, words are contained in blue rectangles, and blocks are contained in larger bounding boxes and are numbered (though the numbers are a bit difficult to see).

A line has been drawn around each block of text, and it has been given a number indicating the order in which the computer is reading it. Here we can see the source of one of the biggest problems of the OCR-generated text from earlier. Tesseract is in-accurately excluding a lot of words from their proper blocks. In the above photo, the word “Pen” is a good example. It is a part of block 20, but it has been interpreted by the computer as it’s own block — block 21 — and has been set aside to appear in the text file after block 20. Attempting to solve this problem will be our next hurdle, and hopefully we can catch you up after we are successful.

Using OCR to extract text from digital images can be a frustrating endeavor if accuracy is a necessity, but it is also a fascinating illustration of the way computers see and make decisions. Anytime we ask computers to perform tasks that interface with humans, we will always be grappling with similar issues.

A Look Back: Black History Month Activities of the UCSF Black Caucus

This February, we’re taking a look back at some of the Black History Month activities organized by the UCSF Black Caucus. The Black Caucus was formed in 1968 as a forum for students, faculty, and staff at UCSF to discuss issues of race and advocate for change. You can learn more about the group’s history here and current information here.

The newsletter and flyers featured here are included in the UCSF Archives digital collection of the UCSF Black Caucus records.

We will soon be digitizing more material from the Black Caucus records through the support of California Revealed, a California State Library-funded initiative to digitize, preserve, and serve historically significant Californiana in partnership with archives and other repositories across the state.

UCSF Awarded California Revealed grant

We’re happy to announce that we have been selected to digitize several of our collections through the latest round of the California Revealed digitization granting program. California Revealed is a California State Library-funded initiative to digitize, preserve, and serve historically significant Californiana in partnership with archives and other repositories across the state.

For this latest round of digitization, which will begin in April of this year, we will be digitizing our Tales and Traditions scrapbooks, several of our scrapbooks documenting the experiences of Hospital Unit 30 in World War II, and several folders from the records of the Black Caucus, specifically production materials for their Black Bulletin newsletter.

All of these collections combined document some fascinating slices of California history where it intersects with the history of UCSF. Since UCSF is one of the older UCs (though it has changed form several times), it should come as no surprise that they intersect a lot! Just as a sample, these materials contain histories of the development of a public health program in the state of California, an account of California survivors of WWII war crimes such as Nazi medical experiments and the dropping of the Atomic Bomb of Hiroshima, the development of one of the first summer camps created specifically for people with diabetes, the medical questions that were at the beginning of the California drug craze, and the development of the civil rights movement in California and the intimate ties between organizers who were employed at UCSF and the larger nationwide movement.

We’re excited to get these materials digitized and available to everyone, no matter their location. We’ll announce when they’re online.

New AIDS History Project Collections Online

Materials newly digitized as part of our NEH grant-funded project The Bay Area’s Response to the AIDS Epidemic are available online on Calisphere.

Most of theses collections represent a “mass digitization” approach to putting materials online. In most cases (Ephemera Collection excepted), the collections are scanned at the folder level. The objects on Calisphere correspond to the folder titles you see in the collection guides found on the Online Archive of California. 

These objects contain a multi-page pdf of all the papers in each folder. Click the image to download a keyword-searchable pdf.

In some cases, when a folder title actually refers to a group of several folders, you’ll see multiple images (one for each folder) in a carousel below the main image, such as People vs. Owen Bathhouse Closure Records, Sex Clubs-Bathhouses Subject Files.

These collections are ready to research, plenty more on the way.

Bobbi Campell Diary

AIDS History Project Ephemera Collection

ACT-UP Golden Gate

People vs. Owen Bathhouse Closure (San Francisco Public Library)

Barbara Cameron Papers (San Francisco Public Library)

AIDS Treatment News Digital Collection

We’re announcing the publication of the AIDS Treatment News (ATN) digital collection.

The collection includes a sampling of the ATN records that we house at UCSF. ATN is a publication created by John S. James that investigates and reports on both conventional and experimental treatments for HIV/AIDS and related social and political issues. It began publication in 1986.

This collection is made possible through the generous support of an NHPRC grant from the National Archives.

The digital collection includes ephemera, photographs, and documents that illustrate AIDS-related activism and ATN’s operations.

Notable material includes images of the 1988 protest at the Federal Drug Administration (FDA) offices in Washington, DC, flyers related to the NAMES Project AIDS Memorial Quilt, and ACT UP San Francisco material.

View the complete digital collection on Calisphere. Contact us to visit the reading room and work with the ATN records.