BPS School Desegregation Project history, Widget and a Website (part 3)

bps_deseg_proj

(This is part 3: part 1 is here, part 2 is here, part 4 is here)

Scanning projects largely wrapped up in 2016.  Slowly but surely, (thanks largely to the tenacity of Jessica Sedgwick, then the BLC metadata consultant) our repositories began to implement a technical connector that allowed our individual repositories to connect with Digital Commonwealth/DPLA.

A COLLABORATIVE HOME?

Although Northeastern was coordinating the construction of this collection, we tried to follow a distributed ownership model, starting with our first piece of digital infrastructure, our blog.  We posted each entry on “What’s New,” Snell Library’s blog, but there was an understanding that content belonged to the community and could be repurpused by partners on their own institutional blogs.

With the digital collection coming together, the group started to talk about portals, connectors, and how we can make this a true collection, not just several thousand contributed items to Digital Commonwealth/DPLA.  We discussed purchasing and owning separate digital infrastructure collaboratively, but as the project was designed as a “lightweight, nimble project that attempts to lay the technical and descriptive groundwork for cross-institutional collaboration through the technical infrastructure of the DPLA and Digital Commonwealth” , we decided against it.  Separate digitial infrastructure was neither lightweight nor nimble.  In the end, we decided to create a widget that could live on multiple websites and use Northeastern’s already existing web tools to create a portal to the collection.

WIDGET TECH

To maintain the distributed ownership model, the group decided to create a tool that could search the collection directly from any website– essentially bring the collection into the organization’s own website rather than partners pointing to an external resource.

We  looked at a widget created by Dean Farrell and Josh Wilson that allows a user to easily install a DPLA-branded search box on any website.  When you type a word or phrase into the box, the widget goes to the DPLA and automatically populates a search. After clicking ‘search,’ the next screen that opens is a DPLA page with the results.  Could this widget be modified to automatically populate desegregation search terms in addition to what has been typed in the box? At DPLAFest in 2016 (presenting on the Deseg project), I met Audrey Altman from DPLA, who played with the code.  By using the search term ‘cat’, she determined that it was, in fact, possible.   In 2017, Northeastern’s technical team (Eli Zoller, Ernesto Valencia) forked the github code, modified it, and created a working widget for the project. 

SEARCH STRING

We then needed to determine the search terms that would bring up desegregation material from DPLA.  Instead of solely using our common search string, we decided to riff on the https://www.umbrasearch.org model that uses a set of individual words to search for items rather than an LCSH subject heading.  The group first constructed a boolean search (school* OR educat*) AND Boston AND (desegregation OR segregation OR integration) that we thought would retrieve most items, including those from our collections.  In testing, we found a glitch– we recieved an expected number of results using Digital Commonwealth’s search tool and an unexpected result from DPLA, either a higher or lower number depending on which set of terms was listed first.

We reached out to Mark Breedlove at DPLA, who confirmed our suspicions– the search tool DPLA uses, (Elasticsearch, and Lucene) doesn’t honor “AND” and “OR” and “( parentheses )” so our boolean markers weren’t being observed. Although not ideal, we modified our search string to [ school* + integration + Boston ] 

Mark suggested that we use an API instead of using the DPLA search box– it would allow us to use a more robust search string including fielded data (DPLA doesn’t have an “Advanced” search function, just API availability.  His suggested search to start was:

curl -X GET \   ‘https://api.dp.la/v2/items?api_key=[YOUR API KEY] &q=integration%20OR%20desegregation&sourceResource.spatial=boston&sourceResource.subject=school%20OR%20schools&fields=id%2CsourceResource.title%2CsourceResource.description&page_size=50′

The use of an API does allow robust searching and would allow us to retrieve much more specific results.  Unfortuantely, it requires the results of that search to be embedded into an owned website, which was not part of the initial scope.

WE MAKE A WEBSITE ANYWYAY:

For all of its wonderful aggregating power, DPLA and Digital Commonwealth are not set up for full-text searching OCRed collections, something that our own Digital Repository is capable of.  We decided to make use of the tools, infrastructure built by Northeastern’s Digital Scholarship Group (CERES) to make a Northeastern portal to the collections.

Although a Northeastern-specific portal was the primary reason for bpsdesegregation.library.northeastern.edu to be built, it started to serve as the collaborative collection portal, and later partners agreed to use it also as the WWW place for information about the collection to live.  Currently, the website is where information about the widget lives, and serves as a gathering place for a growing set of descriptive, information, and pedagogical tools.

(This is part 3: part 1 is here, part 2 is here)

Advertisements

BPS School Desegregation Project history– Planning and Building (part 2)

(This is part 2: part 1 is here, part 3 is here, part 4 is here)

bps_deseg_projWith BLC funding secured and a search for a techical consultant underway, BLC archival partners (University Archives and Special Collections at UMass Boston, the Northeastern University Archives and Special Collections, The State Library of Massachusetts’ Special Collections, and Boston College’s John J. Burns Library of Rare Books and Special Collections) met and started formulating the project in January 2015.  The group determined that we would employ a ‘big tent’ approach to collection building, and invited non-BLC institutions to join as non-fiscal partners. The Moakley Archive and Institute at Suffolk University, the Boston City Archives, and WGBH) were added later that spring.  We announced the project, including our arguments for collection creation in a Blog Post. 

SCOPING AND SCANNING

A full list of collections that were contributed to the collection is here, including partners added later in the project.  My initial idea for the project was to start in 1954 and end in 1974 when the court order was implemented.  This is what we intended to focus on at Northeastern.  The group agreed to stretch that date range into the 1980s and 1990s for two main reasons.  Pragmatically, many institutions had strengths in that time period.  Philosophically, students and teachers doing the day-to-day work of integration was equally as challenging as getting to an integration decision.  Once the project was scoped, we determined that each institution would have the ability and freedom to contribute the most relevant items based on their collection strengths.

BUILDING via SUBJECT HEADING

The group determined that metadata would be the glue that would bind our collections together in Digital Commonwealth/DPLA. More on the decisions on this topic are reflected in this Blog Post written by Jessica Sedgwick.  We considered a non-LCSH option that could mark records that were part of a BLC-specific project, but ultimately decided to use the single subject heading “Segregation in education — Massachusetts — Boston — History” as a determinant of participation.   Our reasoning was twofold; one subject heading is a very low bar for participation, and it would also allow the collection to grow easily if other additional collections decided to participate.  This “everyone can participate” decision risks significant duplication and will most certainly invite unevenness and irrelevant items– a DPLA participant might tag something with ‘our’ subject heading that we might not consider ‘significant’ or even relevant or even deployed mistakenly.  A counter argument is that if we wanted to create a static, even, balanced collection we could have followed the blueprints of the many self-contained e-collections on varying subjects.  DPLA describes searching in its collections as like “drinking from the fire hose,” overwhelming, but offering the kind of varied perspectives and collection strengths that a curated collection can’t.

SCANNING and METADATA

The group shared internal scanning policies and procedures.  Some institutions provided folder-level metadata, others item-level.  Participants used several different metadata standards, including Dublin Core and METS.   Scanning standards (dpi, color/greyscale, sizing) varied.  As this variety is also reflected in Digital Commonwealth/DPLA, we determined that we would also apply a low bar/high participation policy to our scanning and metadata standards. Thus, we did not require or suggest any kind of standard metadata, scanning standards or levels of description.

In May of 2015 Northeastern hired Sociology PhD candidate Meghan Doran (formerly of BBDP) and UMass Boston Public History MA student Corrinne Bermon (who worked on UMB’s Omeka site) to work on selecting, scanning, and providing basic metadata for items.   Corrinne describes going through the collection in this Blog Post. 

The group also talked about putting together an external personography that collated non-LCNAF name authorities in list form.  However, recent scholarship has described and problematized library and archival technical structure structure (and LCNAF) for its inability to understand relationships between people, organizations, and families.  A new standard has been emerging to address that, EAC-CPF.  In the summer/fall of 2016, graduate student Elizabeth (Betts) Coup initiated a pilot instance instance for our collections as an internship.  Eventually the goal is to incorporate that information into our Northeastern and our partners’ descriptive practices.  Betts’ blog post about her project is here. 

(This is part 2: part 1 is here, part 3 is here)

BPS School Desegregation Project history– Beginnings, 2006-2014 (part 1)

bps_deseg_proj

(This is part 1: part 2 is here, part 3 is here, part 4 is here)

February 1, 2018, we launched the BPS Desegregation Portal and I thought I would spend a few minutes talking about the origins of the project.

My predecessor served on the advisory committee of the “Lower Roxbury Black History Project,” which ran from 2006-2009.   It was not an archives-driven project; it was managed and funded through City Affairs.  Project funding ran out unexpectedly in 2009, so when I arrived at Northeastern in June of 2013, the interviews, digitized images, and photographs were sitting on an external hard drive on a shelf in the Archives, uncataloged and unfindable.  Community members often asked “where is the oral history project?” when I met with them.

Library administration and I applied for some money to expand on this project and create an electronic home for it called ” The Lower Roxbury Digital Library.”  The funder we approached was not interested– they asked us to come up with project that more closely resembled the Shoah Foundation— an oral history collection of national importance.  Although the reason for this rejection rankled (Local history is the foundation of our work!), it got me thinking about what about our collections could qualify as of “national importance,” despite having a hyper-local focus.  I surveyed our collections and determined that some of our most-used collections related to school desegregation in Boston– commonly known as ‘busing.’

And there was an anniversary– 2014 marked 40 years since the Garrity decision court ordered schools to desegregate.  There were commemorative events and press– Boston City Council hearings, articles in the Boston Globe, the Mayor talked about his own personal experiences.  To gague the temperature of a project on campus, I convened Northeastern faculty who were already using Boston’s school desegregation troubles pedagogically, from English, Education, History, Public History.  Meghan Doran, then a Northeastern Sociology Ph.D.  student, introduced me to the Union of Minority Neighborhood’s Boston Busing Desegregation Project, whose work completely altered the direction/thrust of our ideas (concepts of the long civil rights movement, focus on Boston’s education civil rights heroes, “it’s not the bus, it’s us”).    The work continued: I was contacted by Josue Sakata, a curriculum designer for BPS, “was there material he could include in a new unit on Desegregation?”  Fabulous NU student Martha Pearson started working on an independent study, “Boston Before Busing” which made use of new tools being built by the newly formed Digital Scholarship Group of the library.  UMass Boston’s Public History program focused on school desegregation, creating another wonderful exhibit “Stark and Subtle Divisions.”, which introduced me to the wonderful collections at the Boston City Archives.

I mentioned all of this churn, and the possibility to colleagues at the Boston Library Consortium (BLC).  We brainstormed– wouldn’t it be great for this information to be widely available and become part of public discourse?  And then, could we unite our collections in one place? And if that was even possible, where?  DPLA?

I did some Digital Commonwealth/DPLA searching and realized that the only material related to Boston’s desegregation fight were from WGBH. There were other collections, but they were all  from the south and western parts of the country.  This wasn’t surprising– the bulk of the most relevant collections were at Northeastern, UMass Boston, Suffolk University, and the City and State Archives– and none of us were adding metadata to Digital Commonwealth/DPLA.

We applied to BLC for two separate but related grants– technical assistance to help assist BLC institutions to develop the technical connection  from our repositories to DPLA, and scanning funds for the school desegregation project.  The goal of the two simultaneous grants would result in

  • BLC institutions connected to DPLA
  • Determining the feasibility of developing a roadmap for other collaborative collections.

If both projects were successful, it would also result in a collaborative desegregation collection. BLC funded both projects.

The press release:

https://bpsdesegregation.library.northeastern.edu/wp-content/uploads/2018/01/BPS-Desegregation-Collection-Press-Release-1.pdf