End to End Gracenote API Metadata Enrichment

Previously SCH Tech built 3 software workflow services in order to produce Gracenote enriched metadata for a Major broadcast client as outlined in the following posts:

The software services are written in C# .net as windows services are able to conduct the following operations:

  • Receive content provider packages and updates then enrich the metadata using the Gracenote API services.
  • Monitor the Gracenote API for metadata updates.
  • Generate Update ADI packages and deliver them to the enrichment service.
  • Enrich the update package and deliver to the customer ingest point.

All the services required are now running in a production environment and successfully delivering enhanced metadata packages into the clients ingest platform, with the addition of the proactive checks we can ensure the images and metadata remain current with the metadata service provider; in this case Gracenote.

Pre-Enrichment end to end workflow diagram

The following Workflow diagram provides a short overview of the Windows services created in order to deliver the project requirements.

Gracenote enrichment workflow

All services are created independent of each other allowing maintenance on each service without disruption to running operations by the other services.


CP Delivered Pre-Enrichment Workflow Overview & Diagram

The primary pre-enrichment workflow is a Windows C# .net service that was created as the primary service for this project.

This service carries out the following basic functions:

  1. Reads service configuration for the following items:-
    • Input, processing, delivery, failure directories.
    • Logfile and location
    • DB Access
    • Gracenote Access details
    • Polling Intervals
    • ADI Provider-specific flag
  2. Validates the incoming package
  3. Calls Gracenote for mapping data and parses
  4. Calls Gracenote for Layer1 Metadata and Images (program specific)
  5. Calls Gracenote for Layer2 Metadata and Images (series/season-specific)
  6. Parses the data and creates a Full or Update ADI Package dependant on incoming ADI Version Major / Minor against the current DB.
  7. Delivers the Enriched package to the correct delivery location.

The Image on the next page aims to show the high-level workflow for the Enrichment delivery from CP to Ingest delivery.

Main Enrichment Services using Gracenote On Api

Pro-active Gracenote Enrichment Workflow Overview & Diagram

The primary Pro-active enrichment workflow is a Windows C# .net service that was created to periodically check the Gracenote API against previously ingested data to ensure the deliveries have the most up to date images and metadata.

To achieve this the workflow services initially take the lowest update id in the database and check this against the Gracenote API with a configurable poll time and return data limit, once the initial call is made a tracking table is updated with the next valid id for all three Gracenote layers and workflow processing will continue from this UpdateId on the next poll.

This service carries out the following basic functions:

  1. Reads service configuration for the following items:-
    • Logfile and location
    • DB Access
    • Gracenote Access details
    • Polling Interval
    • API Results Limit – the number of records allows for each call
  2. Obtain the next valid update id from the tracking DB table, if null take the lowest update id stored in a pre-enrichment database.
  3. Call GN API for Mappings and compare results against the DB and parse only those that match previously enriched items, thus discarding non required data.
    • If matched, then flag the DB table for the unique id that this deliverable requires enrichments
  4. Calls Gracenote for Layer1 Metadata and Images (program specific)
  5. If matched check mapping and layer2 for this current asset using a unique id does not have the requires enrichment flag set.
    • The flag is false = Proceed to set enrichment requires true at layer2
    • Flag in Mappings or layer2 is true = Discard this update as the other layer will carry the update.
  6. Calls Gracenote for Layer2 Metadata and Images (series/season-specific) this is a repeat of point 4 and carries out the same enrichment check against Mapping and Layer 1.
  7. Updates the Tracking and Correct API layer DB Tables
  8. Completes workflow until the next poll period

The Image on the next page aims to show the high-level workflow for the Proactive Enrichment workflow.

Proactive Gracenote Update checker

ADI Generator Service Workflow Overview & Diagram

The ADI Generator workflow is a Windows C# .net service that was created to periodically check the Pro-active enrichment DB for items flagged as “RequiresEnrichment” and creates a correct update package for the ingested deliverable.

This service carries out the following basic functions:

  1. Reads service configuration for the following items:-
    • Input, processing, delivery, failure directories
    • Logfile and location
    • DB Access
    • Polling Interval
  2. Checks the Ingest database every “N” minutes for items requiring enrichment.
  3. Builds a list of the items requiring enrichment.
  4. Iterates the list and carries out the following functions:
  5. Obtain the previous Update ADI from the Ingest database.
    • If update is null obtain the previously enriched ADI
    • If Enriched is null fail the package.
    • Parse the ADI file and remove non required fields to build a default template.
    • Obtain the previous version minor value from the Ingest database.
    • Update the ADI template with an incremented version minor.
    • Package the ADI.
    • Deliver the ADI Package
  6. End workflow and wait for the next poll interval.

The Image on the next page aims to show the high-level workflow for the ADI Generation workflow.

ADI Generator and Delivery service

Conclusion

The project has been challenging at times and SCH Tech has managed to keep up with the requirements and changes effectively to deliver the end to end platform under time and within budget, we are now committed to ensuring any ongoing CR’s or PR’s are effectively dealt with in the same manner.