Project Origin
Building trust at the origin
Explore the tech behind Project Origin
These days it’s hard to distinguish synthesized or manipulated media from real content. Some of this is due to the breakneck speed of how social media operates but it also has to do with how good these “fakes” really are. To the naked eye synthetic media is almost indiscernible. Ironically, AI itself – the very technology that is making it easier to manipulate media – won’t be able to detect it either – casting a shadow of doubt on the legitimacy of what we see or hear.
One promising direction is to provide viewers with reliable information about the source or provenance of a media object through a verifiable and trustworthy media authentication service. Project Origin has been created for this effort: an alliance of leading organizations from the publishing and technology worlds working together to create a process where the provenance and technical integrity of content can be confirmed. Establishing a chain of trust from the publisher to the consumer. This would allow consumers to make informed decisions about a media object’s trustworthiness. For example, knowing the source of a news item lets a consumer know that the work meets the publisher’s editorial standards.
This is technically possible through a system called Authentication of Media via Provenance (AMP).
AMP allows media publishers to certify content at the time of capture or publication. AMP also allows publishers to add tamper-proof metadata: for example, GPS coordinates or a name for a video clip. We use the term manifest for the digitally signed tamper-protected digital envelope for the media and metadata. AMP manifests can be used on their own or can be registered with a public or permissioned ledger (blockchain) service such that provided by the Confidential Consortium Framework (CCF), for the upmost trust and transparency.
We expect that manifest checking could be built into web sites, browser, and social media applications to allow user to be better informed about the media that they are viewing.
AMP system overview
The following diagram shows how AMP integrates into an existing capture, production, distribution, and rendering pipeline: a content provider uses an AMP-enabled service – or AMP Authoring Tools – to create signed manifests at one or several points in the pipeline. The manifests can authenticate the media as original work or can indicate that a media object was edited and assembled from earlier work (from the same or from other publishers.)
The AMP manifests can be embedded in the media objects themselves or can be uploaded to one or more databases. AMP manifests can also be registered on the AMP ledger service.
The manifests can either be created locally by the publisher using AMP tools, or manifest creation can be integrated into a cloud-hosted media distribution workflow.
In some scenarios, media follows from publisher to consumer without changes. However, increasingly, media is shared on social media, and these platforms and services transform and optimize the media to support differing device, screen, and network conditions. For example, video can be encoded for a range of screen sizes and resolutions, and a variety of network conditions. In this case, the transformed video must be re-authenticated – either by the original publisher, or by the social media platform.
AMP manifests
The manifest is the central data structure in AMP. It authenticates media objects (including various cryptographic hashes of their encodings) and binds them to their publisher-provided metadata. Manifests support simple media objects, streaming media, progressive download, and adaptive bitrate streaming.
- AMP manifests allow any metadata to be embedded. The metadata can be simple text or binary data (for example, a JPG thumbnail.) AMP manifests can also be used to protect exiting metadata tags, such as EXIF.
- A manifest can also record the attribution of derived works through “back-pointers” to one or more source objects, as well as descriptions of how the original works were transformed.
- AMP manifests can be distributed with the media files themselves or stored in centralized or distributed databases.
Authoring tools and cloud services
People use tools to create images and videos. These tools range from a camera app on a cellphone, to sophisticated image and video editing apps. An individual might then share the media objects using a social media app or services; a news organization might use Azure Media Services to process the video for publication on the web. Existing tools allow people and organizations to add tags and other metadata but resulting media tags and objects can easily be changed in misleading ways.
Future tools will be enhanced with AMP technology to use create cryptographically protected manifests for media objects and their tags so that users know exactly where media comes from and who processed it. Note that AMP does not stop people changing the tags or imagery, but it does let consumers know that changes have been made.
AMP clients
People view media using web sites and apps. AMP-enhanced players and services will compare media object with the associated manifests to see if the video is unchanged. If the manifest is properly signed, and the video fingerprints in the manifest match the video being played, then the publisher and tags will be displayed (perhaps behind an “info” button or icon.) If the video does not match the manifest or the signature is invalid a warning may be given.
It will take time before the majority of Internet media is protected using AMP, but we hope that journalists and news organizations will take the lead towards increasing the trustworthiness of media on the web.
Anatomy of the AMP manifest
Media binding
In addition to metadata, AMP manifests authenticate media objects; we call this media binding. AMP supports a variety of ways to authenticate media.
The simplest technique uses cryptographic hashing: a technique for producing a short “fingerprint” for a document, with the characteristic that it is extremely unlikely that any other document hashes to the same value. With it, even the tiniest change to a media will result in a different fingerprint: this is good – because impossible for an attacker to create content that matches the manifest but is not the original – but also bad – because transformations like recompressing without changing the meaning or appearance will break the binding between the content and the manifest. An alternative to cryptographic hashing is called soft hashing: a fingerprint for a media object that survives modest changes – like recompression.
AMP manifests and streaming media
In the past, media files had to be downloaded in their entirety before they could be played. Modern video delivery is far more sophisticated: modern video codecs and players start after a few seconds of download – the user does not wait for the entire file download before playing it.
AMP supports streaming authentication to complement streaming delivery and playback. In addition to the manifest containing the hash of the entire video file, it also includes an array of hashes of short segments. Different codecs and network transport layers are best served by different chunking schemes, but all chunking schemes allow authentication to begin after a few seconds of video has been downloaded.
Adaptive bit rate video
Another innovation in modern video delivery is called Adaptive Bitrate Streaming (ABR) – for example, MPEG-DASH. ABR lets a publisher or platform create a set of video files optimized for different players, screen sizes/resolutions, and network conditions. For example, a publisher can create a video to be played on a 4K connected TV, and another version for a mobile device on cellular network. Smart web pages and apps can pick the stream that matches the screen and current network conditions and can even dynamically switch between streams as network conditions change.
AMP supports ABR very similarly to the JPG image facsimiles introduced earlier. Each video is treated as a facsimile of the original, and the manifest contains bindings (chunk-hashes) for all versions of the video produced by the publisher.
We use the term facsimile to describe a video (or image) that is a simple re-encoding of an original. An AMP manifest can authenticate any number of facsimiles of a video that the publisher deems equivalent.
AMP and derived works
In some cases, the media being authenticated is original. In other cases, editing and production tools are used to assemble new images, audio, or video from earlier works. Examples of such derived works include applying a blurring filter to an image or editing together several video clips to produce a news story.
AMP manifests can authenticate original works, but can also indicate that content was assembled from other works. An AMP Transformation Manifest can be used with “back pointers” to indicate the source works that were used to create a derived work, as well as the transformations applied (e.g., clipping, cropping, addition of captions, etc.)
Conclusion
The proposed AMP media provenance certification and verification system can only be successful if it becomes a widely adopted industry standard. Thus, we are forming a partnership with media organizations and additional technology providers. We plan to put this collaboration on a formal footing with the formation of an industry alliance similar to the Alliance for Open Media. Other companies can join, either as active contributors or supporters. A key goal of such an effort should be to promote the development of an open standard, to motivate fast adoption.
The implementation of an AMP system can be a significant step in increasing trust in media, while protecting viewers and publisher.
