The MediaConch team is excited to announce that the Internet Engineering Steering Group of the IETF has approved the CELLAR (Codec Encoding for LossLess Archiving and Realtime transmission) working group charter. The CELLAR working group will focus on the standardization of Matroska, FFV1, and FLAC file formats. MediaConch team member Tessa Fallon will server as a working group co-chair along with Tim Terriberry.
With the formation of this working group, development work and change control on the specifications shall move from their original developer communities into the IETF. Those interesting in following or participating in the standardization process can join the CELLAR mailing list here.
Resources
CELLAR WG Site: The approved charter, timeline, and resource information for the IETF Charter Working Group.
FFV1 Spec GitHub Repository: Ongoing development of the FFV1 specification.
EBML Spec GitHub Repository: Ongoing development of the Extensible Binary Meta Language format, which is the basis of Matroska.
Matroska Specification: Current draft of the Matroska Media container specification.
FLAC Specification: Current draft of the FLAC specification.
MediaConch Specification Appendix: March 2015 report from the MediaConch team on standardization of FFV1 and Matroska.
Profiles of Formats
Matroska
Matroska is a open-licensed audiovisual container format with extensive and flexible features and an active user community. Matroska is based on EBML, Extensible Binary Meta Language. An EBML file is comprised of one of many defined "Elements". Each element is comprised of an identifier, a value that notes the size of the element's data payload, and the data payload itself. The data payload can either be stored data or more nested elements. The top level elements of Matroska focus on track definition, chapters, attachment management, metadata tags, and encapsulation of audiovisual data. The Matroska element is analogous to QuickTime's atom and AVI's chunk.
Matroska integrates a flexible and semantically comprehensive hierarchical metadata structure as well as digital preservation features such as the ability to provide Cyclic Redundancy Check (CRC) checksums internally per selected elements. Because of its ability to use internal, regional CRC protection it is possible to update a Matroska file during OAIS events without any compromise to the fixity of its audiovisual payload.
FFV1
FFV1 is an efficient lossless video stream that is designed in a manner responsive to the requirements of digital preservation. Version 3 of this lossless codec is highly self-descriptive and stores its own information regarding field dominance, aspect ratio, and colorspace so that it is not reliant on a container format to store this information. Other streams that rely heavily on its container for technical description often face interoperability challenges.
FFV1 version 3 mandates storage of CRCs in frame headers to allow verification of the encoded data and stores error status messages. FFV1 version 3 is also a very flexible codec allowing adjustments to the encoding process based on different priorities such as size efficiency, data resilience, or encoding speed.
The specification documentation for FFV1 is partially complete and has recently been funded by vendors utilizing FFV1 as a codec for audiovisual preservation and large-scale digitisation efforts.
FLAC
FLAC, Free Lossless Audio Codec, is an audio coding format for lossless compression of digital audio. As FFV1 does, FLAC employs a mechanism to store fixity data within the stream to allow for error concealment and detection of fixity errors. The header of a FLAC file contains an MD5 checksum or signature that represents the original audio data that is encoded. Since FLAC is a lossless format this MD5 also represents the audio data that should be decoded from the FLAC file. For more granular fixity FLAC audio samples are grouped into audio frames which themselves are checksummed with a CRC value allowing bit rot or other corruption to be pinpointed. Additionally the FLAC format has comprehensive metadata and image handling as well as mechanisms to store foreign data from source formats such as arbitrary chunks of an input WAV file.
MediaConch is part of the PREFORMA (PREservation FORMAts for culture information/e-archives) project, co-funded by the European Commission under the FP7-ICT programme. Learn more about MediaConch here.
Kind Regards,
The MediaConch Team