Sunday, March 02, 2008

Metadata Matters

In recent builds of the CineForm tools and codecs, we are focusing a lot more on the robustness and a flexibility of metadata. The CineForm codec has been carrying a special class of metadata for the last 18 months; we use the term "active metadata" to describe how it differs from classic data like edge codes and copyright info. Active metadata is the mechanism that allows camera control unit functionality, such as white balance, curves, color matrix and 3D looks, to be stored non-destructively to maintain all your source image information, yet allow the playback to turn on and off or alter the active decoding/display of the metadata -- I need to do a post on this as it is very cool (or please come and see us at HD Expo (March 6) or NAB (April 14-17.)) We had traditional passive metadata for the same period of time; we just haven't started to emphasize it until now.

The CineForm structure for metadata data is little different than other streaming formats (AVI, MOV, MXF, etc), and as the metadata is embedded within each compressed video sample no matter the wrapper type. Streaming format typically attach metadata to a channel within the file wrapper, parallel to the video stream, or as a separate XML file, which is fine if you can stick with one wrapper type (AVI,MOV,MXF) and keep your files together. However, re-wrapping the stream or breaking up streams can result in the loss of this metadata. As CineForm is wrapper agnostic, we needed to store it differently so that it can't be lost. We still can use the wrapper standard metadata streams when needed, we are simply not depending on it.

As metadata is in each compressed sample, we can achieve greater flexibility as the video decoder is now aware of the metadata (not possible in parallel streams or side car files,) allowing applications that otherwise have little to no support for metadata to act on the data in new ways -- this is how our active metadata works. An example of this use is in our latest DPX to CineForm to DPX tools will completely preserve all the DPX metadata and reconstruct it when export back to DPX at a later date. We are working toward allowing passive metadata to be selected for optional burn-in display, in addition to all the search and retrieval applications of this data. Lots of cool stuff happening.

For those developers programmatically implementing CineForm compression into their tools, (Wafain, Silicon Imaging and Iridas are good examples), it is easy to directly set and control the adding of metadata. Via our SDK you get frame accuracy for metadata insertion. Yet we have added a mechanism to allow the non-programmer to add free-form metadata to any of their CineForm encodes. For Windows users, there two registry keys allowing you to add your own metadata :

Here is an example of using MetadataGlobal:

Windows Registry Editor Version 5.00

"Copyright Owner"="David Newman"
"Computer Used"="M90 Laptop"
"Random info"="put whatever you want in the file"
"Add Numbers"=dword:00000050

and save as MyGlobalData.reg. Of course you can change and add your own text and numerical data (hex format, that dword:00000050 = 80) after the HKEY line. Double click on the .reg file to install. Now every frame of every capture, or render, with have this data embedded. I've been using this in the office to know which PC any particular clip originated.

The second registry entry, MetadataLocal, only stores changes made since the last frame's encode. This is useful for attaching slowly changing data, like GPS data, or other changing data streams that occasionally change or change on every frame. Or you have a large amount of data you only want attached occasionally, rather than store it in every frame. I have yet to do much with this feature myself, but I sure it will have some powerful applications in the future.

Update: Use MetadataGlobal for changing data if you want that data to be present in every frame. Timecode is a changing field that is stored in every frame, and therefore should be using the global class. Accurate frame based changes require SDK level access.

Of course there are many new features in the works that I can't yet disclose, but please comment on your own ideas for metadata features your like to see.


Anonymous said...

Whistling...quietly to myself…

VB running adding data to Global keys, as hdlink captures.

David said...

Someone needs to came up we a cool connection that uses a twitter feed, or similar, to add relevant metadata during capture. It is certainly easy to do WiFi based shot logging now.

Moocycles said...

I am not sure if you read these, but I hope you do.

Has anyone captured camera META that is then migrated through post all the way to the web and the general public?

Mark Coleran did some really nice demos at NAB. He basically added an effect script in AE to a slider container and controlled the effect parameters with the slider. This has been done again and again for effects, trasition, whatever in Flash (WWW = Big Audience), but I don't think it has been possible to easily pull META, let alone camera META all the way through a production, and made that data searchable and NAVIGATABLE by End Users...I live in the Tetons so if you want to give it a shot in Jellystone on top of a huge mountain or down a mountain bike trail lets put Adobe to the test!

Lets give it a try.