Category: OMS Video

Sun Microsystems’ Open Media Stack and OMS Video codec

After a lively debate, the IETF appears to be moving forward with a royalty-free audio codec standardization activity.  Here’s to its successful launch and positive outcome.

I’ve put a brief summary at the mpegrf.com site, and there is a good summary here.

The group’s email discussion alias is here — and my view, expressed there (echoing this), is pretty straightforward:

“[codec] Royalty Free codec standards — don’t settle for less”

Here is my view, perhaps you share it, perhaps you don’t.

What the world needs now is royalty-free, standardized codecs. This is critical to the future of the Web, and the progress the Internet has brought to the world, and will bring to the world.

Video, audio, transport, the whole thing. Evaluated, vetted for patents. Under an appropriate, responsible and complete royalty free process. No less.

IETF, ITU, and ISO/MPEG should all get going on this important activity — after all why shouldn’t all of these organizations include this as core to their mission.

I have, and no doubt you have too, seen countless explanations why this should not, could not, will not, rather not, might not, or can not happen. Some well meaning and sincere, some from vested interests.

There are too many “powerful” interests against it. “Important” commercial interests are ambivalent. It is too hard “legally” or “politically” or “technically”. It is just too confusing to think through. There is no longer a critical mass that cares enough about keeping the future of the Open Internet open and royalty free. The well meaning are ignorant, or naive. Etc.

Don’t settle. Take the issue of royalty free, standardized codecs all the way to the top of these organizations. Do what it takes. If it requires new organizations, start them. It it requires revised processes, revise them. This is the spirit that built the Web and the Internet, this is the spirit that is its lifeblood, and this is the spirit that needs to be at the heart of its future.

Don’t settle. Don’t let those who have tried hard already, or have only half-heartedly tried, justify the status quo or their half-heartedness. Encourage them to focus on how to take the next steps.

Don’t let convenient “interpretations” of standards processes be an excuse for never starting, never finishing, or never setting up processes that will work. Need more legal background? Find it. More technical information? Get it.

Don’t settle. The world has plenty of patent-encumbered media standards, plenty of proprietary solutions, and plenty of standards in other domains that have figured out how to deliver royalty free.

But the world does not have enough royalty-free codec standards, so this is the task that needs to be addressed.

Rob

Here is my view, perhaps you share it, perhaps you don't.

What the world needs now is royalty-free, standardized codecs. This is critical to the future of the Web, and the progress the Internet has brought to the world, and will bring to the world. Video, audio, transport, the whole thing. Evaluated, vetted for patents. Under an appropriate, responsible and complete royalty free process. No less. IETF, ITU, and ISO/MPEG should all get going on this important activity -- after all why shouldn't all of these organizations include this as core to their mission. I have, and no doubt you have too, seen countless explanations why this should not, could not, will not, rather not, might not, or can not happen. Some well meaning and sincere, some from vested interests. There are too many "powerful" interests against it. "Important" commercial interests are ambivalent. It is too hard "legally" or "politically" or "technically". It is just too confusing to think through. There is no longer a critical mass that cares enough about keeping the future of the Open Internet open and royalty free. The well meaning are ignorant, or naive. Etc. Don't settle. Take the issue of royalty free, standardized codecs all the way to the top of these organizations. Do what it takes. If it requires new organizations, start them. It it requires revised processes, revise them. This is the spirit that built the Web and the Internet, this is the spirit that is its lifeblood, and this is the spirit that needs to be at the heart of its future. Don't settle. Don't let those who have tried hard already, or have only half-heartedly tried, justify the status quo or their half-heartedness. Encourage them to focus on how to take the next steps. Don't let convenient "interpretations" of standards processes be an excuse for never starting, never finishing, or never setting up processes that will work. Need more legal background? Find it. More technical information? Get it. Don't settle. The world has plenty of patent-encumbered media standards, plenty of proprietary solutions, and plenty of standards in other domains that have figured out how to deliver royalty free. But the world does not have enough royalty-free codec standards, so this is the task that needs to be addressed. 

Rob

In a virtual rematch of 18 months ago, the HTML 5 community has again stalemated on how to include video capability.  On June 29, HTML 5 Editor Ian Hickson posted on the whatwg mailing list:

After an inordinate amount of discussions, both in public and privately, on the situation regarding codecs for <video> and <audio> in HTML5, I have reluctantly come to the conclusion that there is no suitable codec that all vendors are willing to implement and ship. I have therefore removed the two subsections in the HTML5 spec in which codecs would have been required, and have instead left the matter undefined …

An unresolveable squabble between professional boxing underdogs — unacceptably overpriced h.264 codec vs. Ogg/Theora, an open source derivative of On2’s VP3 (the predecessor to the VP6 codec in Flash)?

The Web and the world must demand a better answer.  The fundamental challenge of open video on the Web is political, not technical or legal.

Unfortunately, the industry is, politically speaking, running east looking for sunset.  Hickson has painted a grim picture of risky, slow-boat fragmentation:

Going forward, I see several (not mutually exclusive) possibilities, all of which will take several years:

1. Ogg Theora encoders continue to improve. Off-the-shelf hardware Ogg Theora decoder chips become available. Google ships support for the codec for long enough without getting sued that Apple’s concern regarding submarine patents is reduced. => Theora becomes the de facto codec for the Web.

2. The remaining H.264 baseline patents owned by companies who are not willing to license them royalty-free expire, leading to H.264 support being available without license fees. => H.264 becomes the de facto codec for the Web.

France and the color TV war

Years?  Try decades.  Lawsuits are still being filed about digital TV patents — over a specification adopted by the FCC in 1996, 13 years ago!

The world is still recovering from the patent-fueled cold-war, carve-the-world gambits of NTSC, PAL, and SECAM.  Though “[n]inety-five percent of the three systems is based on the original American patent rights”, the March 1965 France/Soviet Union accord that effectively blocked an ITU standard was hailed by the French minister of information as a “glorious day for the human race”.  Pass the freedom fries, please.

And don’t count on patent expiration — this empty hope hasn’t worked for over a century.  Like the head of telegraph monopoly Western Union argued in 1883 against government  takeover:

“Besides, the Government would labor under great embarrassments; for though the Morse patents have expired, all the best forms of telegraph instruments, batteries, and other appurtenances are protected by more recent patents, which the Government  has no more right, than any citizen or corporation, to use without the consent of the owners.” (emphasis added)

And this we’ll-just-keep-getting-new-patents gotcha was after the U.S. Supreme Court (in 1854) limited the Morse patent claims and the U.S. Congress (in 1866) granted right-of-ways on public land in exchange for regulatory control!

Indeed, patent pools never die, they just update their portfolios.

But there is a better answer than protracted proxy war of uncertain outcome.  Dust off the standards shoes and do a new royalty free standard right.

Several standards groups already have some, perhaps all, of the needed competencies — right IPR policy to support a royalty free profile or standard, technical chops, organizational charter, critical mass of industry and political support and credibility, access to sufficient resources, and willingness to go the distance.

One or more, or a joint team like the Joint Video Team formed in 2001 from ITU-T Study Group 16 (VCEG) and ISO/IEC JTC 1 SC 29 / WG 11 (MPEG) to develop h.264, could, and should, overcome shortcomings and hesitations and rise to the task.

Let’s consider some candidates:

World Wide Web Consortium.  Right IPR policy, at the epicenter of industry need, digital media experience, and thus perhaps the natural first choice of the HTML community.

At the recent Open Video Conference, representatives from Ogg, Dirac and OMS Video all publicly expressed their enthusiastic support to join and contribute to a full-fledged standardization of a royalty free codec under the umbrella of the W3C.  There might be some questions about whether Theora, or On2 themselves, would have the right to contribute from the VP3 technology based on the original limited open source license — but this is exactly the kind of issue a credible standards initiative would be able to look into and resolve.

MPEG.  There may be cracks in the death star to the royalty free community, MPEG, which undoubtedly has the technical chops but is historically perceived as reluctant to bite the well-funded hand of international patent interests.

MPEG convenor Leonardo Chiariglione, who perhaps more than anyone merits the “invented MPEG” mantle, expressed a fascinating change of heart last fall in contemplating a nomination for the “European Inventors of the Year Award”:

“I fear that the virtuous circle … whereby the reward from innovation is used to create more innovation may be coming to an end. I believe MPEG should enlarge its portfolio of standards by offering some that are expected to be royalty free and typically less performing and with less functionality next to those that are state of the art, more performing and with more functionality.” (emphasis added)

ITU. With over 140 years of history under its belt, the ITU predates the telephone itself.  And though the h.26 series seemed at one point to be overshadowed by the ISO MPEG committee, the JVT and  h.264 have put ITU back in the codec news, albeit with a more MPEG-oriented patent pool.

China AVS.  A relative newcomer, AVS may be the only operating codec developing standards group today with a proactive IPR process (if one considers SMPTE as more a ratifying organization of VC-1 and VC-2).

ATSC, DVB.  On paper the leading digital TV standards groups, wide membership bases, responsive to government policy directives, flexible IPR policy (ATSC is flexibly not an ANSI member, which opens its IPR policy options; DVB MOU is pool-friendly, but procedurally its IPR policy is not unworkable).

Khronos.  Strong IPR policy, already standardized codec primitives in the OpenMAX layer.

SMPTE.  Standardizing the production-oriented Dirac.

No doubt there may be others, or even an insurgent standards group forming in a garage as I write.

So how to proceed?

W3C seems a natural first choice.  After that, time to heal cold war scars and address root causes.  Perhaps a new partnership between ITU and W3C, reminiscent of the JVT,  but including interested global standardization participants willing to stick to a strict royalty-free policy for the activity.

References

Rhonda J. Crane, The Politics of International Standards: France and the Color TV War 18, 72-73 (Ablex Publishing Corporation, Norwood NJ, 1979)

Dr. Norvin Green, The Government and the Telegraph, The North American Review 422, 428 (1883)

Daniel J. Boorstin, The Americans:  The Democratic Experience 57 (1973) (“There was hardly a major invention in the century after the Civil War which did not become a legal battlefield.”)

After an inordinate amount of discussions, both in public and privately,on the situation regarding codecs for <video> and <audio> in HTML5, I havereluctantly come to the conclusion that there is no suitable codec thatall vendors are willing to implement and ship.

Sun’s OMS Video codec work was mentioned in Matt Raible’s notes from the “The State of the Web 2009” session at this week’s Web Directions North conference:

“Very specifically, there’s no royalty-free codec for video. This is nothing that standards bodies can solve. The most promising is that Sun Microsystems is developing an open codec and spending money to make sure they’re not infringing on patents.”

omsvideo

Also, “open video codec” topped the panelists’ biggest-thing lists:

“After each panelist talked, John asked them questions about what’s the biggest thing they’d like to see implemented by everyone (open video codec, geo location api were the winners).”

(emphasis added)

The whole write-up is worth reading here.

“Patent and legal issues” topped, at least numerically, the community goals developed at the recently-held Foundations of Open Media 2009 workshop, a write-up of which was just posted here.

Also noted in “Patents and the bright future of open media codecs”, the FOMS group has set aside 15% of its budget to support patent analysis.

For those who haven’t seen it, check the interesting article “Patent Status of MPEG-1, H.261 and MPEG-2” and associated wiki here.  Comments on calculating expiration dates can be found in the responses to article here.

The “Open Media Stack – Video Specification V0.9” is now available for community review at the Open Media Commons website.

OMS, announced in April 2008, is a project of Sun Microsystems’ Open Media Commons initiative to define a complete, royalty-free media specification, including codecs and associated elements.

The OMS Video draft is a key milestone and a big deal, and not just because I am proud to have contributed to this effort at Sun.   This is the first public release of a video codec specification with a vetted royalty-free methodology that is a determined, bottom-up invent-around of the royalty-bearing interlocking and cross-coordinating set of MPEG codec licenses – MPEG-2, MPEG-4, and h.264/AVC (an orchestrated complex of licenses managed through the same license administrator, MPEG-LA, which is owned and controlled, no surprise, by MPEG patent holders themselves).

But do royalties on things like video codecs really matter?  Many people don’t even realize that royalties like these exist, or assume they are a just a nuisance cost-of-business borne somewhere in the value chain.

But consider — MPEG and their related royalties are:

– Big. Think on the order of over $2 billion per year, on some one billion or so MPEG integrated circuits a year and growing.  That’s royalties, not sales, so better to think of them on the profit side of the business equation — and that’s on the order of ten percent of the entire profit of the world’s 37 largest consumer electronics companies, calculated from a recent Deloitte study of consumer products industries.  And given that consumer electronics manufacturers are in a notoriously thin-margin business (according to the Deloitte study, average net profit margin of 3.3%, the second lowest profit in all consumer products industries, ahead of only tires) — the $2.50 per MPEG-2 device royalty alone likely exceeds the manufacturer profit on vast numbers of consumer electronics products.

– Applicable across the entire value chain of devices, content, and encoders.  Consider also the $100 million or more a year in royalties of 3 cents on “packaged content” on 90% of all DVD discs produced, collected through disc duplicators.  Or the reported 100M Euros in royalties in 2005 on MP3.  Or the temporary exemption on “Internet broadcast” royalties — expiring December 31, 2008 on MPEG-4 (perhaps to the chagrin of MPEG-4 licensee DiVX, who noted in a recent SEC filing “Our license agreement with MPEG LA, under its MPEG-4 Part 2 Visual Patent Portfolio will expire on December 31, 2008. MPEG LA has the right to renew the license agreement for successive terms of five years, upon notice to us.”), and expiring December 31, 2010 on AVC/h.264.

– Applicable to you (and virtually everyone else on the network connected planet).  Just check your iPhone EULA (“The iPhone Software and iPhone Software Updates contain AVC encoding and/or decoding functionality, commercial use of H.264/AVC requires additional licensing.”) or Flash licensing restrictions:

“The end-user license for Flash Player allows users to play back H.264 content for their own noncommercial use. Commercial applications of Flash Player to decode H.264 video may require a separate license…. Usage categories that may require a license and involve royalty fees include advanced video coding products, title-by-title video, and subscription video among others. Most categories apply to commercial use and implementation, but some are more broad”.

Sure, some will assert that this doesn’t really apply to “you”, just figure out a clever “not-me” work-around — but to quote Russell Long and countless others:  “Don’t tax you, don’t tax me. Tax that fellow behind the tree.”

Gosh, one can see how one study of consumer electronics patents/standards commented: “Standardization activities are political negotiations and not a forum for assessing which technologies excel over others”.

So here’s to hoping that OMS Video will add to the growing but still under-the-radar open media movement, including Xiph, Dirac, and more to come.  Please take a look at the OMS Video specification and provide comments if you are so inclined.