- July 26, 2010 at 11:20 am #30227
I learned about the GL_APPLE_rgb_422 OpenGL ES extension by Apple ( http://www.opengl.org/registry/specs/APPLE/rgb_422.txt ), which is apparently supported by iOS 4 and which should support video texture decoding. Thus, I got curious and learned on this page:
that CoreVideo is supported by iOS 4.
Now, I never have done any Apple development, but according to this page:
Core Video allows you to send CVImageBuffers to OpenGL textures. (And with functions like captureStillImageAsynchronouslyFromConnection in AVCaptureStillImageOutput you apparently can get at least still images from an input source.)
I know that this is out of the scope of PowerVR, but can anyone confirm that it is already possible to get video images into OpenGL ES on iOS 4 devices?
ThanksOctober 30, 2010 at 11:36 am #34167
Just a follow-up to myself:
The issue is also discussed here (including most of the links below): https://www.imgtec.com/forum/forum_posts.asp?TID=1015
If supported (not on iOS 4.1), the best way to go is to use GL_IMG_texture_stream2. But I couldn’t find any official documentation of it. Texas Instruments is providing code that uses it under Linux: http://software-dl.ti.com/dsps/dsps_public_sw/apps_processors/OMAP35x_AM35x_Video_Texture_Streaming/1_0/index_FDS.html
On iOS, one apparently has to get a pointer to the video data in main memory (see here: http://cmgresearch.blogspot.com/2010/10/augmented-reality-on-iphone-with-ios40.html or here: http://developer.apple.com/library/ios/#qa/qa2010/qa1702.html ) and then load it into a texture with glTexImage2D and possibly do any conversion in a fragment shader (see here: http://developer.bada.com/blog/?p=373 but that particular implementation appears to be quite inefficient since it uses several calls to glTexImage2D )
Thus, currently there are no video textures on iOS 4.1 but the performance might still be OK because it boils down to a call to glTexImage2D. Things would probably be much easier if Apple would implement the full CoreVideo framework on iOS (the OpenGL part is missing on iOS) or integrate the GL_IMG_texture_stream2 extension.