1. hokuson's Avatar
    Thanks for the reply.

    I added the @protocol code in the header file
    and I also added [previewView retain]; in camAppDelegate.m.

    Then I can preview the camera!

    Last edited by hokuson; 2009-07-21 at 02:35 AM.
    2009-07-21 01:45 AM
  2. svdae's Avatar
    Hello All,

    Did somebody found any successful way to Surface-callback installation? How to get the raw data from the video flow?

    2009-07-23 02:19 PM
  3. hokuson's Avatar

    I set "CameraMode" to 1, and "canCaptureVideo" returned TRUE.
    After that, I'm calling "startVideoCaptureAtPath" method,
    then "isCapturingVideo" returns TRUE.

    But, delegate doesn't get called.
    I'm not sure what's wrong.

    	if([cameraController performSelector:@selector(canCaptureVideo)]){
    		NSLog(@"can capture video!");
    	NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    	NSString *documentsDirectory = [paths objectAtIndex:0];
    	[cameraController performSelector:@selector(startVideoCaptureAtPath:) withObject:documentsDirectory];
    	if([cameraController performSelector:@selector(isCapturingVideo)]){
    - (void)applicationDidFinishLaunching:(UIApplication *)application {    
        application.statusBarHidden = YES;
    	self.cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
    	[cameraController setDelegate:self];
    	UIView *previewView = [cameraController performSelector:@selector(previewView)];
    	[window addSubview:previewView];
    	[cameraController performSelector:@selector(startPreview)];
     	[cameraController performSelector:@selector(setCameraMode:) withObject:1];
        // Override point for customization after app launch    
        [window addSubview:viewController.view];
        [window makeKeyAndVisible];
    	[NSTimer scheduledTimerWithTimeInterval:5
    	   addedVideoAtPath:(NSString *)path
    -(void)cameraControllerVideoCaptureDidStart:(id)sender {
    2009-07-23 05:34 PM
  4. troffmo5's Avatar
    > Did somebody found any successful way to Surface-callback
    > installation? How to get the raw data from the video flow?

    Look in "camAppDelegate.h" (one of the Hokuson previous post) at the functions:

    - (id)_createPreviewImage;
    - (void *)_createPreviewIOSurface;

    perhaps is what you are looking for.

    _createPreviewImage should return an UIImage.
    _createPreviewIOSurface should return a CoreSurfaceBuffer.

    Good luck
    2009-07-24 10:46 AM
  5. zniff's Avatar
    Hi all!

    I've been following this thread for a while. You guys seems to have figured it out, but I can't seem to get this approach to work.. I'm trying to capture a photo on an iPhone 2G on firmware 3.0.

    My readyStateChanged delegate gets called if I do a [cameraController _setIsReady]; and my modeDidChange gets called if I do a [cameraController _setCameraMode:0 force:TRUE]; - so my delegates doesn't seem to be the problem.

    I just can't get isReady to return YES and my tookPicture delegate never gets called.

    I don't need a preview on the screen or anything else, I just need it to capture a photo and call the tookPicture delegate.

    (NSLog indicates that my call to capturePhoto takes about 2 seconds, so I know it does something, I just wish it would let me know what )

    @interface Scratch: UIApplication
    	id cameraController;
    - (void)applicationDidFinishLaunching:(NSNotification *)aNotification;
    @end /* @interface Scratch */
    @implementation Scratch
    - (void)cameraControllerReadyStateChanged:(NSNotification *)aNotification {
    - (void)cameraController:(id)sender tookPicture:(UIImage*)picture withPreview:(UIImage*)preview jpegData:(NSData*)rawData imageProperties:(struct __CFDictionary *)imageProperties {
    - (void)cameraController:(id)sender addedVideoAtPath:(NSString *)path withPreviewSurface:(id)surface metadata:(id)meta wasInterrupted:(BOOL)interrupt {
    - (void)cameraController:(id)sender modeDidChange:(int)mode {
    - (void)cameraControllerVideoCaptureDidStart:(id)sender {
    - (void)cameraControllerVideoCaptureDidStop:(id)sender {
    - (void)cameraControllerFocusFinished:(id)sender {
    - (void)applicationDidFinishLaunching:(NSNotification*)aNotification {
    	freopen([@"/tmp/scratchlog.txt" fileSystemRepresentation], "w", stderr);
    	cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
    	[cameraController setDelegate:self];
    // 	[cameraController setCameraMode: 0];
    //	[cameraController _setCameraMode:0 force:TRUE]; // Will trigger modeDidChange delegate
    	[cameraController setFocusDisabled:YES];
    	[cameraController setDontShowFocus:YES];
    	[cameraController setCaptureAtFullResolution:YES];
    //	[cameraController _setIsReady]; // Will trigger readyStateChanged delegate
    	NSLog(@"CameraMode: %d", [cameraController cameraMode]);
    	NSLog(@"isReady: %d", [cameraController isReady]);
    	NSLog(@"Starting preview");
    	[cameraController startPreview];
    	NSLog(@"isReady: %d", [cameraController isReady]);
    	NSLog(@"isReady: %d", [cameraController isReady]);
    	NSLog(@"Capturing photo");
    	[cameraController capturePhoto:NO];
    	NSLog(@"Stopping preview");
    	[cameraController stopPreview];
    int main (int argc, char **argv) {
    	int ret;
    	NSAutoreleasePool *pool;
    	pool = [ [ NSAutoreleasePool alloc ] init ];
    	ret = UIApplicationMain (argc, argv, @"Scratch", @"Scratch");
    	[pool release];
    	return ret;
    @end /* @implementation Scratch */
    What is it that I am missing? Any help would be appreciated!
    2009-07-24 02:51 PM
  6. hokuson's Avatar
    Hi troffmo5

    The UIImage returned from _createPreviewImage is a screen capture image, not a raw data.
    So, if there are another layers on the camera layer, they are also captured.

    Do you have any idea to get raw image?

    Last edited by hokuson; 2009-07-24 at 05:49 PM.
    2009-07-24 03:26 PM
  7. rilocr's Avatar
    Hi troffmo5

    Yes, _createPreviewImage is a screen capture, so if you have a navigation bar or a tool bar, they will appear on the image, I have tried also with _createPreviewIOSurface and it is the same, it's a screen capture.

    Also, both functions return, what it seems, a preprocessed image/buffer, so the size is way too big than the size of the raw data buffer...

    But, yes I think that's a solution, but at least, for my app, I really need the raw data :-(

    2009-07-24 04:38 PM
  8. ankitgupta00's Avatar

    This is a great thread and have been following it for a while. I wanted to modify the raw video frames that the user records before saving it to disk. I did my own investigation and I feel that this isn't possible WHILE the user is actually taking the video. It seems this is the case because the PLCameraController (using AVCapture) starts/stops/pauses recording and saves the video to a particular URL on its own. All it gives are delegate methods of notifications of these actions. Thus, it is not possible to access the video frame by frame and modify them while recording.

    Can someone confirm if this is true?

    If it is, then all we can do is postprocess the video (stored in tmp/capture/capturedvideo.mov). Thus, can someone point me to a good library to read an mov file frame by frame and re-write it? I tried opencv but it doesn't handle audio..

    2009-07-28 01:45 AM
  9. rilocr's Avatar

    I was able to modify every frame but with the old SurfaceBuffer hack, by reading the raw buffer. With the 3.0 and those delegates, I don't think you can modified them.

    But, you can do it if you use the function _createPreviewIOSurface, as stated in a previous post, but you have to verify if you can use it for your app, and if you can use a screen capture.

    2009-07-28 05:03 PM
  10. basm's Avatar

    first of all this Thread is great! You gave me hope and support when i was about to surrender :-)
    I'm just having a small problem reproducing the code from hokuson 's post.
    I used his approach in exactly the same way but I don't get a preview screen. Just for a split second the camera-preview appears once after launching the app and than the screen goes black. Except for the white bar at the bottom.

    There is just one thing that I am not quite sure about. I read in another post : Mobile Augmented Reality: iPhone:Using PLCameraController how to use the PLCameraController header and added the private framework in /Developer/Platforms/iPhoneOS.platform/ Developer/SDKs/iPhoneOS2.0.sdk/System/Library/

    I also tried the 3.0.sdk version but i get the same result.

    Does Anybody have a clue or can tell me if this is the right way to utilize the PLCameraController header?

    Thanks a lot

    Last edited by basm; 2009-07-29 at 04:59 PM.
    2009-07-29 04:54 PM
  11. rilocr's Avatar
    Hi bas,

    Can you post your camera code? It would be easy to find something.

    There are other solutions in this thread, maybe one of them could work for you.
    2009-07-30 05:39 PM
  12. basm's Avatar
    I figured out my mistake

    I'm new to Objective C and didn't know what the retain methode is used for, or how to use it. But after reading How To's about the NSObject Class i figured out that the retain method return a pointer to the object and that i had to use it like:

    cameraController = [cameraController retain];

    Here is the code from hokuson 's post with the changes described in this thread:


    #import <UIKit/UIKit.h>
    @class AVCapture, PLPreviewView;
    @protocol PLCameraControllerDelegate;
    @interface PLCameraController : NSObject
        AVCapture *_avCapture;
        PLPreviewView *_previewView;
        BOOL _isPreviewing;
        BOOL _isLocked;
        int _cameraMode;
        int _captureOrientation;
        int _focusCount;
        int _autofocusCount;
        unsigned int _previousSimpleRemotePriority;
        id <PLCameraControllerDelegate> _delegate;
        double _startTime;
        struct {
            unsigned int supportsVideo:1;
            unsigned int supportsAccurateStillCapture:1;
            unsigned int supportsFocus:1;
            unsigned int capturingVideo:1;
            unsigned int deferStopPreview:1;
            unsigned int deferStartVideoCapture:1;
            unsigned int inCall:1;
            unsigned int continuousAutofocusDuringCapture:1;
            unsigned int focusDisabled:1;
            unsigned int focusedAtPoint:1;
            unsigned int wasInterrupted:1;
            unsigned int resumePreviewing:1;
            unsigned int isReady:1;
            unsigned int didSetPreviewLayer:1;
            unsigned int didNotifyCaptureEnded:1;
            unsigned int dontShowFocus:1;
            unsigned int isChangingMode:1;
            unsigned int lowResolutionCapture:1;
            unsigned int delegateModeDidChange:1;
            unsigned int delegateTookPicture:1;
            unsigned int delegateReadyStateChanged:1;
            unsigned int delegateVideoCaptureDidStart:1;
            unsigned int delegateVideoCaptureDidStop:1;
            unsigned int delegateVideoAdded:1;
            unsigned int delegateFocusFinished:1;
        } _cameraFlags;
    + (id)sharedInstance;
    - (id)init;
    - (void)dealloc;
    - (void)_inCallStatusChanged:(BOOL)fp8;
    - (BOOL)inCall;
    - (void)_setIsReady;
    - (BOOL)isReady;
    - (BOOL)canCaptureVideo;
    - (int)cameraMode;
    - (void)_setCameraMode:(int)fp8 force:(BOOL)fp12;
    - (void)setCameraMode:(int)fp8;
    - (void)_applicationSuspended;
    - (void)_applicationResumed;
    - (void)_tookPicture:(struct CGImage *)fp8 jpegData:(struct __CFData *)fp12 imageProperties:(struct __CFDictionary *)fp16;
    - (void)_tookPhoto:(id)fp8;
    - (void)_previewStarted:(id)fp8;
    - (void)_previewStopped:(id)fp8;
    - (void)_setVideoPreviewLayer;
    - (BOOL)_setupCamera;
    - (void)viewDidAppear;
    - (void)_tearDownCamera;
    - (void)setDelegate:(id)fp8;
    - (id)delegate;
    - (id)previewView;
    - (void)startPreview;
    - (void)_destroyAVCapture;
    - (void)stopPreview;
    - (void)resumePreview;
    - (BOOL)supportsAccurateStillCapture;
    - (void)capturePhoto:(BOOL)fp8;
    - (BOOL)isCapturingVideo;
    - (void)_captureStarted:(id)fp8;
    - (id)_createPreviewImage;
    - (void *)_createPreviewIOSurface;
    - (void)_captureCompleted:(id)fp8;
    - (void)_didStopCapture;
    - (void)_wasInterrupted:(id)fp8;
    - (void)_interruptionEnded:(id)fp8;
    - (BOOL)canStartVideoCapture;
    - (BOOL)startVideoCaptureAtPath:(id)fp8;
    - (void)_stopVideoCaptureAndPausePreview:(id)fp8;
    - (void)stopVideoCaptureAndPausePreview:(BOOL)fp8;
    - (id)videoCapturePath;
    - (BOOL)focusAtPoint:(struct CGPoint)fp8;
    - (void)restartAutoFocus;
    - (void)autofocus;
    - (void)lockFocus;
    - (BOOL)isFocusing;
    - (void)setDontShowFocus:(BOOL)fp8;
    - (void)setFocusDisabled:(BOOL)fp8;
    - (void)setCaptureAtFullResolution:(BOOL)fp8;
    - (void)_commonFocusFinished;
    - (void)_focusOperationFinished;
    - (void)_autofocusOperationFinished;
    - (void)_focusCompleted:(id)fp8;
    - (void)_focusWasCancelled:(id)fp8;
    - (void)_focusStarted:(id)fp8;
    - (void)_focusHasChanged:(id)fp8;
    - (int)videoCaptureOrientation;
    - (void)irisWillClose;
    - (void)_serverDied:(id)fp8;
    @interface camAppDelegate : NSObject <UIApplicationDelegate> {
        UIWindow *window;
    	PLCameraController *cameraController;
    @property (nonatomic, retain) IBOutlet UIWindow *window;

    #import "camAppDelegate.h"
    @implementation camAppDelegate
    @synthesize window;
    - (void)cameraControllerReadyStateChanged:(NSNotification *)aNotification
        NSLog(@"cameraControllerReadyStateChanged: %@", aNotification);	
    		imageProperties:(struct __CFDictionary *)imageProperties
    	NSLog(@"Saving and re-initing...");
    	UIImageWriteToSavedPhotosAlbum(picture, self, @selector(imageSavedToPhotosAlbum: didFinishSavingWithError: contextInfo:), nil);  
    	[cameraController performSelector:@selector(stopPreview)];
    	[cameraController performSelector:@selector(startPreview)];
    	   addedVideoAtPath:(NSString *)path
    		  modeDidChange:(int)mode {
    -(void)cameraControllerVideoCaptureDidStart:(id)sender {
    -(void)cameraControllerVideoCaptureDidStop:(id)sender {
    -(void)cameraControllerFocusFinished:(id)sender {
    - (void)applicationDidFinishLaunching:(UIApplication *)application {    
    	cameraController = [(id)objc_getClass("PLCameraController") performSelector:@selector(sharedInstance)];
    	cameraController = [cameraController retain];
    	[cameraController setDelegate:self];
    	[cameraController setFocusDisabled:YES];
    	[cameraController setCaptureAtFullResolution:NO];
    	[cameraController setDontShowFocus:YES];
    	UIView *previewView = [cameraController performSelector:@selector(previewView)];
    	[window addSubview:previewView];
    	[cameraController performSelector:@selector(startPreview)];
    	[cameraController performSelector:@selector(setCameraMode:) withObject:0];
    	// Override point for customization after application launch
        [window makeKeyAndVisible];
    - (void)dealloc {
        [window release];
        [super dealloc];
    2009-07-30 11:39 PM
  13. DrMembrane's Avatar
    I've used your code but I only get a blank screen - am I supposed to do something in the MainWindow.xib?
    2009-07-31 02:18 PM
  14. basm's Avatar
    if with blank screen you mean a white screen than you did not include the private framework as described in Mobile Augmented Reality: iPhone:Using PLCameraController

    i did not change the MainWindow.xib. Just used the "Window-based Application" Template within Xcode.
    2009-07-31 02:28 PM
  15. natios's Avatar

    Great thread.
    I've been trying to work the camera out on a new 3GS, but met several problems.
    Getting the camera controller seems to work, as in:
    self.cameraController = [objc_getClass("PLCameraController") sharedInstance];
    but when I try getting the controller's _camera variable, as in:
    char *p = NULL;
    object_getInstanceVariable(cameraController,"_came ra",(void**)&p);
    if (!p) return NULL;
    I keep getting nil in p.

    Anybody has any idea what I might be doing wrong, or whether this approach doesn't always work?

    Thanks guys,
    2009-08-18 06:57 AM
  16. rilocr's Avatar
    Hi Nate,

    The _camera variable was removed from the new SDK 3.0, that's also the reason why the SurfaceBuffer hack stopped working.

    You should try any other of the solutions in this thread.

    2009-08-18 05:59 PM
  17. natios's Avatar
    Hi rilocr, thanks for your reply.
    I found out I was indeed trying to access the non-existing member _camera, looking at the old header.
    However, If I'm not wrong, the posts on this thread were only successful at previewing the image, which I managed. What I need is the actual raw video data (rather than a single photo) for further processing.
    Perhaps I'm missing on something, but is there any solution to this for the 3.0 SDK so far?
    Last edited by natios; 2009-08-19 at 12:42 AM.
    2009-08-19 12:09 AM
  18. rilocr's Avatar
    Well, I have not found a solution on 3.0 similar to the SurfaceBuffer hack on 2.2.1.

    But, as stated in a previous post, I think you can used one of this functions:

    - (id)_createPreviewImage;
    - (void *)_createPreviewIOSurface;

    Both return a screen capture, but I think is the closest to the raw buffer data.

    For my app, I used the createPreviewIOSurface which returns a CoreSurfaceBufferRef similar to the 2.2.1 hack.

    That was my workaround, until someone finds how to get the real raw buffer data.
    2009-08-19 01:12 AM
  19. natios's Avatar
    What event do you use to access the surface buffer then?
    2009-08-19 01:26 AM
  20. rilocr's Avatar
    I just call the _createPreviewIOSurface function inside a thread
    2009-08-19 05:32 PM
64 1234