Canon Liveview with libgphoto2

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Canon Liveview with libgphoto2

Keith
I am trying to capture Liveview images from my Canon camera. I want to capture
them at a rate of 30 fps which I can do using Canon's SDK. However, when using
libgphoto2 I appear to be getting images at only about 8-10 fps which is too
slow. I am running Ubuntu 10.04 and have compiled with optimization. I am using
gp_camera_capture_preview to capture the images in Liveview mode.


------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|

Re: Canon Liveview with libgphoto2

Marcus Meissner-4
On Fri, Dec 28, 2012 at 01:26:45AM +0000, Keith wrote:
> I am trying to capture Liveview images from my Canon camera. I want to capture
> them at a rate of 30 fps which I can do using Canon's SDK. However, when using
> libgphoto2 I appear to be getting images at only about 8-10 fps which is too
> slow. I am running Ubuntu 10.04 and have compiled with optimization. I am using
> gp_camera_capture_preview to capture the images in Liveview mode.

Which camera is that, a Powershot or a EOS?

There might be room for improvement.

Ciao, Marcus

------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|

Re: Canon Liveview with libgphoto2

Keith
Hi Marcus,

Thanks for getting back with me. I am using a Canon EOS 5D mark II. In liveview mode the 5D has the capability to push out 30 fps. Each frame comes within a 32-34 msec rate. We can see this working under Windows with the Canon SDK and are trying to duplicate it under Linux using Libgphoto2. We are using Ubuntu 10.04 and libgphoto2-2.5.0.

With no modifications to libgphoto2 default build, we get about 8-10 fps in general. frame rates are inconsistent though. Sometime we might have a frame rate of 40 msec and the next frame is received in 84 msec or so. We have found that the logging and debug code was having an effect on the frame rate and have figured out how to turn off both logging and debug code. This improved performance significantly to about 15-20 fps, but not enough for us.

We started digging into the code more and found several more issues that helped performance and resolved some other issues. These included:

In camlibs/ptp2/library.c

Line 1723 in camera_capture_preview changed

<             int             tries = 100;
---
>             int             tries = 1000;

Line 1771-1775 in camera_capture_preview commented out line

<                         xret = ptp_check_eos_events (params);
<                         if (xret != PTP_RC_OK)
<                             return translate_ptp_result (xret);
<                         gp_context_idle (context);
<                         usleep (50*1000);
---
> //                        xret = ptp_check_eos_events (params);
> //                        if (xret != PTP_RC_OK)
> //                            return translate_ptp_result (xret);
> //                        gp_context_idle (context);
> //                        usleep (50*1000);

We did the above two changes because if the event was not ready, it would sleep for 50 msec before trying again. This would already cause us to miss frames. So we increased tries to 1000 and just looped until the next frame was there rather than checking for events. 100 tries was not enough, so we increased it.

Line 1761 in camera_capture_preview changed

<                     gp_file_append ( file, (char*)data+8, len );
---
>                     gp_file_append ( file, (char*)data+8, len-8 );

Delving into the code, we discovered that go_file_append provided an extra 8 bytes of data past the EOI marker, so we just subtracted 8 bytes from the length. Further looking into these extra bytes, there appears to be about 4k more bytes after the EOI and that this data may have other information in it. Could you somehow provide a pointer to it so that it could be accessed from the application level?

--------------------------------------------------------


In camlibs/ptp2/ptp.h we changed line 79

< #define PTP_USB_BULK_HS_MAX_PACKET_LEN_READ   512
---
> #define PTP_USB_BULK_HS_MAX_PACKET_LEN_READ   65535

We just tried increasing this value to see if it would make a difference and it did. However, it is probably not a good long term fix.

-------------------------------------------------

In camlibs/ptp2/usb.c line 290

<         if (usbdata.length == 0xffffffffU) {
---
>         if (usbdata.length >= 0xffffU) {

In this code - " if (usbdata.length == 0xffffffffU)", is checking if usbdata.length is exactly 4,294,967,295 bytes long. Probably not what you want. We changed it to 65,535 which is the limit of a USB bulk packet. So if the data was larger than a USB bulk packet the code would be properly executed.

----------------------------------------------


All these changes got us much closer to a 30 fps rate for JPEG images with the following caveat. JPEGs around 100K bytes or less, we would see virtually no missed frames.  As the JPEG size went up from there we would see a corresponding increase of missed packets, so by around 200K bytes we are seeing a considerable amount of missed frames.

We instrumented some of the usb code and found that there was a very fast response in receiving the packets and so believing above the usb level is where delays are being introduced. We have just started to look at that when your email came in.


I hope this helps and appreciate any input you have.

Regards,
Keith





On Fri, Jan 18, 2013 at 12:51 PM, Marcus Meissner <[hidden email]> wrote:
On Fri, Dec 28, 2012 at 01:26:45AM +0000, Keith wrote:
> I am trying to capture Liveview images from my Canon camera. I want to capture
> them at a rate of 30 fps which I can do using Canon's SDK. However, when using
> libgphoto2 I appear to be getting images at only about 8-10 fps which is too
> slow. I am running Ubuntu 10.04 and have compiled with optimization. I am using
> gp_camera_capture_preview to capture the images in Liveview mode.

Which camera is that, a Powershot or a EOS?

There might be room for improvement.

Ciao, Marcus


------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122912
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|

Re: Canon Liveview with libgphoto2

Marcus Meissner-4
On Fri, Jan 18, 2013 at 04:30:05PM -0800, Keith Laband wrote:

> Hi Marcus,
>
> Thanks for getting back with me. I am using a Canon EOS 5D mark II. In
> liveview mode the 5D has the capability to push out 30 fps. Each frame
> comes within a 32-34 msec rate. We can see this working under Windows with
> the Canon SDK and are trying to duplicate it under Linux using Libgphoto2.
> We are using Ubuntu 10.04 and libgphoto2-2.5.0.
>
> With no modifications to libgphoto2 default build, we get about 8-10 fps in
> general. frame rates are inconsistent though. Sometime we might have a
> frame rate of 40 msec and the next frame is received in 84 msec or so. We
> have found that the logging and debug code was having an effect on the
> frame rate and have figured out how to turn off both logging and debug
> code. This improved performance significantly to about 15-20 fps, but not
> enough for us.
>
> We started digging into the code more and found several more issues that
> helped performance and resolved some other issues. These included:
>
> In camlibs/ptp2/library.c
>
> Line 1723 in camera_capture_preview changed
>
> <             int             tries = 100;
> ---
> >             int             tries = 1000;
>
> Line 1771-1775 in camera_capture_preview commented out line
>
> <                         xret = ptp_check_eos_events (params);
> <                         if (xret != PTP_RC_OK)
> <                             return translate_ptp_result (xret);
> <                         gp_context_idle (context);
> <                         usleep (50*1000);
> ---
> > //                        xret = ptp_check_eos_events (params);
> > //                        if (xret != PTP_RC_OK)
> > //                            return translate_ptp_result (xret);
> > //                        gp_context_idle (context);
> > //                        usleep (50*1000);
>
> We did the above two changes because if the event was not ready, it would
> sleep for 50 msec before trying again. This would already cause us to miss
> frames. So we increased tries to 1000 and just looped until the next frame
> was there rather than checking for events. 100 tries was not enough, so we
> increased it.


Here I would have recommended just reducing the wait time first.

Some EOS have the issue that if you excessively poll them you will waste
the camera CPU power and might reduce the framerate.


 

> Line 1761 in camera_capture_preview changed
>
> <                     gp_file_append ( file, (char*)data+8, len );
> ---
> >                     gp_file_append ( file, (char*)data+8, len-8 );
>
> Delving into the code, we discovered that go_file_append provided an extra
> 8 bytes of data past the EOI marker, so we just subtracted 8 bytes from the
> length. Further looking into these extra bytes, there appears to be about
> 4k more bytes after the EOI and that this data may have other information
> in it. Could you somehow provide a pointer to it so that it could be
> accessed from the application level?


This is probably difficult :( I will think about that.


> --------------------------------------------------------
>
>
> In camlibs/ptp2/ptp.h we changed line 79
>
> < #define PTP_USB_BULK_HS_MAX_PACKET_LEN_READ   512
> ---
> > #define PTP_USB_BULK_HS_MAX_PACKET_LEN_READ   65535
>
> We just tried increasing this value to see if it would make a difference
> and it did. However, it is probably not a good long term fix.


How much impact did it have?


> -------------------------------------------------
>
> In camlibs/ptp2/usb.c line 290
>
> <         if (usbdata.length == 0xffffffffU) {
> ---
> >         if (usbdata.length >= 0xffffU) {
>
> In this code - " if (usbdata.length == 0xffffffffU)", is checking if
> usbdata.length is exactly 4,294,967,295 bytes long. Probably not what you
> want. We changed it to 65,535 which is the limit of a USB bulk packet. So
> if the data was larger than a USB bulk packet the code would be properly
> executed.

This part of the code is correct with 32bit as-is ...

usbdata.length is the length of the whole data transfer and is stored
in a 32bit unsigned integer. It is not related to the BULK transfer size.
0xffffffffU means "is larger than 4GB" and "transfer until a short read happens".
So this part of the check is for files >4GB and not usually taken.


The code is supposed to go further on to below the retry: label and read the
data there in the <4GB case.


> ----------------------------------------------
> All these changes got us much closer to a 30 fps rate for JPEG images with
> the following caveat. JPEGs around 100K bytes or less, we would see
> virtually no missed frames.  As the JPEG size went up from there we would

Good to hear!

> see a corresponding increase of missed packets, so by around 200K bytes we
> are seeing a considerable amount of missed frames.

The SDK can still do that in that case?

> We instrumented some of the usb code and found that there was a very fast
> response in receiving the packets and so believing above the usb level is
> where delays are being introduced. We have just started to look at that
> when your email came in.
>
> I hope this helps and appreciate any input you have.

I added some comments inline above.

Check the reducing the usleep() amount so you do not poll it
that often (but perhaps the 5D is resistant to this).

Ciao, Marcus

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122912
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel