I'm interested in taking and downloading continuous automatic camera shots, as quickly as possible. Camera is Sony A7R-ii and no memory card. Works great with today's gphoto2 from github. Throughput is about 0.3 shots per seconds using gphoto2 --capture-image-and-download. Due to pipelining, throughput much higher (over 1 hertz) with gphoto2 --capture-tethered and mashing the shutter release. I want the high throughput but not the physical button pressing. I suspect that libgphoto2/samples/sample-trigger-capture.c is exactly what I am looking for. However it fails with the following message. Looking for guidance. $ ./sample-trigger-capture loop is at 1482029726.742791 loop is at 1482029726.750857 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1482029727.059034 triggering capture 1 Reading PTP event failed: Timeout reading from or writing to the port (-10) Reading PTP event failed: Timeout reading from or writing to the port (-10) Reading PTP event failed: Timeout reading from or writing to the port (-10) done triggering loop is at 1482029728.542823 loop is at 1482029728.550838 loop is at 1482029728.558757 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1482029728.863078 loop is at 1482029729.086873 triggering capture 2 Reading PTP event failed: Timeout reading from or writing to the port (-10) done triggering loop is at 1482029729.636847 triggering capture 3 Reading PTP event failed: Timeout reading from or writing to the port (-10) Reading PTP event failed: Timeout reading from or writing to the port (-10) done triggering loop is at 1482029730.773895 loop is at 1482029730.966827 loop is at 1482029730.974719 Reading PTP event failed: Timeout reading from or writing to the port (-10) File / / capt0000.jpg added to queue. starting download 1 (queuelength = 1) camera readfile of / / capt0000.jpg at offset 0 You need to specify a folder starting with /store_xxxxxxxxx/ *** Contexterror *** You need to specify a folder starting with /store_xxxxxxxxx/ 'gp_filesystem_read_file (camera->fs, folder, file, type, offset, buf, size, context)' failed: -6 gp_camera_file_read failed: -6 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Is .3 shots/second about 3 seconds/shot? (I want to confirm this) Are you talking about the examples directory (I don't see samples)On Sat, Dec 17, 2016 at 10:02 PM, Jeff Breidenbach <[hidden email]> wrote:
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
> Is .3 shots/second about 3 seconds/shot? (I want to confirm this)
Confirmed. Most of the time is presumably spent on things like image processing and JPEG compression. Data transfer takes a fairly small portion of the overall time. >Are you talking about the examples directory (I don't see samples) You are correct, libgphoto2/examples/sample-trigger-capture.c ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Jeff Breidenbach
> Is .3 shots/second about 3 seconds/shot? (I want to confirm this) Confirmed. Most of the time is presumably spent on things like image processing and JPEG compression. Data transfer takes a fairly small portion of the overall time. >Are you talking about the examples directory (I don't see samples) You are correct, libgphoto2/examples/sample-trigger-capture.c ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Well this is interesting, I have the same type of problem. I'm using a Sony ICLE-6300 I tried a number of things:On Sat, Dec 17, 2016 at 10:38 PM, Jeff Breidenbach <[hidden email]> wrote:
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
I'm not surprised that there is a ~3 second latency from shutter release to completion
of download. Demosaic and other onboard camera processing takes time. I see this with all tethering software, not just gphoto2. The trick for higher throughput is to take advantage of pipelining; e.g. transfer data from one image while another one is getting prepared on the camera. I think sample-trigger-capture.c does exactly the right thing, I just can't get it to work. Suspect that gp_camera_file_read() doesn't understand the concept of transferring an image from RAM rather than the SD card. The rest of gphoto2 doesn't seem to have any problems. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Jeff Breidenbach
On Sat, Dec 17, 2016 at 07:02:41PM -0800, Jeff Breidenbach wrote:
> I'm interested in taking and downloading continuous automatic > camera shots, as quickly as possible. Camera is Sony A7R-ii > and no memory card. Works great with today's gphoto2 from > github. > > Throughput is about 0.3 shots per seconds using gphoto2 > --capture-image-and-download. Due to pipelining, throughput > much higher (over 1 hertz) with gphoto2 --capture-tethered and > mashing the shutter release. I want the high throughput but not the > physical button pressing. > > I suspect that libgphoto2/samples/sample-trigger-capture.c is > exactly what I am looking for. However it fails with the following > message. Looking for guidance. > > $ ./sample-trigger-capture The tool was buggy for the Sony capture ... I fixed it in current git. With the Sony A58 it gets 1 picture per second if I am counting right. Ciao, Marcus ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Jeff Breidenbach
A couple of years ago I was using libphoto2 and gphoto2 to get 30 frames per second in liveview mode on a Canon EOS 5D Mark II. It required the removal of all the debug and some optimizations to get that frame rate. If interested, I can try to locate specific information on what the changes were to obtain the 30 fps rate.
Regards, Keith On Sun, Dec 18, 2016 at 12:51 PM, <[hidden email]> wrote: Send Gphoto-devel mailing list submissions to ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Marcus Meissner-4
Thank you! The program now runs for me. There's a little bit of cleanup needed in the error handling for gp_file_get_data_and_size() in line 88. The A7R-ii is producing 22 shots per minute, which is less than expected. I will try to figure out why. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
gp_camera_trigger_capture() was taking more than a second. I'm pretty sure it was related to the "half press" focus commands in the Sony section of camera_trigger_capture(). Which makes sense since I'm using a manual focus lens. Commenting that portion out brings us up to about 41 shots per minute. Which is still below my goal of 1 hertz, but much better. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
gp_camera_wait_for_event() takes 1.1 seconds for GP_EVENT_FILE_ADDED.
This seems high, considering only 6MB is transferred. Any thoughts? --- 1482121557.620650 gp_camera_wait_for_event start waiting for events timeout 100 ms Current port timeout is 20000 milliseconds. Setting port timeout to 150 milliseconds. Setting port timeout to 20000 milliseconds. Reading PTP event failed: Timeout reading from or writing to the port (-10) Sending PTP_OC 0x9209 (PTP_OC_SONY_ Reading PTP_OC 0x9209 (PTP_OC_SONY_ Reading PTP_OC 0x9209 (PTP_OC_SONY_ DEBUG== 0xd215 after capture = 32788 SONY ObjectInMemory count change seen, retrieving file Sending PTP_OC 0x1008 (Get object info) (0xffffc001) request... Reading PTP_OC 0x1008 (Get object info) data... Reading PTP_OC 0x1008 (Get object info) response... ObjectInfo for 'DSC06019.JPG': Object ID: 0xffffc001 StorageID: 0x00010000 ObjectFormat: 0x3801 ProtectionStatus: 0x0000 ObjectCompressedSize: 6199497 ThumbFormat: 0x0000 ThumbCompressedSize: 0 ThumbPixWidth: 0 ThumbPixHeight: 0 ImagePixWidth: 0 ImagePixHeight: 0 ImageBitDepth: 0 ParentObject: 0x00000000 AssociationType: 0x0000 AssociationDesc: 0x00000000 SequenceNumber: 0x00000000 ModificationDate: 0x00000000 CaptureDate: 0x00000000 trying to get object size=0x5e98c9 Sending PTP_OC 0x1009 (Get object) (0xffffc001) request... Reading PTP_OC 0x1009 (Get object) data... 0x183a3d8 with status 0 requeuing complete transfer 0x183a3d8 0x183a5e8 with status 0 requeuing complete transfer 0x183a5e8 Reading PTP_OC 0x1009 (Get object) response... Append //capt0009.jpg to filesystem Lookup folder '/'... Found! / is 0x1831460 Internal append capt0009.jpg to folder / Adding file 'capt0009.jpg' to folder '/' (type 1)... Lookup folder / file capt0009.jpg Lookup folder '/'... Found! / is 0x1831460 Adding file 'capt0009.jpg' from folder '/' to the fscache LRU list (type 1)... Lookup folder / file capt0009.jpg Lookup folder '/'... Found! / is 0x1831460 File 'capt0009.jpg' from folder '/' added in fscache LRU list. Checking fscache LRU list integrity... fscache LRU list ok with 1 items (61362832 bytes) 1482121558.746412 gp_camera_wait_for_event done 1482121560.044420 evtype was 2 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In this trace, we "requeue complete transfer" five times, consuming well over a second for 11MB of data. Does anyone know why this happens? 3.126 trying to get object size=0xad0a61 3.126 Sending PTP_OC 0x1009 (Get object) (0xffffc001) request... 3.127 Reading PTP_OC 0x1009 (Get object) data... 3.407 0xfed838 with status 0 3.407 requeuing complete transfer 0xfed838 3.563 0xfe7fc8 with status 0 3.563 requeuing complete transfer 0xfe7fc8 3.935 0xfef8b8 with status 0 3.935 requeuing complete transfer 0xfef8b8 4.223 0xfe8d88 with status 0 4.223 requeuing complete transfer 0xfe8d88 4.568 0xfe7d18 with status 0 4.568 requeuing complete transfer 0xfe7d18 4.568 Reading PTP_OC 0x1009 (Get object) response... 4.571 Append //capt0005.jpg to filesystem ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/intel _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Jeff Breidenbach-2
On Sun, Dec 18, 2016 at 08:47:46PM -0800, Jeff Breidenbach wrote:
> gp_camera_wait_for_event() takes 1.1 seconds for GP_EVENT_FILE_ADDED. > This seems high, considering only 6MB is transferred. Any thoughts? Could you regular time image downloads using gphoto2 -P or similar just to get a feeling of how much time that takes? Ciao, Marcus ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/intel _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
In reply to this post by Jeff Breidenbach-2
On Mon, Dec 19, 2016 at 11:03:03AM -0800, Jeff Breidenbach wrote:
> In this trace, we "requeue complete transfer" five times, consuming well > over a > second for 11MB of data. Does anyone know why this happens? > > 3.126 trying to get object size=0xad0a61 > 3.126 Sending PTP_OC 0x1009 (Get object) (0xffffc001) request... > 3.127 Reading PTP_OC 0x1009 (Get object) data... > 3.407 0xfed838 with status 0 > 3.407 requeuing complete transfer 0xfed838 > 3.563 0xfe7fc8 with status 0 > 3.563 requeuing complete transfer 0xfe7fc8 > 3.935 0xfef8b8 with status 0 > 3.935 requeuing complete transfer 0xfef8b8 > 4.223 0xfe8d88 with status 0 > 4.223 requeuing complete transfer 0xfe8d88 > 4.568 0xfe7d18 with status 0 > 4.568 requeuing complete transfer 0xfe7d18 > 4.568 Reading PTP_OC 0x1009 (Get object) response... > 4.571 Append //capt0005.jpg to filesystem These paired lines: > 4.568 0xfe7d18 with status 0 > 4.568 requeuing complete transfer 0xfe7d18 are the interleaved USB Interrupt handling. They basically happen between bulk blocks and are small, like 16 bytes usually. You see that they take up no noticable time. What you can try, unrelated to these lines, is increasing the read buffer size, in camlibs/ptp2/usb.c In current git (and your version) it is: #define READLEN 512*1024 /* read blob size, mostly to avoid reading all of it at once. */ Bump to 1024*1024 or 2048*1024, but I think it will not get much faster. CIao, Marcus ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/intel _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
I set READLEN set to 14MB, and commented out the focus commands. The images for this test are 12MB JPEG. If I place 21 images in RAM with gpoto2 --trigger-capture, I can download them with gpthoto2 --cpature-tethered at a rate of 60 images per minute. If I run examples/sample-trigger-capture, I can capture and download 35 images per minute. Will do some deeper investigation to see exactly where the time is going. ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/intel _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Getting back to maximizing shots-per-second, I pre-placed a bunch of photographs in RAM then profiled downloading. About half the time is spent on data transfer, and the other half is inside camera_wait_for_event(). That part is a little mysterious to me, any idea what is going on during this time? 790 ms : XXX entering sony do loop in camera_wait_for_event 790 ms : Current port timeout is 20000 milliseconds. 790 ms : Setting port timeout to 150 milliseconds. 790 ms : 0x14879b8 with status 0 790 ms : requeuing complete transfer 0x14879b8 790 ms : Setting port timeout to 20000 milliseconds. 790 ms : event: nparams=0x1, code=0xC202, trans_id=0xFFFFFFFF, p1=0xFFFFC001, p2=0x0, p3=0x0 790 ms : waiting for events timeout 500 ms 790 ms : XXX entering sony do loop in camera_wait_for_event 790 ms : Current port timeout is 20000 milliseconds. 790 ms : Setting port timeout to 150 milliseconds. 804 ms : 0x1486018 with status 0 804 ms : requeuing complete transfer 0x1486018 804 ms : Setting port timeout to 20000 milliseconds. 804 ms : event: nparams=0x1, code=0xC201, trans_id=0xFFFFFFFF, p1=0xFFFFC001, p2=0x0, p3=0x0 804 ms : waiting for events timeout 487 ms 804 ms : XXX entering sony do loop in camera_wait_for_event 804 ms : Current port timeout is 20000 milliseconds. 804 ms : Setting port timeout to 150 milliseconds. 1104 ms : Setting port timeout to 20000 milliseconds. 1104 ms : Reading PTP event failed: Timeout reading from or writing to the port (-10) 1105 ms : Sending PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) request... 1105 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) data... 1108 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) response... 1108 ms : ptp_sony_getalldevicepropdesc: d215: value 32790 -> 32789 1108 ms : DEBUG== 0xd215 after capture = 32789 1108 ms : SONY ObjectInMemory count change seen, retrieving file ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Hi Jeff,
There was one place where we waited an additional 200ms. I applied the attached patch to current GIT, perhaps this makes it faster. Ciao, Marcus On Wed, Jan 04, 2017 at 03:51:33PM -0800, Jeff Breidenbach wrote: > Getting back to maximizing shots-per-second, I pre-placed a bunch of > photographs in RAM then profiled downloading. About half the time is > spent on data transfer, and the other half is > inside camera_wait_for_event(). > That part is a little mysterious to me, any idea what is going on during > this > time? > > 790 ms : XXX entering sony do loop in camera_wait_for_event > 790 ms : Current port timeout is 20000 milliseconds. > 790 ms : Setting port timeout to 150 milliseconds. > 790 ms : 0x14879b8 with status 0 > 790 ms : requeuing complete transfer 0x14879b8 > 790 ms : Setting port timeout to 20000 milliseconds. > 790 ms : event: nparams=0x1, code=0xC202, trans_id=0xFFFFFFFF, > p1=0xFFFFC001, p2=0x0, p3=0x0 > 790 ms : waiting for events timeout 500 ms > 790 ms : XXX entering sony do loop in camera_wait_for_event > 790 ms : Current port timeout is 20000 milliseconds. > 790 ms : Setting port timeout to 150 milliseconds. > 804 ms : 0x1486018 with status 0 > 804 ms : requeuing complete transfer 0x1486018 > 804 ms : Setting port timeout to 20000 milliseconds. > 804 ms : event: nparams=0x1, code=0xC201, trans_id=0xFFFFFFFF, > p1=0xFFFFC001, p2=0x0, p3=0x0 > 804 ms : waiting for events timeout 487 ms > 804 ms : XXX entering sony do loop in camera_wait_for_event > 804 ms : Current port timeout is 20000 milliseconds. > 804 ms : Setting port timeout to 150 milliseconds. > 1104 ms : Setting port timeout to 20000 milliseconds. > 1104 ms : Reading PTP event failed: Timeout reading from or writing to the > port (-10) > 1105 ms : Sending PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) > request... > 1105 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) data... > 1108 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) > response... > 1108 ms : ptp_sony_getalldevicepropdesc: d215: value 32790 -> 32789 > 1108 ms : DEBUG== 0xd215 after capture = 32789 > 1108 ms : SONY ObjectInMemory count change seen, retrieving file ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Marcus, that was helpful. Results are interesting. I pre-loaded the camera with 20 near-identical shots in SDRAM, and gave the camera time to settle. Then I looped through wait_event_and_ The first 7 images come very quickly, we get them off the camera is roughly half a second. loop is at 1484952941.731218 File / / capt0003.jpg added to queue. camera getfile of / / capt0003.jpg ending download 0, deleting file. loop is at 1484952942.225195 Then things suddenly slow down, and we get timeouts during camera communication. When looking at this very closely, the camera mysteriously takes a long time before setting property 0xd215 (PTP_DPC_SONY_ObjectInMemory) to the correct value. It's completely mysterious to me, because those pictures are sitting patiently in RAM. What is the camera doing? I tried to ask Sony using their preferred mechanism and got nowhere. Anyway, that's where we are at. Thanks again for your terrific work and the significant progress on this topic. loop is at 1484952947.888985 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.201827 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.510054 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.812401 File / / capt0010.jpg added to queue. camera getfile of / / capt0010.jpg ending download 0, deleting file. loop is at 1484952949.335957 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
(Resending without the big log attachment) Marcus, that was helpful. Results are interesting. I pre-loaded the camera with 20 near-identical shots in SDRAM, and gave the camera time to settle. Then I looped through wait_event_and_downloa The first 7 images come very quickly, we get them off the camera is roughly half a second. loop is at 1484952941.731218 File / / capt0003.jpg added to queue. camera getfile of / / capt0003.jpg ending download 0, deleting file. loop is at 1484952942.225195 Then things suddenly slow down, and we get timeouts during camera communication. When looking at this very closely, the camera mysteriously takes a long time before setting property 0xd215 (PTP_DPC_SONY_ObjectInMemory) to the correct value. It's completely mysterious to me, because those pictures are sitting patiently in RAM. What is the camera doing? I tried to ask Sony using their preferred mechanism and got nowhere. Anyway, that's where we are at. Thanks again for your terrific work and the significant progress on this topic. loop is at 1484952947.888985 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.201827 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.510054 Reading PTP event failed: Timeout reading from or writing to the port (-10) loop is at 1484952948.812401 File / / capt0010.jpg added to queue. camera getfile of / / capt0010.jpg ending download 0, deleting file. loop is at 1484952949.335957 ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Gphoto-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gphoto-devel |
Free forum by Nabble | Edit this page |