maximizing shots per second

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

maximizing shots per second

Jeff Breidenbach
I'm interested in taking and downloading continuous automatic 
camera shots, as quickly as possible. Camera is Sony A7R-ii 
and no memory card. Works great with today's gphoto2 from 
github.

Throughput is about 0.3 shots per seconds using gphoto2 
--capture-image-and-download. Due to pipelining, throughput 
much higher (over 1 hertz)  with gphoto2 --capture-tethered and 
mashing the shutter release. I want the high throughput but not the
physical button pressing.

I suspect that libgphoto2/samples/sample-trigger-capture.c is
exactly what I am looking for. However it fails with the following
message. Looking for guidance.

$ ./sample-trigger-capture 
loop is at 1482029726.742791
loop is at 1482029726.750857
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1482029727.059034
triggering capture 1
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029728.542823
loop is at 1482029728.550838
loop is at 1482029728.558757
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1482029728.863078
loop is at 1482029729.086873
triggering capture 2
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029729.636847
triggering capture 3
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029730.773895
loop is at 1482029730.966827
loop is at 1482029730.974719
Reading PTP event failed: Timeout reading from or writing to the port (-10)
File / / capt0000.jpg added to queue.
starting download 1 (queuelength = 1)
camera readfile of / / capt0000.jpg at offset 0
You need to specify a folder starting with /store_xxxxxxxxx/

*** Contexterror ***              
You need to specify a folder starting with /store_xxxxxxxxx/
'gp_filesystem_read_file (camera->fs, folder, file, type, offset, buf, size, context)' failed: -6
gp_camera_file_read failed: -6

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

marty leisner
Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Are you talking about the examples directory (I don't see samples)

marty

On Sat, Dec 17, 2016 at 10:02 PM, Jeff Breidenbach <[hidden email]> wrote:
I'm interested in taking and downloading continuous automatic 
camera shots, as quickly as possible. Camera is Sony A7R-ii 
and no memory card. Works great with today's gphoto2 from 
github.

Throughput is about 0.3 shots per seconds using gphoto2 
--capture-image-and-download. Due to pipelining, throughput 
much higher (over 1 hertz)  with gphoto2 --capture-tethered and 
mashing the shutter release. I want the high throughput but not the
physical button pressing.

I suspect that libgphoto2/samples/sample-trigger-capture.c is
exactly what I am looking for. However it fails with the following
message. Looking for guidance.

$ ./sample-trigger-capture 
loop is at 1482029726.742791
loop is at 1482029726.750857
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1482029727.059034
triggering capture 1
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029728.542823
loop is at 1482029728.550838
loop is at 1482029728.558757
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1482029728.863078
loop is at 1482029729.086873
triggering capture 2
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029729.636847
triggering capture 3
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Reading PTP event failed: Timeout reading from or writing to the port (-10)
done triggering
loop is at 1482029730.773895
loop is at 1482029730.966827
loop is at 1482029730.974719
Reading PTP event failed: Timeout reading from or writing to the port (-10)
File / / capt0000.jpg added to queue.
starting download 1 (queuelength = 1)
camera readfile of / / capt0000.jpg at offset 0
You need to specify a folder starting with /store_xxxxxxxxx/

*** Contexterror ***              
You need to specify a folder starting with /store_xxxxxxxxx/
'gp_filesystem_read_file (camera->fs, folder, file, type, offset, buf, size, context)' failed: -6
gp_camera_file_read failed: -6

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
> Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Confirmed. Most of the time is presumably spent on things like image 
processing and JPEG compression. Data transfer takes a fairly small 
portion  of the overall time.

>Are you talking about the examples directory (I don't see samples)

You are correct, libgphoto2/examples/sample-trigger-capture.c

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach
In reply to this post by Jeff Breidenbach
> Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Confirmed. Most of the time is presumably spent on things like image 
processing and JPEG compression. Data transfer takes a fairly small 
portion  of the overall time.

>Are you talking about the examples directory (I don't see samples)

You are correct, libgphoto2/examples/sample-trigger-capture.c


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

marty leisner
Well this is interesting, I have the same type of problem.  I'm using a Sony ICLE-6300

I tried a number of things:
      reading images off flash (mounting on linux) I got 13Mbyte/sec.
      saving raw images (not sure what they are, gphoto saves malformed tiff files in 10 seconds)
     small, standard jpeg to large, extra fine jpeg takes 2.5-4 seconds

There seems to be a half second of "startup".

marty


On Sat, Dec 17, 2016 at 10:38 PM, Jeff Breidenbach <[hidden email]> wrote:
> Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Confirmed. Most of the time is presumably spent on things like image 
processing and JPEG compression. Data transfer takes a fairly small 
portion  of the overall time.

>Are you talking about the examples directory (I don't see samples)

You are correct, libgphoto2/examples/sample-trigger-capture.c



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach
I'm not surprised that there is a ~3 second latency from shutter release to completion
of download. Demosaic and other onboard camera processing takes time. I see this
with all tethering software, not just gphoto2.  The trick for higher throughput is to take 
advantage of pipelining; e.g. transfer data from one image while another one is getting 
prepared on the camera. 

I think sample-trigger-capture.c does exactly the right thing, I just can't get it to work.
Suspect that gp_camera_file_read() doesn't understand the concept of transferring
an image from RAM rather than the SD card. The rest of gphoto2 doesn't seem to have
any problems.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Marcus Meissner-4
In reply to this post by Jeff Breidenbach
On Sat, Dec 17, 2016 at 07:02:41PM -0800, Jeff Breidenbach wrote:

> I'm interested in taking and downloading continuous automatic
> camera shots, as quickly as possible. Camera is Sony A7R-ii
> and no memory card. Works great with today's gphoto2 from
> github.
>
> Throughput is about 0.3 shots per seconds using gphoto2
> --capture-image-and-download. Due to pipelining, throughput
> much higher (over 1 hertz)  with gphoto2 --capture-tethered and
> mashing the shutter release. I want the high throughput but not the
> physical button pressing.
>
> I suspect that libgphoto2/samples/sample-trigger-capture.c is
> exactly what I am looking for. However it fails with the following
> message. Looking for guidance.
>
> $ ./sample-trigger-capture

The tool was buggy for the Sony capture ... I fixed it in current git.

With the Sony A58 it gets 1 picture per second if I am counting
right.

Ciao, Marcus

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Keith
In reply to this post by Jeff Breidenbach
A couple of years ago I was using libphoto2 and gphoto2 to get 30 frames per second in liveview mode on a Canon EOS 5D Mark II. It required the removal of all the debug and some optimizations to get that frame rate. If interested, I can try to locate specific information on what the changes were to obtain the 30 fps rate.

Regards, Keith


On Sun, Dec 18, 2016 at 12:51 PM, <[hidden email]> wrote:
Send Gphoto-devel mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.sourceforge.net/lists/listinfo/gphoto-devel
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Gphoto-devel digest..."


Today's Topics:

   1. Re: maximizing shots per second (Jeff Breidenbach)
   2. Re: maximizing shots per second (Jeff Breidenbach)
   3. Re: maximizing shots per second (marty leisner)
   4. Beginner: Using libgphoto2,       how to find set-config-values?
      (Karmaschinken)
   5. Re: maximizing shots per second (Jeff Breidenbach)
   6. Re: Beginner: Using libgphoto2, how to find
      set-config-values? (Marcus Meissner)
   7. Re: maximizing shots per second (Marcus Meissner)


----------------------------------------------------------------------

Message: 1
Date: Sat, 17 Dec 2016 19:35:40 -0800
From: Jeff Breidenbach <[hidden email]>
Subject: Re: [gphoto-devel] maximizing shots per second
To: marty leisner <[hidden email]>
Cc: [hidden email]
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

> Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Confirmed. Most of the time is presumably spent on things like image
processing and JPEG compression. Data transfer takes a fairly small
portion  of the overall time.

>Are you talking about the examples directory (I don't see samples)

You are correct, libgphoto2/examples/sample-trigger-capture.c
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 2
Date: Sat, 17 Dec 2016 19:38:51 -0800
From: Jeff Breidenbach <[hidden email]>
Subject: Re: [gphoto-devel] maximizing shots per second
To: marty leisner <[hidden email]>
Cc: [hidden email]
Message-ID:
        <CAHjiUbo0-0p2iyNKJrs+OWTf1JBq1f3FWzAZehVE=[hidden email]>
Content-Type: text/plain; charset="utf-8"

> Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)

Confirmed. Most of the time is presumably spent on things like image
processing and JPEG compression. Data transfer takes a fairly small
portion  of the overall time.

>Are you talking about the examples directory (I don't see samples)

You are correct, libgphoto2/examples/sample-trigger-capture.c
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 3
Date: Sun, 18 Dec 2016 11:15:18 -0500
From: marty leisner <[hidden email]>
Subject: Re: [gphoto-devel] maximizing shots per second
To: Jeff Breidenbach <[hidden email]>
Cc: [hidden email]
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Well this is interesting, I have the same type of problem.  I'm using a
Sony ICLE-6300

I tried a number of things:
      reading images off flash (mounting on linux) I got 13Mbyte/sec.
      saving raw images (not sure what they are, gphoto saves malformed
tiff files in 10 seconds)
     small, standard jpeg to large, extra fine jpeg takes 2.5-4 seconds

There seems to be a half second of "startup".

marty


On Sat, Dec 17, 2016 at 10:38 PM, Jeff Breidenbach <[hidden email]>
wrote:

> > Is .3 shots/second about 3 seconds/shot?  (I want to confirm this)
>
> Confirmed. Most of the time is presumably spent on things like image
> processing and JPEG compression. Data transfer takes a fairly small
> portion  of the overall time.
>
> >Are you talking about the examples directory (I don't see samples)
>
> You are correct, libgphoto2/examples/sample-trigger-capture.c
>
>
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 4
Date: Sun, 18 Dec 2016 09:37:01 -0700 (MST)
From: Karmaschinken <[hidden email]>
Subject: [gphoto-devel] Beginner: Using libgphoto2,     how to find
        set-config-values?
To: [hidden email]
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

Hello girls and guys!

I successfully installed gphoto2 and gphoto2-python and now am able to run
the python samples using my camera.

Now before I switched to python, I tried a little with gphoto2-CLI, and
there I have the --list-config command, which lists the values that I can
use to remote control my camera or query informations.

Now these commands show up like this:

$ gphoto2 --list-config
.
.
/main/settings/reviewtime
/main/imgsettings/iso
/main/capturesettings/picturestyle
.
.

But how do I find the values within ligphoto2? Does anyone have an example
how to query the camera for a list of values (or indizes) that can be read
and set?

And also, does anyone know if the basic values are camera independent? I
mean, when setting the aperture an a Canon 600D, will the same code also set
the aperture on another Canon or even on a Nikon or so?

Thanks a lot! Any help appreciated very much!

Martin



--
View this message in context: http://gphoto-software.10949.n7.nabble.com/Beginner-Using-libgphoto2-how-to-find-set-config-values-tp16449.html
Sent from the gphoto-devel mailing list archive at Nabble.com.



------------------------------

Message: 5
Date: Sun, 18 Dec 2016 10:40:02 -0800
From: Jeff Breidenbach <[hidden email]>
Subject: Re: [gphoto-devel] maximizing shots per second
To: marty leisner <[hidden email]>
Cc: [hidden email]
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

I'm not surprised that there is a ~3 second latency from shutter release to
completion
of download. Demosaic and other onboard camera processing takes time. I see
this
with all tethering software, not just gphoto2.  The trick for higher
throughput is to take
advantage of pipelining; e.g. transfer data from one image while another
one is getting
prepared on the camera.

I think sample-trigger-capture.c does exactly the right thing, I just can't
get it to work.
Suspect that gp_camera_file_read() doesn't understand the concept of
transferring
an image from RAM rather than the SD card. The rest of gphoto2 doesn't seem
to have
any problems.
-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------

Message: 6
Date: Sun, 18 Dec 2016 21:46:23 +0100
From: Marcus Meissner <[hidden email]>
Subject: Re: [gphoto-devel] Beginner: Using libgphoto2, how to find
        set-config-values?
To: Karmaschinken <[hidden email]>
Cc: [hidden email]
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

Hi,

On Sun, Dec 18, 2016 at 09:37:01AM -0700, Karmaschinken wrote:
> Hello girls and guys!
>
> I successfully installed gphoto2 and gphoto2-python and now am able to run
> the python samples using my camera.
>
> Now before I switched to python, I tried a little with gphoto2-CLI, and
> there I have the --list-config command, which lists the values that I can
> use to remote control my camera or query informations.
>
> Now these commands show up like this:
>
> $ gphoto2 --list-config
> .
> .
> /main/settings/reviewtime
> /main/imgsettings/iso
> /main/capturesettings/picturestyle
> .
> .
>
> But how do I find the values within ligphoto2? Does anyone have an example
> how to query the camera for a list of values (or indizes) that can be read
> and set?
>
> And also, does anyone know if the basic values are camera independent? I
> mean, when setting the aperture an a Canon 600D, will the same code also set
> the aperture on another Canon or even on a Nikon or so?
>
> Thanks a lot! Any help appreciated very much!

We try to keep the semantics of the values the same between camera brands. So the
values for aperture, shutterspeed, iso etc. should be same between cameras.

Which python bindings are those? They likely offer a list option.

gp_camera_list_config() is very new, and gives a flat list of the short names,
gp_camera_get_single_config() would be getting the CameraWidget of a single
value.

gp_camera_get_config() would be the older version that gets the whole configuration tree,
including the values.

Ciao, Marcus



------------------------------

Message: 7
Date: Sun, 18 Dec 2016 21:48:33 +0100
From: Marcus Meissner <[hidden email]>
Subject: Re: [gphoto-devel] maximizing shots per second
To: [hidden email]
Cc: [hidden email]
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

On Sat, Dec 17, 2016 at 07:02:41PM -0800, Jeff Breidenbach wrote:
> I'm interested in taking and downloading continuous automatic
> camera shots, as quickly as possible. Camera is Sony A7R-ii
> and no memory card. Works great with today's gphoto2 from
> github.
>
> Throughput is about 0.3 shots per seconds using gphoto2
> --capture-image-and-download. Due to pipelining, throughput
> much higher (over 1 hertz)  with gphoto2 --capture-tethered and
> mashing the shutter release. I want the high throughput but not the
> physical button pressing.
>
> I suspect that libgphoto2/samples/sample-trigger-capture.c is
> exactly what I am looking for. However it fails with the following
> message. Looking for guidance.
>
> $ ./sample-trigger-capture

The tool was buggy for the Sony capture ... I fixed it in current git.

With the Sony A58 it gets 1 picture per second if I am counting
right.

Ciao, Marcus



------------------------------

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot

------------------------------

_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel


End of Gphoto-devel Digest, Vol 127, Issue 3
********************************************


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
In reply to this post by Marcus Meissner-4
Thank you! The program now runs for me. There's a little bit of cleanup 
needed in the error handling for gp_file_get_data_and_size() in line 88.
The A7R-ii is producing 22 shots per minute, which is less than expected. 
I will try to figure out why.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
gp_camera_trigger_capture() was taking more than a second. I'm pretty sure
it was related to the "half press" focus commands in the Sony section of 
camera_trigger_capture(). Which makes sense since I'm using a manual focus 
lens. Commenting that portion out brings us up to about 41 shots per minute. 
Which is still below my goal of 1 hertz, but much better.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
gp_camera_wait_for_event() takes 1.1 seconds for GP_EVENT_FILE_ADDED.  
This seems high, considering only 6MB is transferred. Any thoughts?

---

1482121557.620650 gp_camera_wait_for_event start
waiting for events timeout 100 ms
Current port timeout is 20000 milliseconds.
Setting port timeout to 150 milliseconds.
Setting port timeout to 20000 milliseconds.
Reading PTP event failed: Timeout reading from or writing to the port (-10)
Sending PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) request...
Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) data...
Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) response...
DEBUG== 0xd215 after capture = 32788
SONY ObjectInMemory count change seen, retrieving file
Sending PTP_OC 0x1008 (Get object info) (0xffffc001) request...
Reading PTP_OC 0x1008 (Get object info) data...
Reading PTP_OC 0x1008 (Get object info) response...
ObjectInfo for 'DSC06019.JPG':
  Object ID: 0xffffc001
  StorageID: 0x00010000
  ObjectFormat: 0x3801
  ProtectionStatus: 0x0000
  ObjectCompressedSize: 6199497
  ThumbFormat: 0x0000
  ThumbCompressedSize: 0
  ThumbPixWidth: 0
  ThumbPixHeight: 0
  ImagePixWidth: 0
  ImagePixHeight: 0
  ImageBitDepth: 0
  ParentObject: 0x00000000
  AssociationType: 0x0000
  AssociationDesc: 0x00000000
  SequenceNumber: 0x00000000
  ModificationDate: 0x00000000
  CaptureDate: 0x00000000
trying to get object size=0x5e98c9
Sending PTP_OC 0x1009 (Get object) (0xffffc001) request...
Reading PTP_OC 0x1009 (Get object) data...
0x183a3d8 with status 0
requeuing complete transfer 0x183a3d8
0x183a5e8 with status 0
requeuing complete transfer 0x183a5e8
Reading PTP_OC 0x1009 (Get object) response...
Append //capt0009.jpg to filesystem
Lookup folder '/'...
Found! / is 0x1831460
Internal append capt0009.jpg to folder /
Adding file 'capt0009.jpg' to folder '/' (type 1)...
Lookup folder / file capt0009.jpg
Lookup folder '/'...
Found! / is 0x1831460
Adding file 'capt0009.jpg' from folder '/' to the fscache LRU list (type 1)...
Lookup folder / file capt0009.jpg
Lookup folder '/'...
Found! / is 0x1831460
File 'capt0009.jpg' from folder '/' added in fscache LRU list.
Checking fscache LRU list integrity...
fscache LRU list ok with 1 items (61362832 bytes)
1482121558.746412 gp_camera_wait_for_event done 
1482121560.044420 evtype was 2


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
In this trace, we "requeue complete transfer" five times, consuming well over a 
second for 11MB of data. Does anyone know why this happens?

3.126  trying to get object size=0xad0a61
3.126  Sending PTP_OC 0x1009 (Get object) (0xffffc001) request...
3.127  Reading PTP_OC 0x1009 (Get object) data...
3.407  0xfed838 with status 0
3.407  requeuing complete transfer 0xfed838
3.563  0xfe7fc8 with status 0
3.563  requeuing complete transfer 0xfe7fc8
3.935  0xfef8b8 with status 0
3.935  requeuing complete transfer 0xfef8b8
4.223  0xfe8d88 with status 0
4.223  requeuing complete transfer 0xfe8d88
4.568  0xfe7d18 with status 0
4.568  requeuing complete transfer 0xfe7d18
4.568  Reading PTP_OC 0x1009 (Get object) response...
4.571  Append //capt0005.jpg to filesystem


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Marcus Meissner-4
In reply to this post by Jeff Breidenbach-2
On Sun, Dec 18, 2016 at 08:47:46PM -0800, Jeff Breidenbach wrote:
> gp_camera_wait_for_event() takes 1.1 seconds for GP_EVENT_FILE_ADDED.
> This seems high, considering only 6MB is transferred. Any thoughts?

Could you regular time image downloads using gphoto2 -P or similar
just to get a feeling of how much time that takes?

Ciao, Marcus

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Marcus Meissner-4
In reply to this post by Jeff Breidenbach-2
On Mon, Dec 19, 2016 at 11:03:03AM -0800, Jeff Breidenbach wrote:

> In this trace, we "requeue complete transfer" five times, consuming well
> over a
> second for 11MB of data. Does anyone know why this happens?
>
> 3.126  trying to get object size=0xad0a61
> 3.126  Sending PTP_OC 0x1009 (Get object) (0xffffc001) request...
> 3.127  Reading PTP_OC 0x1009 (Get object) data...
> 3.407  0xfed838 with status 0
> 3.407  requeuing complete transfer 0xfed838
> 3.563  0xfe7fc8 with status 0
> 3.563  requeuing complete transfer 0xfe7fc8
> 3.935  0xfef8b8 with status 0
> 3.935  requeuing complete transfer 0xfef8b8
> 4.223  0xfe8d88 with status 0
> 4.223  requeuing complete transfer 0xfe8d88
> 4.568  0xfe7d18 with status 0
> 4.568  requeuing complete transfer 0xfe7d18
> 4.568  Reading PTP_OC 0x1009 (Get object) response...
> 4.571  Append //capt0005.jpg to filesystem

These paired lines:

> 4.568  0xfe7d18 with status 0
> 4.568  requeuing complete transfer 0xfe7d18

are the interleaved USB Interrupt handling. They basically happen between
bulk blocks and are small, like 16 bytes usually. You see that they take
up no noticable time.

What you can try, unrelated to these lines, is increasing the read buffer size,
in camlibs/ptp2/usb.c

In current git (and your version) it is:
#define READLEN 512*1024 /* read blob size, mostly to avoid reading all of it at once. */

Bump to 1024*1024 or 2048*1024, but I think it will not get much faster.

CIao, Marcus

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
I set READLEN set to 14MB, and commented out the focus commands. 
The images for this test are 12MB JPEG. 

If I place 21 images in RAM with gpoto2 --trigger-capture, I can download
them with gpthoto2 --cpature-tethered at a rate of 60 images per minute.

If I run examples/sample-trigger-capture, I can capture and download 35
images per minute.

Will do some deeper investigation to see exactly where the time is going.



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
Getting back to maximizing shots-per-second, I pre-placed a bunch of
photographs in RAM then profiled downloading.  About half the time is
spent on data transfer, and the other half is inside camera_wait_for_event().
That part is a little mysterious to me, any idea what is going on during this 
time?

790 ms : XXX entering sony do loop in camera_wait_for_event
790 ms : Current port timeout is 20000 milliseconds.
790 ms : Setting port timeout to 150 milliseconds.
790 ms : 0x14879b8 with status 0
790 ms : requeuing complete transfer 0x14879b8
790 ms : Setting port timeout to 20000 milliseconds.
790 ms : event: nparams=0x1, code=0xC202, trans_id=0xFFFFFFFF, p1=0xFFFFC001, p2=0x0, p3=0x0
790 ms : waiting for events timeout 500 ms
790 ms : XXX entering sony do loop in camera_wait_for_event
790 ms : Current port timeout is 20000 milliseconds.
790 ms : Setting port timeout to 150 milliseconds.
804 ms : 0x1486018 with status 0
804 ms : requeuing complete transfer 0x1486018
804 ms : Setting port timeout to 20000 milliseconds.
804 ms : event: nparams=0x1, code=0xC201, trans_id=0xFFFFFFFF, p1=0xFFFFC001, p2=0x0, p3=0x0
804 ms : waiting for events timeout 487 ms
804 ms : XXX entering sony do loop in camera_wait_for_event
804 ms : Current port timeout is 20000 milliseconds.
804 ms : Setting port timeout to 150 milliseconds.
1104 ms : Setting port timeout to 20000 milliseconds.
1104 ms : Reading PTP event failed: Timeout reading from or writing to the port (-10)
1105 ms : Sending PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) request...
1105 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) data...
1108 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) response...
1108 ms : ptp_sony_getalldevicepropdesc: d215: value 32790 -> 32789
1108 ms : DEBUG== 0xd215 after capture = 32789
1108 ms : SONY ObjectInMemory count change seen, retrieving file




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel

profile.png (35K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Marcus Meissner-4
Hi Jeff,

There was one place where we waited an additional 200ms.

I applied the attached patch to current GIT, perhaps this makes it faster.

Ciao, Marcus
On Wed, Jan 04, 2017 at 03:51:33PM -0800, Jeff Breidenbach wrote:

> Getting back to maximizing shots-per-second, I pre-placed a bunch of
> photographs in RAM then profiled downloading.  About half the time is
> spent on data transfer, and the other half is
> inside camera_wait_for_event().
> That part is a little mysterious to me, any idea what is going on during
> this
> time?
>
> 790 ms : XXX entering sony do loop in camera_wait_for_event
> 790 ms : Current port timeout is 20000 milliseconds.
> 790 ms : Setting port timeout to 150 milliseconds.
> 790 ms : 0x14879b8 with status 0
> 790 ms : requeuing complete transfer 0x14879b8
> 790 ms : Setting port timeout to 20000 milliseconds.
> 790 ms : event: nparams=0x1, code=0xC202, trans_id=0xFFFFFFFF,
> p1=0xFFFFC001, p2=0x0, p3=0x0
> 790 ms : waiting for events timeout 500 ms
> 790 ms : XXX entering sony do loop in camera_wait_for_event
> 790 ms : Current port timeout is 20000 milliseconds.
> 790 ms : Setting port timeout to 150 milliseconds.
> 804 ms : 0x1486018 with status 0
> 804 ms : requeuing complete transfer 0x1486018
> 804 ms : Setting port timeout to 20000 milliseconds.
> 804 ms : event: nparams=0x1, code=0xC201, trans_id=0xFFFFFFFF,
> p1=0xFFFFC001, p2=0x0, p3=0x0
> 804 ms : waiting for events timeout 487 ms
> 804 ms : XXX entering sony do loop in camera_wait_for_event
> 804 ms : Current port timeout is 20000 milliseconds.
> 804 ms : Setting port timeout to 150 milliseconds.
> 1104 ms : Setting port timeout to 20000 milliseconds.
> 1104 ms : Reading PTP event failed: Timeout reading from or writing to the
> port (-10)
> 1105 ms : Sending PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData)
> request...
> 1105 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData) data...
> 1108 ms : Reading PTP_OC 0x9209 (PTP_OC_SONY_GetAllDevicePropData)
> response...
> 1108 ms : ptp_sony_getalldevicepropdesc: d215: value 32790 -> 32789
> 1108 ms : DEBUG== 0xd215 after capture = 32789
> 1108 ms : SONY ObjectInMemory count change seen, retrieving file


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel

xx.pat (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: maximizing shots per second

Jeff Breidenbach-2
Marcus, that was helpful. Results are interesting. I pre-loaded the camera 
with 20 near-identical shots in SDRAM, and gave the camera time to settle. 
Then I looped through wait_event_and_download() from sample-trigger-capture.c 

The first 7 images come very quickly, we get them off the camera is roughly
half a second.

loop is at 1484952941.731218
File / / capt0003.jpg added to queue.
camera getfile of / / capt0003.jpg
ending download 0, deleting file.
loop is at 1484952942.225195


Then things suddenly slow down, and we get timeouts during camera
communication. When looking at this very closely, the camera mysteriously
takes a long time before setting property 0xd215 (PTP_DPC_SONY_ObjectInMemory)
to the correct value. It's completely mysterious to me, because those pictures
are sitting patiently in RAM. What is the camera doing?

I tried to ask Sony using their preferred mechanism and got nowhere.

Anyway, that's where we are at. Thanks again for your terrific work and the
significant progress on this topic.

loop is at 1484952947.888985
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.201827
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.510054
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.812401
File / / capt0010.jpg added to queue.
camera getfile of / / capt0010.jpg
ending download 0, deleting file.
loop is at 1484952949.335957



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel

sony.txt (159K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Fwd: maximizing shots per second

Jeff Breidenbach-2
(Resending without the big log attachment)

Marcus, that was helpful. Results are interesting. I pre-loaded the camera 
with 20 near-identical shots in SDRAM, and gave the camera time to settle. 
Then I looped through wait_event_and_download() from sample-trigger-capture.c 

The first 7 images come very quickly, we get them off the camera is roughly
half a second.

loop is at 1484952941.731218
File / / capt0003.jpg added to queue.
camera getfile of / / capt0003.jpg
ending download 0, deleting file.
loop is at 1484952942.225195


Then things suddenly slow down, and we get timeouts during camera
communication. When looking at this very closely, the camera mysteriously
takes a long time before setting property 0xd215 (PTP_DPC_SONY_ObjectInMemory)
to the correct value. It's completely mysterious to me, because those pictures
are sitting patiently in RAM. What is the camera doing?

I tried to ask Sony using their preferred mechanism and got nowhere.

Anyway, that's where we are at. Thanks again for your terrific work and the
significant progress on this topic.

loop is at 1484952947.888985
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.201827
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.510054
Reading PTP event failed: Timeout reading from or writing to the port (-10)
loop is at 1484952948.812401
File / / capt0010.jpg added to queue.
camera getfile of / / capt0010.jpg
ending download 0, deleting file.
loop is at 1484952949.335957




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gphoto-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gphoto-devel
Loading...