Camera detection
#1
Posted 22 January 2013 - 12:12 PM
#2
Posted 22 January 2013 - 02:41 PM
Well, I know that must be done under programming, but I don't know the limit of .NETMF.
The show-stopper would be probaly ram size. A single video frame at decent resolution is already bigger than 100k.
#3
Posted 22 January 2013 - 05:27 PM
i dont think thats possible. object detection needs a lot of resources.
you can send the image taken to pc, render/detect there, than send the result back
#4
Posted 22 January 2013 - 06:15 PM
Having done quite a bit with Motion and Object Detection for both Security and Paranormal Investigation I agree with NooM. The Netduino cannot handle a video feed at a fast enough frame rate to do this kind of detection. You'll need to lean towards a board more like the Raspberry Pi where you can run robust software and use USB Cameras or the like.
#5
Posted 22 January 2013 - 06:57 PM
First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers
Steve
Twiiter: https://twitter.com/Gutworks
#6
Posted 22 January 2013 - 07:13 PM
First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers
Steve
Speed and Space issue IMHO. With video you'd be lucky to get 1-3fps with the Netduino+2 which is hardly enough to detect or track motion properly. If you are looking to simply detect objects this may be able to be done since that can be done off a single shot. It really depends on what you are trying to accomplish.
#7
Posted 22 January 2013 - 07:18 PM
I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video. You make another good point in that it really does depend on what you're trying to accomplish.
Thanks Dave.
Steve
Twiiter: https://twitter.com/Gutworks
#8
Posted 22 January 2013 - 08:31 PM
I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video.
A uncompressed 320×240 video frame with 8-bit color depth (e.g. 256 levels of gray) consumes 75 KB, which is more than available RAM on Netduino and probably at the limit of continuous block on Netduino Plus 2; excluding any intermediate buffers, variables and such.
To receive the video stream at (relatively low rate) 15 fps, the required bandwidth is about 1 MB per second (= 320*240*15).
A trivial algorithm that just reads every pixel, implemented as one loop to access pixels stored in a linear array of bytes in memory will execute approx. 320*240*(1 read + 1 increment + 1 store + 1 compare + 1 jump + 1 array read + what I forgot) instructions, which is not easy to estimate if these are IL (.NET) opcodes, because I don't know exactly how many native instructions each IL opcode interpretation takes. If we assume about 100, then it's roughly 320*240*10*100 = ~77 million instructions, at 15 fps = 76.8*15 = 1152 million instructions per second (MIPS). Netduino's AT91SAM7X claimed performance is 0.9 MIPS/MHz, so at nominal 48 MHz it's about 43 MIPS. Netduino Plus 2's STM32F4 performance is 1.25 DMIPS/MHz (*), at nominal 168 MHz it's 210 DMIPS. Thus, to [just] read the video at 15 fps we'd need processor that is about 26× more powerful than Netuino and about 6× more powerful than Netduino Plus 2; plus power required to actually receive the data (**). Any reasonable image processing algorithm needs to access at least a few adjacent pixels, perform some calculations or make several passes, so the real power required for such image processing is at least one magnitude higher.
(*) DMIPS is not the same as MIPS, but we ignore it, because it's just for illustration.
(**) STM32F405/7 STM32F407 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.
Even if you use significant reduction, like checking color of several pixels at predefined locations, neither Netduino nor Netduino Plus 2 has enough power to do that in managed code, not to mention the memory limitation. Maybe, Netduino Plus 2 running native (C/C++) code with the hardware camera interface could handle it, but most likely not full frames...
Edit: Fixed STM32F407 model, 405 does not have DCMI.
Edited by CW2, 23 January 2013 - 03:29 PM.
#9
Posted 22 January 2013 - 08:43 PM
...and now I know
Great explanation as usual CW2. Thanks!
Twiiter: https://twitter.com/Gutworks
#10
Posted 23 January 2013 - 12:47 PM
(**) STM32F405 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.
Looking @ datasheet stating that STM32F405 doesn't support interface with camera module. Only STM32F407 insterface with camera module.
READ DATASHEET.
Gutworks stating that " Gadgeteer has a Camera module for their NEMTF boards".........that is FEZ HYDRA. It incorporates a 240Mhz AT91SAMRL ARM9 processor.
Supra
#11
Posted 23 January 2013 - 03:28 PM
Looking @ datasheet stating that STM32F405 doesn't support interface with camera module. Only STM32F407 insterface with camera module.
Good catch, fixed. Thanks for reporting.
Also tagged with one or more of these keywords: visual studio, camera, detection, neural networks, video
Hardware →
Netduino 3 →
"An unhandled exception of type 'System.Exception' occurred in Microsoft.SPOT.Hardware.dll" Error with N3 on VS...Started by ishotjr, 05 Jul 2015 Visual Studio, VS2015 and 1 more... |
|
|||
Answered
Hardware →
Netduino 3 →
Visual Studio version for N3?Started by ishotjr, 03 Jul 2015 VS, N3, VS2013, VS2015 and 1 more... |
|
|||
Programming Languages and Tools →
Visual Studio →
Visual Studio 2015Started by Nevyn, 29 Jun 2015 Visual Studio |
|
|||
General →
General Discussion →
Visual Studio, Netduino - Mac or SurfaceStarted by Nevyn, 06 Jun 2015 Visual Studio, Mac, Surface Pro 3 |
|
|||
Hardware →
Netduino Go →
New to Netduino Go, question about VS environment.Started by Robert Hugo, 01 Jun 2014 Visual Studio, Netduino Go and 1 more... |
|
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users