Camera detection

#1 theo2f

New Member

Members
1 posts

Posted 22 January 2013 - 12:12 PM

Hi, guys.

I'm new to netduino, but very excited about the range of possibilities!

I'm looking a way to develop object detection with video.

When I find some object, like a bottle for example, I wanna which the LED blink some times.

Well, I know that must be done under programming, but I don't know the limit of .NETMF.

Have some material related to this? Do you know something made for video detection + netduino?

Please, any help will be very welcome.

Thank you all.

#2 TKS

New Member

Members
4 posts

Posted 22 January 2013 - 02:41 PM

Well, I know that must be done under programming, but I don't know the limit of .NETMF.

The show-stopper would be probaly ram size. A single video frame at decent resolution is already bigger than 100k.

#3 NooM

Advanced Member

Members
490 posts

LocationAustria

Posted 22 January 2013 - 05:27 PM

i dont think thats possible. object detection needs a lot of resources.

you can send the image taken to pc, render/detect there, than send the result back

My PlayGround and some Code

#4 JerseyTechGuy

Advanced Member

Members
870 posts

Posted 22 January 2013 - 06:15 PM

Having done quite a bit with Motion and Object Detection for both Security and Paranormal Investigation I agree with NooM. The Netduino cannot handle a video feed at a fast enough frame rate to do this kind of detection. You'll need to lean towards a board more like the Raspberry Pi where you can run robust software and use USB Cameras or the like.

#5 Gutworks

Advanced Member

Members
363 posts

LocationOttawa, Ontario

Posted 22 January 2013 - 06:57 PM

First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers

Steve

Twiiter: https://twitter.com/Gutworks

#6 JerseyTechGuy

Advanced Member

Members
870 posts

Posted 22 January 2013 - 07:13 PM

First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers

Steve

Speed and Space issue IMHO. With video you'd be lucky to get 1-3fps with the Netduino+2 which is hardly enough to detect or track motion properly. If you are looking to simply detect objects this may be able to be done since that can be done off a single shot. It really depends on what you are trying to accomplish.

#7 Gutworks

Advanced Member

Members
363 posts

LocationOttawa, Ontario

Posted 22 January 2013 - 07:18 PM

I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video. You make another good point in that it really does depend on what you're trying to accomplish.

Thanks Dave.

Steve

Twiiter: https://twitter.com/Gutworks

#8 CW2

Advanced Member

Members
1592 posts

LocationCzech Republic

Posted 22 January 2013 - 08:31 PM

I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video.

A uncompressed 320�240 video frame with 8-bit color depth (e.g. 256 levels of gray) consumes 75 KB, which is more than available RAM on Netduino and probably at the limit of continuous block on Netduino Plus 2; excluding any intermediate buffers, variables and such.

To receive the video stream at (relatively low rate) 15 fps, the required bandwidth is about 1 MB per second (= 320*240*15).

A trivial algorithm that just reads every pixel, implemented as one loop to access pixels stored in a linear array of bytes in memory will execute approx. 320*240*(1 read + 1 increment + 1 store + 1 compare + 1 jump + 1 array read + what I forgot) instructions, which is not easy to estimate if these are IL (.NET) opcodes, because I don't know exactly how many native instructions each IL opcode interpretation takes. If we assume about 100, then it's roughly 320*240*10*100 = ~77 million instructions, at 15 fps = 76.8*15 = 1152 million instructions per second (MIPS). Netduino's AT91SAM7X claimed performance is 0.9 MIPS/MHz, so at nominal 48 MHz it's about 43 MIPS. Netduino Plus 2's STM32F4 performance is 1.25 DMIPS/MHz (*), at nominal 168 MHz it's 210 DMIPS. Thus, to [just] read the video at 15 fps we'd need processor that is about 26� more powerful than Netuino and about 6� more powerful than Netduino Plus 2; plus power required to actually receive the data (**). Any reasonable image processing algorithm needs to access at least a few adjacent pixels, perform some calculations or make several passes, so the real power required for such image processing is at least one magnitude higher.

(*) DMIPS is not the same as MIPS, but we ignore it, because it's just for illustration.

(**) ~~STM32F405/7~~ STM32F407 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.

Even if you use significant reduction, like checking color of several pixels at predefined locations, neither Netduino nor Netduino Plus 2 has enough power to do that in managed code, not to mention the memory limitation. Maybe, Netduino Plus 2 running native (C/C++) code with the hardware camera interface could handle it, but most likely not full frames...

Edit: Fixed STM32F407 model, 405 does not have DCMI.

Edited by CW2, 23 January 2013 - 03:29 PM.

Gutworks and NooM like this

#9 Gutworks

Advanced Member

Members
363 posts

LocationOttawa, Ontario

Posted 22 January 2013 - 08:43 PM

...and now I know

Great explanation as usual CW2. Thanks!

Twiiter: https://twitter.com/Gutworks

#10 supra

Advanced Member

Members
210 posts

LocationOntario, Canada

Posted 23 January 2013 - 12:47 PM

(**) STM32F405 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.

Looking @ datasheet stating that STM32F405 doesn't support interface with camera module. Only STM32F407 insterface with camera module.

READ DATASHEET.

Gutworks stating that " Gadgeteer has a Camera module for their NEMTF boards".........that is FEZ HYDRA. It incorporates a 240Mhz AT91SAMRL ARM9 processor.

Supra