Netduino home hardware projects downloads community

Jump to content


The Netduino forums have been replaced by new forums at community.wildernesslabs.co. This site has been preserved for archival purposes only and the ability to make new accounts or posts has been turned off.
Photo

Camera detection

visual studio camera detection neural networks video

  • Please log in to reply
10 replies to this topic

#1 theo2f

theo2f

    New Member

  • Members
  • Pip
  • 1 posts

Posted 22 January 2013 - 12:12 PM

Hi, guys.
 
I'm new to netduino, but very excited about the range of possibilities!
 
I'm looking a way to develop object detection with video.
When I find some object, like a bottle for example, I wanna which the LED blink some times.
 
Well, I know that must be done under programming, but I don't know the limit of .NETMF.
Have some material related to this? Do you know something made for video detection + netduino?
 
Please, any help will be very welcome.
 
Thank you all.


#2 TKS

TKS

    New Member

  • Members
  • Pip
  • 4 posts

Posted 22 January 2013 - 02:41 PM

Well, I know that must be done under programming, but I don't know the limit of .NETMF.

The show-stopper would be probaly ram size. A single video frame at decent resolution is already bigger than 100k.



#3 NooM

NooM

    Advanced Member

  • Members
  • PipPipPip
  • 490 posts
  • LocationAustria

Posted 22 January 2013 - 05:27 PM

i dont think thats possible. object detection needs a lot of resources.

you can send the image taken to pc, render/detect there, than send the result back



#4 JerseyTechGuy

JerseyTechGuy

    Advanced Member

  • Members
  • PipPipPip
  • 870 posts

Posted 22 January 2013 - 06:15 PM

Having done quite a bit with Motion and Object Detection for both Security and Paranormal Investigation I agree with NooM.  The Netduino cannot handle a video feed at a fast enough frame rate to do this kind of detection.  You'll need to lean towards a board more like the Raspberry Pi where you can run robust software and use USB Cameras or the like.



#5 Gutworks

Gutworks

    Advanced Member

  • Members
  • PipPipPip
  • 363 posts
  • LocationOttawa, Ontario

Posted 22 January 2013 - 06:57 PM

First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers :)

 

Steve



#6 JerseyTechGuy

JerseyTechGuy

    Advanced Member

  • Members
  • PipPipPip
  • 870 posts

Posted 22 January 2013 - 07:13 PM

First off I have zero experience with using video on a microcontroller, and so I am curious as to why it wouldn't be possible especially with the faster processing power of the Netduino Plus 2 and the Netduino Go? Is it more of a NETMF issue or a speed issue? Additionally Gadgeteer has a Camera module for their NEMTF boards, so I don't see why a camera couldn't be used with a Netduino. Perhaps if it's a matter of speed and the need for native code, then a Netduino Go Camera module sounds reasonably plausible for someone to create. Anyway, sorry for my many questions and no answers :)

 

Steve

Speed and Space issue IMHO.  With video you'd be lucky to get 1-3fps with the Netduino+2 which is hardly enough to detect or track motion properly.  If you are looking to simply detect objects this may be able to be done since that can be done off a single shot.  It really depends on what you are trying to accomplish.



#7 Gutworks

Gutworks

    Advanced Member

  • Members
  • PipPipPip
  • 363 posts
  • LocationOttawa, Ontario

Posted 22 January 2013 - 07:18 PM

I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video. You make another good point in that it really does depend on what you're trying to accomplish. 

 

Thanks Dave.

 

Steve



#8 CW2

CW2

    Advanced Member

  • Members
  • PipPipPip
  • 1592 posts
  • LocationCzech Republic

Posted 22 January 2013 - 08:31 PM

I hadn't thought of the potential space issue, and obviously I have no idea on how much processing power is required to effectively use video.

 
A uncompressed 320240 video frame with 8-bit color depth (e.g. 256 levels of gray) consumes 75 KB, which is more than available RAM on Netduino and probably at the limit of continuous block on Netduino Plus 2; excluding any intermediate buffers, variables and such.
 
To receive the video stream at (relatively low rate) 15 fps, the required bandwidth is about 1 MB per second (= 320*240*15).
 
A trivial algorithm that just reads every pixel, implemented as one loop to access pixels stored in a linear array of bytes in memory will execute approx. 320*240*(1 read + 1 increment + 1 store + 1 compare + 1 jump + 1 array read + what I forgot) instructions, which is not easy to estimate if these are IL (.NET) opcodes, because I don't know exactly how many native instructions each IL opcode interpretation takes. If we assume about 100, then it's roughly 320*240*10*100 = ~77 million instructions, at 15 fps = 76.8*15 = 1152 million instructions per second (MIPS). Netduino's AT91SAM7X claimed performance is 0.9 MIPS/MHz, so at nominal 48 MHz it's about 43 MIPS. Netduino Plus 2's STM32F4 performance is 1.25 DMIPS/MHz (*), at nominal 168 MHz it's 210 DMIPS. Thus, to [just] read the video at 15 fps we'd need processor that is about 26 more powerful than Netuino and about 6 more powerful than Netduino Plus 2; plus power required to actually receive the data (**). Any reasonable image processing algorithm needs to access at least a few adjacent pixels, perform some calculations or make several passes, so the real power required for such image processing is at least one magnitude higher.
 
(*) DMIPS is not the same as MIPS, but we ignore it, because it's just for illustration.

 

(**) STM32F405/7 STM32F407 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.


Even if you use significant reduction, like checking color of several pixels at predefined locations, neither Netduino nor Netduino Plus 2 has enough power to do that in managed code, not to mention the memory limitation. Maybe, Netduino Plus 2 running native (C/C++) code with the hardware camera interface could handle it, but most likely not full frames...

 

Edit: Fixed STM32F407 model, 405 does not have DCMI.


Edited by CW2, 23 January 2013 - 03:29 PM.


#9 Gutworks

Gutworks

    Advanced Member

  • Members
  • PipPipPip
  • 363 posts
  • LocationOttawa, Ontario

Posted 22 January 2013 - 08:43 PM

...and now I know :)

 

Great explanation as usual CW2. Thanks!



#10 supra

supra

    Advanced Member

  • Members
  • PipPipPip
  • 210 posts
  • LocationOntario, Canada

Posted 23 January 2013 - 12:47 PM

(**) STM32F405 has hardware Digital Camera Interface, up to 54 MB/s. But it's not supported in the current version of .NET MF and the necessary pins are not available on any of Secret Labs boards.

 

 

 

Looking @ datasheet stating that STM32F405 doesn't support interface with camera module. Only STM32F407 insterface with camera module.

 

READ DATASHEET.

 

Gutworks stating that " Gadgeteer has a Camera module for their NEMTF boards".........that is FEZ HYDRA.  It incorporates a 240Mhz AT91SAMRL ARM9 processor.

 

Supra



#11 CW2

CW2

    Advanced Member

  • Members
  • PipPipPip
  • 1592 posts
  • LocationCzech Republic

Posted 23 January 2013 - 03:28 PM

Looking @ datasheet stating that STM32F405 doesn't support interface with camera module. Only STM32F407 insterface with camera module.

Good catch, fixed. Thanks for reporting.







Also tagged with one or more of these keywords: visual studio, camera, detection, neural networks, video

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

home    hardware    projects    downloads    community    where to buy    contact Copyright © 2016 Wilderness Labs Inc.  |  Legal   |   CC BY-SA
This webpage is licensed under a Creative Commons Attribution-ShareAlike License.