Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications

J. Fernández-Berni, R. Carmona-Galán, L. Carranza-González, A. Zarándy, A. Rodríguez-Vázquez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176×144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6mW@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.

Original languageEnglish
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
Volume8012
DOIs
Publication statusPublished - 2011
EventInfrared Technology and Applications XXXVII - Orlando, FL, United States
Duration: Apr 25 2011Apr 29 2011

Other

OtherInfrared Technology and Applications XXXVII
CountryUnited States
CityOrlando, FL
Period4/25/114/29/11

Fingerprint

Image Representation
Focal Plane
Multiresolution
Image sensors
Pixels
Smart sensors
Conveying
Unmanned aerial vehicles (UAV)
Processing
Wireless sensor networks
Robotics
Image processing
Pixel
pixels
distributed processing
Switches
photosensors
Smart Sensors
Specifications
Spatial Filtering

Keywords

  • Focal-plane image processing
  • Power-efficient VLSI implementation
  • Reduced scene representation
  • Wireless sensor networks

ASJC Scopus subject areas

  • Applied Mathematics
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics

Cite this

Fernández-Berni, J., Carmona-Galán, R., Carranza-González, L., Zarándy, A., & Rodríguez-Vázquez, A. (2011). Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 8012). [1] https://doi.org/10.1117/12.883881

Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications. / Fernández-Berni, J.; Carmona-Galán, R.; Carranza-González, L.; Zarándy, A.; Rodríguez-Vázquez, A.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8012 2011. 1.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fernández-Berni, J, Carmona-Galán, R, Carranza-González, L, Zarándy, A & Rodríguez-Vázquez, A 2011, Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 8012, 1, Infrared Technology and Applications XXXVII, Orlando, FL, United States, 4/25/11. https://doi.org/10.1117/12.883881
Fernández-Berni J, Carmona-Galán R, Carranza-González L, Zarándy A, Rodríguez-Vázquez A. Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8012. 2011. 1 https://doi.org/10.1117/12.883881
Fernández-Berni, J. ; Carmona-Galán, R. ; Carranza-González, L. ; Zarándy, A. ; Rodríguez-Vázquez, A. / Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 8012 2011.
@inproceedings{1b4cf121b7074aa1ae76a68ab1b888c7,
title = "Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications",
abstract = "Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176×144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6mW@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.",
keywords = "Focal-plane image processing, Power-efficient VLSI implementation, Reduced scene representation, Wireless sensor networks",
author = "J. Fern{\'a}ndez-Berni and R. Carmona-Gal{\'a}n and L. Carranza-Gonz{\'a}lez and A. Zar{\'a}ndy and A. Rodr{\'i}guez-V{\'a}zquez",
year = "2011",
doi = "10.1117/12.883881",
language = "English",
isbn = "9780819485861",
volume = "8012",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications

AU - Fernández-Berni, J.

AU - Carmona-Galán, R.

AU - Carranza-González, L.

AU - Zarándy, A.

AU - Rodríguez-Vázquez, A.

PY - 2011

Y1 - 2011

N2 - Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176×144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6mW@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.

AB - Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176×144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6mW@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.

KW - Focal-plane image processing

KW - Power-efficient VLSI implementation

KW - Reduced scene representation

KW - Wireless sensor networks

UR - http://www.scopus.com/inward/record.url?scp=80052990220&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052990220&partnerID=8YFLogxK

U2 - 10.1117/12.883881

DO - 10.1117/12.883881

M3 - Conference contribution

AN - SCOPUS:80052990220

SN - 9780819485861

VL - 8012

BT - Proceedings of SPIE - The International Society for Optical Engineering

ER -