Columnar Machine: Fast estimation of structured sparse codes

A. Lőrincz, Zoltán Milacski, Balázs Pintér, Anita L. Vero

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the 'Columnar Machine'. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.

Original languageEnglish
Pages (from-to)19-33
Number of pages15
JournalBiologically Inspired Cognitive Architectures
Volume15
DOIs
Publication statusPublished - Jan 1 2016

Fingerprint

Neurons
Neural networks
Feedback
Data storage equipment
Recognition (Psychology)

Keywords

  • Feed-forward inhibition
  • Minicolumns
  • Sparsity
  • Structured representation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Cognitive Neuroscience
  • Experimental and Cognitive Psychology

Cite this

Columnar Machine : Fast estimation of structured sparse codes. / Lőrincz, A.; Milacski, Zoltán; Pintér, Balázs; Vero, Anita L.

In: Biologically Inspired Cognitive Architectures, Vol. 15, 01.01.2016, p. 19-33.

Research output: Contribution to journalArticle

Lőrincz, A. ; Milacski, Zoltán ; Pintér, Balázs ; Vero, Anita L. / Columnar Machine : Fast estimation of structured sparse codes. In: Biologically Inspired Cognitive Architectures. 2016 ; Vol. 15. pp. 19-33.
@article{603912df96e94d88a7722765dbfef98f,
title = "Columnar Machine: Fast estimation of structured sparse codes",
abstract = "Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the 'Columnar Machine'. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.",
keywords = "Feed-forward inhibition, Minicolumns, Sparsity, Structured representation",
author = "A. Lőrincz and Zolt{\'a}n Milacski and Bal{\'a}zs Pint{\'e}r and Vero, {Anita L.}",
year = "2016",
month = "1",
day = "1",
doi = "10.1016/j.bica.2015.10.003",
language = "English",
volume = "15",
pages = "19--33",
journal = "Biologically Inspired Cognitive Architectures",
issn = "2212-683X",
publisher = "Elsevier BV",

}

TY - JOUR

T1 - Columnar Machine

T2 - Fast estimation of structured sparse codes

AU - Lőrincz, A.

AU - Milacski, Zoltán

AU - Pintér, Balázs

AU - Vero, Anita L.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the 'Columnar Machine'. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.

AB - Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the 'Columnar Machine'. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.

KW - Feed-forward inhibition

KW - Minicolumns

KW - Sparsity

KW - Structured representation

UR - http://www.scopus.com/inward/record.url?scp=84957840429&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84957840429&partnerID=8YFLogxK

U2 - 10.1016/j.bica.2015.10.003

DO - 10.1016/j.bica.2015.10.003

M3 - Article

AN - SCOPUS:84957840429

VL - 15

SP - 19

EP - 33

JO - Biologically Inspired Cognitive Architectures

JF - Biologically Inspired Cognitive Architectures

SN - 2212-683X

ER -