Sparse Coding of Sensory Inputs

About this sample

About this sample


Words: 835 |

Pages: 2|

5 min read

Published: Sep 18, 2018

Words: 835|Pages: 2|5 min read

Published: Sep 18, 2018

This paper describes sparse coding of associative memory patterns and its networks usage for large-scale modeling of the brain. The mechanism of associations in human memory was stated earlier by psychologists and philosophers and it has been a topic of research in the field of neuroscience and artificial neural network for over half-century which continues its investigation even today. The working of associative memory in the human brain is observed when we try to remember and cannot recollect a particular bit of information instantly.

'Why Violent Video Games Shouldn't Be Banned'?

At this state, our mind probably contains the features of current scenario and related information (not the appropriate information) about what we are trying to remember which initiates a back-to-back process of associations from one to another (probably by similarity of meaning or logic in human language) eventually arriving to recognize that information which fits appropriately into the context that stimulated our search. So, relating to contextual information our brain probably acts like information system which associates a new output to a particular input. Technically there are three different mechanisms of association process. One is hetero-association which searches one pattern from a category which is associated to another pattern in another category, second is auto-association which searches within a pattern to recall a complete or exact pattern, and third is a special case of auto-association named as a bidirectional association which goes back and forth between two patterns.

Now the question put forth by neuroscientists is how the process of association in mind is related to (or realized in) neurophysiological mechanisms in the brain. A rule named synaptic plasticity formulated by Donald Hebb explains that the strength of the synapses between any two cells is strengthened by repeated firing of one cell to another cell. This is otherwise called Hebbian theory which paved the way for the development of Neural Associative Memory (NAM) models. Neural Associative Memory model stores the weights of synaptic connections between neurons and is stored in a memory storage matrix. The process of storing and retrieving a set of specified patterns is expressed by using an additive rule or binary rule with an appropriate threshold value for a specific set of patterns. Consider a learning process in matrix formation (memory) wherein each learning step at a time, the change of matrix formation (memory) depends upon the product of presynaptic activity and postsynaptic activity at the synapse at that specific time. Thus synaptic change is computed based on additive rule locally across time and space which is termed as local learning rule. The output of this model is binary {0, 1} which contains the stored pattern that is based on Hebb learning rule and the sparseness of these patterns is more productive for storage and retrieval of information.

Efficiency becomes critical when NAM model is used in many technical applications. Vector-matrix multiplication is done for entries in {0, 1} for retrieval and then with counting, threshold and finally, if input patterns are sparse then the retrieval of stored patterns becomes faster. These sparse binary patterns are used in spoken word recognition, face recognition, written letter recognition which has a large number of classes. Video signals are another example where compression codes are based on the sparseness of signal differences. In the primary visual cortex of visual system of humans and animals sparseness works in edge detecting cells and also in antagonistic activation of retinal ganglion cells. Machine learning and signal processing are the other areas where sparseness principle is used other than associative memory. It is even natural to say that human cognition occurs out of sparseness, as we cannot conceive anything as absent like non-dog or non-apple, as it encloses every concept.

NAM functionalities have become important in parallel computing architectures where inter-process communication is a bottleneck, and so the idea of transmitting addresses of few active neurons which saves transmission rate because of sparse activity was adopted into parallel computing. This concept has been used in a pulse coded neural network which focuses on hardware implementation for brain simulations. In neuroscience, NAM has become the surge of interest for spiking neuron models and synchronization. Based on the work done by Fransén & Lansner [1] it is possible to construct a cognitive behavior which is biologically plausible based on associative memory structures by creating cortical modules which can interact in a meaningful way.

An example work done by Fay [2] for modeling cerebral cortex as a network of associative memory modules which controls the robot to perform a specific action based on command sentences was achieved by building a system containing 30 modules. These larger modules containing thousands of neurons may possibly merge the gap between lower level neurophysiological observations and higher cognitive psychological level behavior at a transitional level where it is open to ideas and interpretations from both the fields. Edelman [3] in their work also discusses the possibilities of ideas and projects that deal with associative memory models.

Get a custom paper now from our expert writers.

Problem Statement: Sparse coding modeling (cerebellum) with the biologically realistic neural network model.

Image of Alex Wood
This essay was reviewed by
Alex Wood

Cite this Essay

Sparse Coding of Sensory Inputs. (2018, September 04). GradesFixer. Retrieved December 1, 2023, from
“Sparse Coding of Sensory Inputs.” GradesFixer, 04 Sept. 2018,
Sparse Coding of Sensory Inputs. [online]. Available at: <> [Accessed 1 Dec. 2023].
Sparse Coding of Sensory Inputs [Internet]. GradesFixer. 2018 Sept 04 [cited 2023 Dec 1]. Available from:
Keep in mind: This sample was shared by another student.
  • 450+ experts on 30 subjects ready to help
  • Custom essay delivered in as few as 3 hours
Write my essay

Still can’t find what you need?

Browse our vast selection of original essay samples, each expertly formatted and styled


Where do you want us to send this sample?

    By clicking “Continue”, you agree to our terms of service and privacy policy.


    Be careful. This essay is not unique

    This essay was donated by a student and is likely to have been used and submitted before

    Download this Sample

    Free samples may contain mistakes and not unique parts


    Sorry, we could not paraphrase this essay. Our professional writers can rewrite it and get you a unique paper.



    Please check your inbox.

    We can write you a custom essay that will follow your exact instructions and meet the deadlines. Let's fix your grades together!


    Get Your
    Personalized Essay in 3 Hours or Less!

    We can help you get a better grade and deliver your task on time!
    • Instructions Followed To The Letter
    • Deadlines Met At Every Stage
    • Unique And Plagiarism Free
    Order your paper now