What to make of: distributed representations summation of

What to make of:  distributed representations  summation of

What to make of: distributed representations summation of inputs Hebbian plasticity ? Competitive nets {r1,, ri,, rN} ri f (jwijrj) wij ri (rj - ) Pattern associators

Autoassociators Competitive Networks discover structures in input space may remove redundancy may orthogonalize may categorize may sparsify representations can separate out even linear combinations can easily be generalized to selforganizing topographic maps m

a oc s u p ..use multiple charts to code environments... p p i H

context position identity s p a m l ..use thea sheet c

to t code position... i r o C x c Discrete attractors, with units m r arranged in

a cortical network o f i r i P Object 1 in position 1 Object 2 in position 1 Object 1 in position 2 Leah Krubitzer, Neuron, 2007

The dorsal cortex takes over hedgehog cat monkey Isocortex is laminated Arealization and Memory in the Cortex Main perspectives:

a) Hierarchical monkey b) Modular c) Content-based The hierarchical perspective ET Rolls, Proc Roy Soc 1992 The hierarchical perspective The Elizabeth Gardner approach ri = g[jwijHEBBrj-]+

..instead of neural activity wijHEBB rirj (as in the Hopfield model).. ..do thermodynamics over connection weights, + r = g

w r - i.e. consider whether i j ij j among all their possible values, there are which satisfy [ ]

The hierarchical perspective The Elizabeth Gardner approach Backpropagation and E-M algorithms Network activation Forward Step: rr Error propagation Backward Step: rw Expectation sampling the world Maximization of the match between the world and our internal model of the world The modular perspective The Braitenberg model

N pyramidal cells N compartments N cells each A pical synapses B asal synapses granulating the dorsal wall, leads to the mammalian Layer IV granules are now (excitatory) interneurons isocortex the brand new `neocortex has

laminated, i.e. inserted a granular layer IV in between two pyramidal cells layers. what does this other granulatio n buy us? Isocortical lamination

emerges together with fine topographic mapping does not apply to the non topographic olfactory system is underdeveloped in caetaceans It might be a computational solution to the need to relay precise information about both where and what sensory stimuli are. the model src recurrent collaterals

patch of cortex sff input station feedforward connections input activity R spatial focus

detailed pattern The activation of units in the previous station is the product of a spatial focus, say, a Gaussian of radius R (which presumably would be picked up by optical imaging, or by multi-unit recording) and a detailed unit-by-unit pattern of activity (which would require single unit recording to be revealed). p patterns of activity (e.g. 2-12) are established at the beginning, drawn at random from a given distribution, and used repeatedly in one simulation. The activation of units in the cortical patch is compared with the activations resulting from the application of each input pattern at each spatial focus, to decode the patternI and focus x of the current activation. This allows measuring as well as

I pos p( x real p( xreal , xdecoded ) p ( xreal ) p ( xdecoded ) , xdecoded ) log 2 p( , ) I p ( ,

) log p( ) p ( ) ident both population measures, reflecting activity in the whole patch real decoded 2 real real

decoded decoded Both recurrent and feedforward weights are modified according to a simple Hebbian associative rule, over the course of several training epochs. Each training epoch involves presenting, in random order, each input pattern at each activation focus. The map is thus pre-wired at a coarse, statistical level, and self-organized at a finer scale. After a training epoch, noisy versions, again of each pattern at each activation focus, are presented for testing, with no weight change. The full information about position and identity cannot be decoded from the activation in the patch, because the activation in the input is noisy

(in practice, e.g. 40% of the input units follow the prescribed pattern, and 60% are randomly activated with the same distribution) If R << Src, it is rather intuitive to predict how much information can be relayed by feedforward projections of spread Sff: I pos log(1/ S I S ident ff ff

) Iident is small initially grows with learning no difference between layers Results for p=4 Ipos is less affected by learning decreases with more diffuse feedforward connections

again, no difference between layers These data, plotted as Ipos vs. Iident, demonstrate the what/where conflict as a boundary using more patterns merely shifts the same boundary

upwards Differentiating a granular layer (IV) in which units receive focused FF connections, also more restricted RC connections, and follow a specific dynamics may nail down the focus of activation within the cortical map (preserving detailed positional information) without interfering with the retrieval of the identity of the specific activation pattern (achieved mainly by the collaterals of the pyramidal layers) the model src

recurrent collaterals patch of cortex sff input station feedforward connections input activity

R spatial focus detailed pattern Indeed it happens! Laminated cortex can relay more combined what and where information than if it were not laminated The advantage is somewhat more evident for larger p

it is small, but should scale up in a network of realistic size The granular layer may nail down the focus of activation within the cortical map (preserving detailed positional information) without interfering with attractor-mediated retrieval of the identity of the specific activation pattern (achieved mainly by the collaterals of the pyramidal layers) A differentiation between supra- and infragranular layers may be usefully coupled to

their different extrinsic connectivity, if: the supragranular layers preserve both positional and identity information, and trasmit it onward for further analysis the infragranular layers relay backwards and downwards identity information freshly squeezed from the attractors, without bothering to replicate positional information Lamination+directio nal connectivity make each layer convey a

better mix of information, beyond the capability of any unlaminated patch, whatever its Sff they also slow down learning, though, so the advantage would be greater if more learning epochs had been allowed (here they are set to 3) A functional hypothesis

A common mode of operation of the primordial sensory neocortex of mammals may have been autoassociative attractor dynamics. Attractors may be formed by self-organizing weight changes on FF and RC connections, and may dominate the dynamics of both SG and IG layers, although the former can be kept in tighter positional register by layer IV. Thanks to Hamish Meffin, with whom I discussed such ideas, with divergent conclusions (see his Ph.D. Thesis, U. of Sidney) 2 suggestions Understanding specific mammalian mechanisms of information representation and retrieval may require

quantitative (information theoretical) analyses at the level of populations of individual neurones Only notions of sufficient abstraction and generality as to apply to each sensory cortex can help explain the appearance, in evolution, of this universal neocortical microchip.

Recently Viewed Presentations

  • The Mayan Codices Bishop Diego de Landa set

    The Mayan Codices Bishop Diego de Landa set

    The Ancient Maya wrote in a beautiful, highly artistic script. They carved with it, painted with it, and wrote it in in their bark-paper books. But, Bishop Diego de Landa did not share my view of their artistry. To him,...
  • Absolutism in Eastern Europe

    Absolutism in Eastern Europe

    FW I (King of P) FW II Elector of Brandenburg continued the work of his grandfather to strengthen Prussia and pull Brandenburg out of HRE. Created strong army- "soldier king" improved farming and ag production, improved the economy and left...
  • Inverse Relations and Inverse Functions

    Inverse Relations and Inverse Functions

    The HLT!!! The Horizontal Line Test The inverse of a relation is a function if and only if each horizontal line intersects the graph of the original function in at most one point. Fails the HLT miserably!!! So, its inverse...
  • Course Reminders Group Projects Required to attend all

    Course Reminders Group Projects Required to attend all

    Week 8: group project workday. Updated total course grade will be posted again with the grade for CT 2. Contact Dr. Cook at any time if you would like to see a breakdown of your grade. CT 2 due Tuesday,...
  • Default Cover Slide Subtitle | Date Section Slide

    Default Cover Slide Subtitle | Date Section Slide

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Targeted organic growth rate: 2-3x Global GDP Lorem ipsum dolor sit amet46%
  • Digital Librarians - University of Kent

    Digital Librarians - University of Kent

    Why become more digital? "Drawing on a survey of 1658 undergraduate students, the paper identifies 11 distinct digital 'benefits' - ranging from flexibilities of time and place, ease of organizing and managing study tasks through to the ability to replay...
  • Quiz, Quiz, Trade Each student is given a

    Quiz, Quiz, Trade Each student is given a

    Question: Prior to the adoption of English common law in Ireland, a native legalsystem, known as _____, had applied throughout the country. Answer: The Brehon law. Question: Mention one reason for why the Laois - Offaly plantation happened. Answer: To...
  • Point of View - Salem State University

    Point of View - Salem State University

    Summary The basic unit of drama is the scene The writer makes choices One of the choice is Point of View, a quality unique to literature The manipulation of POV allows for varying aesthetic effects Final Writing Tips Try writing...