cortex: thesis/cortex.org comparison

comparison thesis/cortex.org @ 548:0b891e0dd809

version 0.2 of thesis complete.

author	Robert McIntyre <rlm@mit.edu>
date	Thu, 01 May 2014 23:41:41 -0400
parents	5d89879fc894
children	c14545acdfba

comparison

equal deleted inserted replaced

-:5d89879fc894
+:0b891e0dd809
 in proportion to the amount of processing each frame. From the
 perspective of the creatures inside the simulation, time always
 appears to flow at a constant rate, regardless of how complicated
 the environment becomes or how many creatures are in the
 simulation. The cost is that =CORTEX= can sometimes run slower than
-real time. Time dialation works both ways, however --- simulations
+real time. Time dilation works both ways, however --- simulations
 of very simple creatures in =CORTEX= generally run at 40x real-time
 on my machine!
 ** All sense organs are two-dimensional surfaces
 Therefore, =CORTEX= must support the ability to create objects and
 then be able to ``paint'' points along their surfaces to describe
 each sense.
 Fortunately this idea is already a well known computer graphics
-technique called /UV-mapping/. In UV-maping, the three-dimensional
+technique called /UV-mapping/. In UV-mapping, the three-dimensional
 surface of a model is cut and smooshed until it fits on a
 two-dimensional image. You paint whatever you want on that image,
 and when the three-dimensional shape is rendered in a game the
 smooshing and cutting is reversed and the image appears on the
 three-dimensional object.
 of muscle contractions to transform the worm's body along a
 specific path through \Phi-space.
 The worm's total life experience is a long looping path through
 \Phi-space. I will now introduce simple way of taking that
-experiece path and building a function that can infer complete
+experience path and building a function that can infer complete
 sensory experience given only a stream of proprioceptive data. This
 /empathy/ function will provide a bridge to use the body centered
 action predicates on video-like streams of information.
 ** Empathy is the process of building paths in \Phi-space
 =longest-thread= takes time proportional to the average number of
 entries in a proprioceptive bin, because for each element in the
 starting bin it performs a series of set lookups in the preceding
 bins. If the total history is limited, then this takes time
-proprotional to a only a constant multiple of the number of entries
+proportional to a only a constant multiple of the number of entries
 in the starting bin. This analysis also applies, even if the action
 requires multiple longest chains -- it's still the average number
 of entries in a proprioceptive bin times the desired chain length.
 Because =longest-thread= is so efficient and simple, I can
 interpret worm-actions in real time.
 worm's \Phi-space is generated from a simple motor script. Then the
 worm is re-created in an environment almost exactly identical to
 the testing environment for the action-predicates, with one major
 difference : the only sensory information available to the system
 is proprioception. From just the proprioception data and
-\Phi-space, =longest-thread= synthesises a complete record the last
+\Phi-space, =longest-thread= synthesizes a complete record the last
 300 sensory experiences of the worm. These synthesized experiences
 are fed directly into the action predicates =grand-circle?=,
 =curled?=, =wiggling?=, and =resting?= from before and their output
 is printed to the screen at each frame.
 #+name: worm-roll
 #+ATTR_LaTeX: :width 12cm
 [[./images/worm-roll.png]]
 #+caption: After completing its adventures, the worm now knows
-#+caption: how its touch sensors are arranged along its skin. These
+#+caption: how its touch sensors are arranged along its skin. Each of these six rectangles are touch sensory patterns that were
-#+caption: are the regions that were deemed important by
+#+caption: deemed important by
 #+caption: =learn-touch-regions=. Each white square in the rectangles
 #+caption: above is a cluster of ``related" touch nodes as determined
-#+caption: by the system. Since each square in the ``cross" corresponds
+#+caption: by the system. The worm has correctly discovered that it has six faces, and has partitioned its sensory map into these six faces.
-#+caption: to a face, the worm has correctly discovered that it has
-#+caption: six faces.
 #+name: worm-touch-map
 #+ATTR_LaTeX: :width 12cm
 [[./images/touch-learn.png]]
 While simple, =learn-touch-regions= exploits regularities in both
 deduce that the worm has six sides. Note that =learn-touch-regions=
 would work just as well even if the worm's touch sense data were
 completely scrambled. The cross shape is just for convenience. This
 example justifies the use of pre-defined touch regions in =EMPATH=.
+** Recognizing an object using embodied representation
+At the beginning of the thesis, I suggested that we might recognize
+the chair in Figure \ref{hidden-chair} by imagining ourselves in
+the position of the man and realizing that he must be sitting on
+something in order to maintain that position. Here, I present a
+brief elaboration on how to this might be done.
+First, I need the feeling of leaning or resting /on/ some other
+object that is not the floor. This feeling is easy to describe
+using an embodied representation.
+#+caption: Program describing the sense of leaning or resting on something.
+#+caption: This involves a relaxed posture, the feeling of touching something,
+#+caption: and a period of stability where the worm does not move.
+#+name: draped
+#+begin_listing clojure
+#+begin_src clojure
+(defn draped?
+"Is the worm:
+-- not flat (the floor is not a 'chair')
+-- supported (not using its muscles to hold its position)
+-- stable (not changing its position)
+-- touching something (must register contact)"
+[experiences]
+(let [b2-hash (bin 2)
+touch (:touch (peek experiences))
+total-contact
+(reduce
++
+(map #(contact all-touch-coordinates %)
+(rest touch)))]
+(println total-contact)
+(and (not (resting? experiences))
+(every?
+zero?
+(-> experiences
+(vector:last-n 25)
+(#(map :muscle %))
+(flatten)))
+(-> experiences
+(vector:last-n 20)
+(#(map (comp b2-hash flatten :proprioception) %))
+(set)
+(count) (= 1))
+(< 0.03 total-contact))))
+#+end_src
+#+end_listing
+#+caption: The =draped?= predicate detects the presence of the
+#+caption: cube whenever the worm interacts with it. The details of the
+#+caption: cube are irrelevant; only the way it influences the worm's
+#+caption: body matters.
+#+name: draped-video
+#+ATTR_LaTeX: :width 13cm
+[[./images/draped.png]]
+Though this is a simple example, using the =draped?= predicate to
+detect the cube has interesting advantages. The =draped?= predicate
+describes the cube not in terms of properties that the cube has,
+but instead in terms of how the worm interacts with it physically.
+This means that the cube can still be detected even if it is not
+visible, as long as its influence on the worm's body is visible.
+This system will also see the virtual cube created by a
+``mimeworm", which uses its muscles in a very controlled way to
+mimic the appearance of leaning on a cube. The system will
+anticipate that there is an actual invisible cube that provides
+support!
+#+caption: Can you see the thing that this person is leaning on?
+#+caption: What properties does it have, other than how it makes the man's
+#+caption: elbow and shoulder feel? I wonder if people who can actually
+#+caption: maintain this pose easily still see the support?
+#+name: mime
+#+ATTR_LaTeX: :width 6cm
+[[./images/pablo-the-mime.png]]
+This makes me wonder about the psychology of actual mimes. Suppose
+for a moment that people have something analogous to \Phi-space and
+that one of the ways that they find objects in a scene is by their
+relation to other people's bodies. Suppose that a person watches a
+person miming an invisible wall. For a person with no experience
+with miming, their \Phi-space will only have entries that describe
+the scene with the sensation of their hands touching a wall. This
+sensation of touch will create a strong impression of a wall, even
+though the wall would have to be invisible. A person with
+experience in miming however, will have entries in their \Phi-space
+that describe the wall-miming position without a sense of touch. It
+will not seem to such as person that an invisible wall is present,
+but merely that the mime is holding out their hands in a special
+way. Thus, the theory that humans use something like \Phi-space
+weakly predicts that learning how to mime should break the power of
+miming illusions. Most optical illusions still work no matter how
+much you know about them, so this proposal would be quite
+interesting to test, as it predicts a non-standard result!
+#+BEGIN_LaTeX
+\clearpage
+#+END_LaTeX
 * Contributions
+The big idea behind this thesis is a new way to represent and
+recognize physical actions, which I call /empathic representation/.
+Actions are represented as predicates which have access to the
+totality of a creature's sensory abilities. To recognize the
+physical actions of another creature similar to yourself, you
+imagine what they would feel by examining the position of their body
+and relating it to your own previous experience.
-The big idea behind this thesis is a new way to represent and
+Empathic representation of physical actions is robust and general.
-recognize physical actions -- empathic representation. Actions are
+Because the representation is body-centered, it avoids baking in a
-represented as predicates which have available the totality of a
+particular viewpoint like you might get from learning from example
-creature's sensory abilities. To recognize the physical actions of
+videos. Because empathic representation relies on all of a
-another creature similar to yourself, you imagine what they would
-feel by examining the position of their body and relating it to your
-own previous experience.
-Empathic description of physical actions is very robust and general.
-Because the representation is body-centered, it avoids the fragility
-of learning from example videos. Because it relies on all of a
 creature's senses, it can describe exactly what an action /feels
 like/ without getting caught up in irrelevant details such as visual
 appearance. I think it is important that a correct description of
-jumping (for example) should not waste even a single bit on the
+jumping (for example) should not include irrelevant details such as
-color of a person's clothes or skin; empathic representation can
+the color of a person's clothes or skin; empathic representation can
-avoid this waste by describing jumping in terms of touch, muscle
+get right to the heart of what jumping is by describing it in terms
-contractions, and the brief feeling of weightlessness. Empathic
+of touch, muscle contractions, and a brief feeling of
-representation is very low-level in that it describes actions using
+weightlessness. Empathic representation is very low-level in that it
-concrete sensory data with little abstraction, but it has the
+describes actions using concrete sensory data with little
-generality of much more abstract representations!
+abstraction, but it has the generality of much more abstract
+representations!
 Another important contribution of this thesis is the development of
 the =CORTEX= system, a complete environment for creating simulated
 creatures. You have seen how to implement five senses: touch,
 proprioception, hearing, vision, and muscle tension. You have seen
 how to create new creatures using blender, a 3D modeling tool.
-I hope that =CORTEX= will be useful in further research projects. To
-this end I have included the full source to =CORTEX= along with a
-large suite of tests and examples. I have also created a user guide
-for =CORTEX= which is included in an appendix to this thesis.
 As a minor digression, you also saw how I used =CORTEX= to enable a
 tiny worm to discover the topology of its skin simply by rolling on
-the ground.
+the ground.  You also saw how to detect objects using only embodied
+predicates.
-In conclusion, the main contributions of this thesis are:
+In conclusion, for this thesis I:
-- =CORTEX=, a comprehensive platform for embodied AI experiments.
-=CORTEX= supports many features lacking in other systems, such
+- Developed the idea of embodied representation, which describes
-proper simulation of hearing. It is easy to create new =CORTEX=
+actions that a creature can do in terms of first-person sensory
-creatures using Blender, a free 3D modeling program.
+data.
-- =EMPATH=, which uses =CORTEX= to identify the actions of a
+- Developed a method of empathic action recognition which uses
-worm-like creature using a computational model of empathy. This
+previous embodied experience and embodied representation of
-empathic representation of actions is an important new kind of
+actions to greatly constrain the possible interpretations of an
-representation for physical actions.
+action.
+- Created =EMPATH=, a program which uses empathic action
+recognition to recognize physical actions in a simple model
+involving segmented worm-like creatures.
+- Created =CORTEX=, a comprehensive platform for embodied AI
+experiments. It is the base on which =EMPATH= is built.
 #+BEGIN_LaTeX
-\newpage
+\clearpage
 \appendix
 #+END_LaTeX
 * Appendix: =CORTEX= User Guide

Mercurial > cortex

comparison thesis/cortex.org @ 548:0b891e0dd809