cortex: thesis/cortex.org comparison

comparison thesis/cortex.org @ 551:d304b2ea7c58

some changes from winston.

author	Robert McIntyre <rlm@mit.edu>
date	Fri, 02 May 2014 13:40:47 -0400
parents	b1d8d9b4b569
children	20f64a70f8c5

comparison

equal deleted inserted replaced

-:b1d8d9b4b569
+:d304b2ea7c58
 [[./images/aurellem-gray.png]]
 * Empathy \& Embodiment: problem solving strategies
-By the end of this thesis, you will have a novel approach to
+By the time you have read this thesis, you will understand a novel
-representing an recognizing physical actions using embodiment and
+approach to representing and recognizing physical actions using
-empathy. You will also see one way to efficiently implement physical
+embodiment and empathy. You will also see one way to efficiently
-empathy for embodied creatures. Finally, you will become familiar
+implement physical empathy for embodied creatures. Finally, you will
-with =CORTEX=, a system for designing and simulating creatures with
+become familiar with =CORTEX=, a system for designing and simulating
-rich senses, which I have designed as a library that you can use in
+creatures with rich senses, which I have designed as a library that
-your own research. Note that I /do not/ process video directly --- I
+you can use in your own research. Note that I /do not/ process video
-start with knowledge of the positions of a creature's body parts and
+directly --- I start with knowledge of the positions of a creature's
-works from there.
+body parts and work from there.
 This is the core vision of my thesis: That one of the important ways
 in which we understand others is by imagining ourselves in their
 position and emphatically feeling experiences relative to our own
 bodies. By understanding events in terms of our own previous
 is happening in a video and being completely lost in a sea of
 incomprehensible color and movement.
 ** The problem: recognizing actions is hard!
-Examine the following image. What is happening? As you, and indeed
+Examine figure \ref{cat-drink}. What is happening? As you, and
-very young children, can easily determine, this is an image of
+indeed very young children, can easily determine, this is an image
-drinking.
+of drinking.
 #+caption: A cat drinking some water. Identifying this action is
 #+caption: beyond the capabilities of existing computer vision systems.
+#+name: cat-drink
 #+ATTR_LaTeX: :width 7cm
 [[./images/cat-drinking.jpg]]
 Nevertheless, it is beyond the state of the art for a computer
 vision program to describe what's happening in this image. Part of
 #+name: hidden-chair
 #+ATTR_LaTeX: :width 10cm
 [[./images/fat-person-sitting-at-desk.jpg]]
 Finally, how is it that you can easily tell the difference between
-how the girls /muscles/ are working in figure \ref{girl}?
+how the girl's /muscles/ are working in figure \ref{girl}?
 #+caption: The mysterious ``common sense'' appears here as you are able
 #+caption: to discern the difference in how the girl's arm muscles
-#+caption: are activated between the two images.
+#+caption: are activated between the two images. When you compare
+#+caption: these two images, do you feel something in your own arm
+#+caption: muscles?
 #+name: girl
 #+ATTR_LaTeX: :width 7cm
 [[./images/wall-push.png]]
 Each of these examples tells us something about what might be going
 their body, and are able to recognize that /feeling/ as drinking.
 So, the label of the action is not really in the pixels of the
 image, but is found clearly in a simulation / recollection inspired
 by those pixels. An imaginative system, having been trained on
 drinking and non-drinking examples and learning that the most
-important component of drinking is the feeling of water sliding
+important component of drinking is the feeling of water flowing
 down one's throat, would analyze a video of a cat drinking in the
 following manner:
 1. Create a physical model of the video by putting a ``fuzzy''
 model of its own body in place of the cat. Possibly also create
 a simulation of the stream of water.
-2. ``Play out'' this simulated scene and generate imagined sensory
+2. Play out this simulated scene and generate imagined sensory
 experience. This will include relevant muscle contractions, a
 close up view of the stream from the cat's perspective, and most
 importantly, the imagined feeling of water entering the mouth.
 The imagined sensory experience can come from a simulation of
 the event, but can also be pattern-matched from previous,
 ** =EMPATH= recognizes actions using empathy
 Exploring these ideas further demands a concrete implementation, so
 first, I built a system for constructing virtual creatures with
 physiologically plausible sensorimotor systems and detailed
-environments. The result is =CORTEX=, which is described in section
+environments. The result is =CORTEX=, which I describe in chapter
 \ref{sec-2}.
 Next, I wrote routines which enabled a simple worm-like creature to
 infer the actions of a second worm-like creature, using only its
 own prior sensorimotor experiences and knowledge of the second
 worm's joint positions. This program, =EMPATH=, is described in
-section \ref{sec-3}. It's main components are:
+chapter \ref{sec-3}. It's main components are:
 - Embodied Action Definitions :: Many otherwise complicated actions
 are easily described in the language of a full suite of
 body-centered, rich senses and experiences. For example,
-drinking is the feeling of water sliding down your throat, and
+drinking is the feeling of water flowing down your throat, and
 cooling your insides. It's often accompanied by bringing your
 hand close to your face, or bringing your face close to water.
 Sitting down is the feeling of bending your knees, activating
 your quadriceps, then feeling a surface with your bottom and
 relaxing your legs. These body-centered action descriptions
 language provided by =CORTEX= resulted in extremely economical
 recognition routines --- about 500 lines in all --- suggesting
 that such representations are very powerful, and often
 indispensable for the types of recognition tasks considered here.
-- Although for expediency's sake, I relied on direct knowledge of
+- For expediency's sake, I relied on direct knowledge of joint
-joint positions in this proof of concept, it would be
+positions in this proof of concept. However, I believe that the
-straightforward to extend =EMPATH= so that it (more
+structure of =EMPATH= and =CORTEX= will make future work to
-realistically) infers joint positions from its visual data.
+enable video analysis much easier than it would otherwise be.
 ** =EMPATH= is built on =CORTEX=, a creature builder.
 I built =CORTEX= to be a general AI research platform for doing
 experiments involving multiple rich senses and a wide variety and
 =CORTEX= is well suited as an environment for embodied AI research
 for three reasons:
 - You can design new creatures using Blender (\cite{blender}), a
-popular 3D modeling program. Each sense can be specified using
+popular, free 3D modeling program. Each sense can be specified
-special blender nodes with biologically inspired parameters. You
+using special blender nodes with biologically inspired
-need not write any code to create a creature, and can use a wide
+parameters. You need not write any code to create a creature, and
-library of pre-existing blender models as a base for your own
+can use a wide library of pre-existing blender models as a base
-creatures.
+for your own creatures.
 - =CORTEX= implements a wide variety of senses: touch,
 proprioception, vision, hearing, and muscle tension. Complicated
 senses like touch and vision involve multiple sensory elements
 embedded in a 2D surface. You have complete control over the
 distribution of these sensor elements through the use of simple
-png image files. =CORTEX= implements more comprehensive hearing
+image files. =CORTEX= implements more comprehensive hearing than
-than any other creature simulation system available.
+any other creature simulation system available.
 - =CORTEX= supports any number of creatures and any number of
 senses. Time in =CORTEX= dilates so that the simulated creatures
 always perceive a perfectly smooth flow of time, regardless of
 the actual computational load.
 =CORTEX= is built on top of =jMonkeyEngine3=
 (\cite{jmonkeyengine}), which is a video game engine designed to
 create cross-platform 3D desktop games. =CORTEX= is mainly written
-in clojure, a dialect of =LISP= that runs on the java virtual
+in clojure, a dialect of =LISP= that runs on the Java Virtual
-machine (JVM). The API for creating and simulating creatures and
+Machine (JVM). The API for creating and simulating creatures and
 senses is entirely expressed in clojure, though many senses are
 implemented at the layer of jMonkeyEngine or below. For example,
 for the sense of hearing I use a layer of clojure code on top of a
 layer of java JNI bindings that drive a layer of =C++= code which
 implements a modified version of =OpenAL= to support multiple
 - exploration of exotic senses and effectors that are not possible
 in the real world (such as telekinesis or a semantic sense)
 - imagination using subworlds
 During one test with =CORTEX=, I created 3,000 creatures each with
-their own independent senses and ran them all at only 1/80 real
+its own independent senses and ran them all at only 1/80 real time.
-time. In another test, I created a detailed model of my own hand,
+In another test, I created a detailed model of my own hand,
 equipped with a realistic distribution of touch (more sensitive at
 the fingertips), as well as eyes and ears, and it ran at around 1/4
 real time.
 #+BEGIN_LaTeX
 its own finger from the eye in its palm, and that it can feel its
 own thumb touching its palm.}
 \end{sidewaysfigure}
 #+END_LaTeX
-* Designing =CORTEX=
+* COMMENT Designing =CORTEX=
-In this section, I outline the design decisions that went into
+In this chapter, I outline the design decisions that went into
 making =CORTEX=, along with some details about its implementation.
 (A practical guide to getting started with =CORTEX=, which skips
 over the history and implementation details presented here, is
 provided in an appendix at the end of this thesis.)
 community and is now (in modified form) part of a system for
 capturing in-game video to a file.
 ** ...but hearing must be built from scratch
-At the end of this section I will have simulated ears that work the
+At the end of this chapter I will have simulated ears that work the
-same way as the simulated eyes in the last section. I will be able to
+same way as the simulated eyes in the last chapter. I will be able to
 place any number of ear-nodes in a blender file, and they will bind to
 the closest physical object and follow it as it moves around. Each ear
 will provide access to the sound data it picks up between every frame.
 Hearing is one of the more difficult senses to simulate, because there
 access the rendered sound data.
 =CORTEX='s hearing is unique because it does not have any
 limitations compared to other simulation environments. As far as I
 know, there is no other system that supports multiple listeners,
-and the sound demo at the end of this section is the first time
+and the sound demo at the end of this chapter is the first time
 it's been done in a video game environment.
 *** Brief Description of jMonkeyEngine's Sound System
 jMonkeyEngine's sound system works as follows:
 My simulated proprioception calculates the relative angles of each
 joint from the rest position defined in the blender file. This
 simulates the muscle-spindles and joint capsules. I will deal with
 Golgi tendon organs, which calculate muscle strain, in the next
-section.
+chapter.
 *** Helper functions
 =absolute-angle= calculates the angle between two vectors,
 relative to a third axis vector. This angle is the number of
 a rotational force dependent on it's orientation to the object in
 the blender file. The function returned by =movement-kernel= is
 also a sense function: it returns the percent of the total muscle
 strength that is currently being employed. This is analogous to
 muscle tension in humans and completes the sense of proprioception
-begun in the last section.
+begun in the last chapter.
 ** =CORTEX= brings complex creatures to life!
 The ultimate test of =CORTEX= is to create a creature with the full
 gamut of senses and put it though its paces.
 hard control problems without worrying about physics or
 senses.
 \newpage
-* =EMPATH=: action recognition in a simulated worm
+* COMMENT =EMPATH=: action recognition in a simulated worm
 Here I develop a computational model of empathy, using =CORTEX= as a
 base. Empathy in this context is the ability to observe another
 creature and infer what sorts of sensations that creature is
 feeling. My empathy algorithm involves multiple phases. First is
 see no errors in action identification compared to my own judgment
 of what the worm is doing.
 ** Digression: Learning touch sensor layout through free play
-In the previous section I showed how to compute actions in terms of
+In the previous chapter I showed how to compute actions in terms of
 body-centered predicates, but some of those predicates relied on
 the average touch activation of pre-defined regions of the worm's
 skin. What if, instead of receiving touch pre-grouped into the six
 faces of each worm segment, the true topology of the worm's skin
 was unknown? This is more similar to how a nerve fiber bundle might
 together on the skin, the process of taking a complicated surface
 and forcing it into essentially a circle requires that some regions
 of skin that are close together in the animal end up far apart in
 the nerve bundle.
-In this section I show how to automatically learn the skin-topology of
+In this chapter I show how to automatically learn the skin-topology of
 a worm segment by free exploration. As the worm rolls around on the
 floor, large sections of its surface get activated. If the worm has
 stopped moving, then whatever region of skin that is touching the
 floor is probably an important region, and should be recorded.
 #+BEGIN_LaTeX
 \clearpage
 #+END_LaTeX
-* Contributions
+* COMMENT Contributions
 The big idea behind this thesis is a new way to represent and
 recognize physical actions, which I call /empathic representation/.
 Actions are represented as predicates which have access to the
 totality of a creature's sensory abilities. To recognize the
 #+BEGIN_LaTeX
 \clearpage
 \appendix
 #+END_LaTeX
-* Appendix: =CORTEX= User Guide
+* COMMENT Appendix: =CORTEX= User Guide
 Those who write a thesis should endeavor to make their code not only
 accessible, but actually usable, as a way to pay back the community
 that made the thesis possible in the first place. This thesis would
 not be possible without Free Software such as jMonkeyEngine3,

Mercurial > cortex

comparison thesis/cortex.org @ 551:d304b2ea7c58