# HG changeset patch # User Robert McIntyre # Date 1398060803 14400 # Node ID 25f23cfd56ceae2e15108f8621aeb252a24221ca # Parent 8e52a280282194a128f9928431e39632335e4a01 alter pics, enhance text. diff -r 8e52a2802821 -r 25f23cfd56ce images/finger-1.png Binary file images/finger-1.png has changed diff -r 8e52a2802821 -r 25f23cfd56ce thesis/cortex.org --- a/thesis/cortex.org Sun Apr 20 21:46:46 2014 -0400 +++ b/thesis/cortex.org Mon Apr 21 02:13:23 2014 -0400 @@ -44,11 +44,14 @@ * Empathy \& Embodiment: problem solving strategies By the end of this thesis, you will have seen a novel approach to - interpreting video using embodiment and empathy. You will have also - seen one way to efficiently implement empathy for embodied + interpreting video using embodiment and empathy. You will also see + one way to efficiently implement physical empathy for embodied creatures. Finally, you will become familiar with =CORTEX=, a system - for designing and simulating creatures with rich senses, which you - may choose to use in your own research. + for designing and simulating creatures with rich senses, which I + have designed as a library that you can use in your own research. + Note that I /do not/ process video directly --- I start with + knowledge of the positions of a creature's body parts and works from + there. This is the core vision of my thesis: That one of the important ways in which we understand others is by imagining ourselves in their @@ -60,7 +63,7 @@ is happening in a video and being completely lost in a sea of incomprehensible color and movement. -** The problem: recognizing actions in video is hard! +** The problem: recognizing actions is hard! Examine the following image. What is happening? As you, and indeed very young children, can easily determine, this is an image of @@ -84,8 +87,8 @@ example, what processes might enable you to see the chair in figure \ref{hidden-chair}? - #+caption: The chair in this image is quite obvious to humans, but I - #+caption: doubt that any modern computer vision program can find it. + #+caption: The chair in this image is quite obvious to humans, but + #+caption: it can't be found by any modern computer vision program. #+name: hidden-chair #+ATTR_LaTeX: :width 10cm [[./images/fat-person-sitting-at-desk.jpg]] @@ -480,7 +483,7 @@ real world instead of a simulation is the matter of time. Instead of simulated time you get the constant and unstoppable flow of real time. This severely limits the sorts of software you can use - to program the AI because all sense inputs must be handled in real + to program an AI, because all sense inputs must be handled in real time. Complicated ideas may have to be implemented in hardware or may simply be impossible given the current speed of our processors. Contrast this with a simulation, in which the flow of @@ -550,10 +553,10 @@ of the retina. In each case, we can describe the sense with a surface and a distribution of sensors along that surface. - The neat idea is that every human sense can be effectively - described in terms of a surface containing embedded sensors. If the - sense had any more dimensions, then there wouldn't be enough room - in the spinal chord to transmit the information! + In fact, almost every human sense can be effectively described in + terms of a surface containing embedded sensors. If the sense had + any more dimensions, then there wouldn't be enough room in the + spinal chord to transmit the information! Therefore, =CORTEX= must support the ability to create objects and then be able to ``paint'' points along their surfaces to describe @@ -2378,14 +2381,14 @@ #+end_listing - =movement-kernel= creates a function that will move the nearest - physical object to the muscle node. The muscle exerts a rotational - force dependent on it's orientation to the object in the blender - file. The function returned by =movement-kernel= is also a sense - function: it returns the percent of the total muscle strength that - is currently being employed. This is analogous to muscle tension - in humans and completes the sense of proprioception begun in the - last section. + =movement-kernel= creates a function that controlls the movement + of the nearest physical node to the muscle node. The muscle exerts + a rotational force dependent on it's orientation to the object in + the blender file. The function returned by =movement-kernel= is + also a sense function: it returns the percent of the total muscle + strength that is currently being employed. This is analogous to + muscle tension in humans and completes the sense of proprioception + begun in the last section. ** =CORTEX= brings complex creatures to life! @@ -2491,6 +2494,8 @@ hard control problems without worrying about physics or senses. +\newpage + * =EMPATH=: action recognition in a simulated worm Here I develop a computational model of empathy, using =CORTEX= as a @@ -2502,8 +2507,8 @@ creature's sensory state space, which I call \Phi-space. Using \Phi-space, I construct an efficient function which takes the limited data that comes from observing another creature and enriches - it full compliment of imagined sensory data. I can then use the - imagined sensory data to recognize what the observed creature is + it with a full compliment of imagined sensory data. I can then use + the imagined sensory data to recognize what the observed creature is doing and feeling, using straightforward embodied action predicates. This is all demonstrated with using a simple worm-like creature, and recognizing worm-actions based on limited data. @@ -2555,9 +2560,9 @@ Embodied representations using multiple senses such as touch, proprioception, and muscle tension turns out be be exceedingly - efficient at describing body-centered actions. It is the ``right - language for the job''. For example, it takes only around 5 lines - of LISP code to describe the action of ``curling'' using embodied + efficient at describing body-centered actions. It is the right + language for the job. For example, it takes only around 5 lines of + LISP code to describe the action of curling using embodied primitives. It takes about 10 lines to describe the seemingly complicated action of wiggling. @@ -2566,14 +2571,16 @@ whether the worm is doing the action they describe. =curled?= relies on proprioception, =resting?= relies on touch, =wiggling?= relies on a Fourier analysis of muscle contraction, and - =grand-circle?= relies on touch and reuses =curled?= as a guard. + =grand-circle?= relies on touch and reuses =curled?= in its + definition, showing how embodied predicates can be composed. #+caption: Program for detecting whether the worm is curled. This is the #+caption: simplest action predicate, because it only uses the last frame #+caption: of sensory experience, and only uses proprioceptive data. Even #+caption: this simple predicate, however, is automatically frame - #+caption: independent and ignores vermopomorphic differences such as - #+caption: worm textures and colors. + #+caption: independent and ignores vermopomorphic \footnote{Like + #+caption: \emph{anthropomorphic}, except for worms instead of humans.} + #+caption: differences such as worm textures and colors. #+name: curled #+begin_listing clojure #+begin_src clojure @@ -2735,7 +2742,7 @@ The trick now is to make the action predicates work even when the sensory data on which they depend is absent. If I can do that, then - I will have gained much, + I will have gained much. ** \Phi-space describes the worm's experiences diff -r 8e52a2802821 -r 25f23cfd56ce thesis/images/empty-sense-nodes.png Binary file thesis/images/empty-sense-nodes.png has changed diff -r 8e52a2802821 -r 25f23cfd56ce thesis/images/finger-UV.png Binary file thesis/images/finger-UV.png has changed diff -r 8e52a2802821 -r 25f23cfd56ce thesis/images/hand-screenshot1.png Binary file thesis/images/hand-screenshot1.png has changed diff -r 8e52a2802821 -r 25f23cfd56ce thesis/rlm-cortex-meng.tex --- a/thesis/rlm-cortex-meng.tex Sun Apr 20 21:46:46 2014 -0400 +++ b/thesis/rlm-cortex-meng.tex Mon Apr 21 02:13:23 2014 -0400 @@ -62,6 +62,9 @@ % Configure minted source code listings. \usemintedstyle{default} \newminted{clojure}{fontsize=\footnotesize} +\newminted{java}{fontsize=\footnotesize} +\newminted{c}{fontsize=\footnotesize} + % Allow colored source code listing to break across pages. \newenvironment{anchoredListing}{\captionsetup{type=listing}}{}