Mercurial > cortex
diff thesis/cortex.org @ 525:25f23cfd56ce
alter pics, enhance text.
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Mon, 21 Apr 2014 02:13:23 -0400 |
parents | 1e51263afdc0 |
children | 96c189d4d15e |
line wrap: on
line diff
1.1 --- a/thesis/cortex.org Sun Apr 20 21:46:46 2014 -0400 1.2 +++ b/thesis/cortex.org Mon Apr 21 02:13:23 2014 -0400 1.3 @@ -44,11 +44,14 @@ 1.4 * Empathy \& Embodiment: problem solving strategies 1.5 1.6 By the end of this thesis, you will have seen a novel approach to 1.7 - interpreting video using embodiment and empathy. You will have also 1.8 - seen one way to efficiently implement empathy for embodied 1.9 + interpreting video using embodiment and empathy. You will also see 1.10 + one way to efficiently implement physical empathy for embodied 1.11 creatures. Finally, you will become familiar with =CORTEX=, a system 1.12 - for designing and simulating creatures with rich senses, which you 1.13 - may choose to use in your own research. 1.14 + for designing and simulating creatures with rich senses, which I 1.15 + have designed as a library that you can use in your own research. 1.16 + Note that I /do not/ process video directly --- I start with 1.17 + knowledge of the positions of a creature's body parts and works from 1.18 + there. 1.19 1.20 This is the core vision of my thesis: That one of the important ways 1.21 in which we understand others is by imagining ourselves in their 1.22 @@ -60,7 +63,7 @@ 1.23 is happening in a video and being completely lost in a sea of 1.24 incomprehensible color and movement. 1.25 1.26 -** The problem: recognizing actions in video is hard! 1.27 +** The problem: recognizing actions is hard! 1.28 1.29 Examine the following image. What is happening? As you, and indeed 1.30 very young children, can easily determine, this is an image of 1.31 @@ -84,8 +87,8 @@ 1.32 example, what processes might enable you to see the chair in figure 1.33 \ref{hidden-chair}? 1.34 1.35 - #+caption: The chair in this image is quite obvious to humans, but I 1.36 - #+caption: doubt that any modern computer vision program can find it. 1.37 + #+caption: The chair in this image is quite obvious to humans, but 1.38 + #+caption: it can't be found by any modern computer vision program. 1.39 #+name: hidden-chair 1.40 #+ATTR_LaTeX: :width 10cm 1.41 [[./images/fat-person-sitting-at-desk.jpg]] 1.42 @@ -480,7 +483,7 @@ 1.43 real world instead of a simulation is the matter of time. Instead 1.44 of simulated time you get the constant and unstoppable flow of 1.45 real time. This severely limits the sorts of software you can use 1.46 - to program the AI because all sense inputs must be handled in real 1.47 + to program an AI, because all sense inputs must be handled in real 1.48 time. Complicated ideas may have to be implemented in hardware or 1.49 may simply be impossible given the current speed of our 1.50 processors. Contrast this with a simulation, in which the flow of 1.51 @@ -550,10 +553,10 @@ 1.52 of the retina. In each case, we can describe the sense with a 1.53 surface and a distribution of sensors along that surface. 1.54 1.55 - The neat idea is that every human sense can be effectively 1.56 - described in terms of a surface containing embedded sensors. If the 1.57 - sense had any more dimensions, then there wouldn't be enough room 1.58 - in the spinal chord to transmit the information! 1.59 + In fact, almost every human sense can be effectively described in 1.60 + terms of a surface containing embedded sensors. If the sense had 1.61 + any more dimensions, then there wouldn't be enough room in the 1.62 + spinal chord to transmit the information! 1.63 1.64 Therefore, =CORTEX= must support the ability to create objects and 1.65 then be able to ``paint'' points along their surfaces to describe 1.66 @@ -2378,14 +2381,14 @@ 1.67 #+end_listing 1.68 1.69 1.70 - =movement-kernel= creates a function that will move the nearest 1.71 - physical object to the muscle node. The muscle exerts a rotational 1.72 - force dependent on it's orientation to the object in the blender 1.73 - file. The function returned by =movement-kernel= is also a sense 1.74 - function: it returns the percent of the total muscle strength that 1.75 - is currently being employed. This is analogous to muscle tension 1.76 - in humans and completes the sense of proprioception begun in the 1.77 - last section. 1.78 + =movement-kernel= creates a function that controlls the movement 1.79 + of the nearest physical node to the muscle node. The muscle exerts 1.80 + a rotational force dependent on it's orientation to the object in 1.81 + the blender file. The function returned by =movement-kernel= is 1.82 + also a sense function: it returns the percent of the total muscle 1.83 + strength that is currently being employed. This is analogous to 1.84 + muscle tension in humans and completes the sense of proprioception 1.85 + begun in the last section. 1.86 1.87 ** =CORTEX= brings complex creatures to life! 1.88 1.89 @@ -2491,6 +2494,8 @@ 1.90 hard control problems without worrying about physics or 1.91 senses. 1.92 1.93 +\newpage 1.94 + 1.95 * =EMPATH=: action recognition in a simulated worm 1.96 1.97 Here I develop a computational model of empathy, using =CORTEX= as a 1.98 @@ -2502,8 +2507,8 @@ 1.99 creature's sensory state space, which I call \Phi-space. Using 1.100 \Phi-space, I construct an efficient function which takes the 1.101 limited data that comes from observing another creature and enriches 1.102 - it full compliment of imagined sensory data. I can then use the 1.103 - imagined sensory data to recognize what the observed creature is 1.104 + it with a full compliment of imagined sensory data. I can then use 1.105 + the imagined sensory data to recognize what the observed creature is 1.106 doing and feeling, using straightforward embodied action predicates. 1.107 This is all demonstrated with using a simple worm-like creature, and 1.108 recognizing worm-actions based on limited data. 1.109 @@ -2555,9 +2560,9 @@ 1.110 1.111 Embodied representations using multiple senses such as touch, 1.112 proprioception, and muscle tension turns out be be exceedingly 1.113 - efficient at describing body-centered actions. It is the ``right 1.114 - language for the job''. For example, it takes only around 5 lines 1.115 - of LISP code to describe the action of ``curling'' using embodied 1.116 + efficient at describing body-centered actions. It is the right 1.117 + language for the job. For example, it takes only around 5 lines of 1.118 + LISP code to describe the action of curling using embodied 1.119 primitives. It takes about 10 lines to describe the seemingly 1.120 complicated action of wiggling. 1.121 1.122 @@ -2566,14 +2571,16 @@ 1.123 whether the worm is doing the action they describe. =curled?= 1.124 relies on proprioception, =resting?= relies on touch, =wiggling?= 1.125 relies on a Fourier analysis of muscle contraction, and 1.126 - =grand-circle?= relies on touch and reuses =curled?= as a guard. 1.127 + =grand-circle?= relies on touch and reuses =curled?= in its 1.128 + definition, showing how embodied predicates can be composed. 1.129 1.130 #+caption: Program for detecting whether the worm is curled. This is the 1.131 #+caption: simplest action predicate, because it only uses the last frame 1.132 #+caption: of sensory experience, and only uses proprioceptive data. Even 1.133 #+caption: this simple predicate, however, is automatically frame 1.134 - #+caption: independent and ignores vermopomorphic differences such as 1.135 - #+caption: worm textures and colors. 1.136 + #+caption: independent and ignores vermopomorphic \footnote{Like 1.137 + #+caption: \emph{anthropomorphic}, except for worms instead of humans.} 1.138 + #+caption: differences such as worm textures and colors. 1.139 #+name: curled 1.140 #+begin_listing clojure 1.141 #+begin_src clojure 1.142 @@ -2735,7 +2742,7 @@ 1.143 1.144 The trick now is to make the action predicates work even when the 1.145 sensory data on which they depend is absent. If I can do that, then 1.146 - I will have gained much, 1.147 + I will have gained much. 1.148 1.149 ** \Phi-space describes the worm's experiences 1.150