Mercurial > cortex
diff thesis/cortex.org @ 452:f339e3d5cc8c
finish draft of chapter 3.
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Wed, 26 Mar 2014 22:17:42 -0400 |
parents | 0a4362d1f138 |
children | 6db37c4aa1ee |
line wrap: on
line diff
1.1 --- a/thesis/cortex.org Wed Mar 26 20:38:17 2014 -0400 1.2 +++ b/thesis/cortex.org Wed Mar 26 22:17:42 2014 -0400 1.3 @@ -211,7 +211,8 @@ 1.4 #+caption: full circle. Imagine how you would replicate this functionality 1.5 #+caption: using low-level pixel features such as HOG filters! 1.6 #+name: grand-circle-intro 1.7 - #+begin_listing clojure 1.8 + #+attr_latex: [htpb] 1.9 +#+begin_listing clojure 1.10 #+begin_src clojure 1.11 (defn grand-circle? 1.12 "Does the worm form a majestic circle (one end touching the other)?" 1.13 @@ -400,7 +401,7 @@ 1.14 :muscles (movement! model)})) 1.15 #+end_src 1.16 #+end_listing 1.17 - 1.18 + 1.19 ** Embodiment factors action recognition into managable parts 1.20 1.21 Using empathy, I divide the problem of action recognition into a 1.22 @@ -436,7 +437,8 @@ 1.23 #+caption: independent and ignores vermopomorphic differences such as 1.24 #+caption: worm textures and colors. 1.25 #+name: curled 1.26 - #+begin_listing clojure 1.27 + #+attr_latex: [htpb] 1.28 +#+begin_listing clojure 1.29 #+begin_src clojure 1.30 (defn curled? 1.31 "Is the worm curled up?" 1.32 @@ -451,7 +453,9 @@ 1.33 #+caption: Program for summarizing the touch information in a patch 1.34 #+caption: of skin. 1.35 #+name: touch-summary 1.36 - #+begin_listing clojure 1.37 + #+attr_latex: [htpb] 1.38 + 1.39 +#+begin_listing clojure 1.40 #+begin_src clojure 1.41 (defn contact 1.42 "Determine how much contact a particular worm segment has with 1.43 @@ -476,7 +480,8 @@ 1.44 #+caption: floor. Note that this function contains no references to 1.45 #+caption: proprioction at all. 1.46 #+name: resting 1.47 - #+begin_listing clojure 1.48 + #+attr_latex: [htpb] 1.49 +#+begin_listing clojure 1.50 #+begin_src clojure 1.51 (def worm-segment-bottom (rect-region [8 15] [14 22])) 1.52 1.53 @@ -495,7 +500,8 @@ 1.54 #+caption: I am able to both use a previous action predicate (=curled?=) 1.55 #+caption: as well as the direct tactile experience of the head and tail. 1.56 #+name: grand-circle 1.57 - #+begin_listing clojure 1.58 + #+attr_latex: [htpb] 1.59 +#+begin_listing clojure 1.60 #+begin_src clojure 1.61 (def worm-segment-bottom-tip (rect-region [15 15] [22 22])) 1.62 1.63 @@ -525,7 +531,8 @@ 1.64 #+caption: wiggle but can't. Frustrated wiggling is very visually different 1.65 #+caption: from actual wiggling, but this definition gives it to us for free. 1.66 #+name: wiggling 1.67 - #+begin_listing clojure 1.68 + #+attr_latex: [htpb] 1.69 +#+begin_listing clojure 1.70 #+begin_src clojure 1.71 (defn fft [nums] 1.72 (map 1.73 @@ -565,7 +572,8 @@ 1.74 #+caption: Use the action predicates defined earlier to report on 1.75 #+caption: what the worm is doing while in simulation. 1.76 #+name: report-worm-activity 1.77 - #+begin_listing clojure 1.78 + #+attr_latex: [htpb] 1.79 +#+begin_listing clojure 1.80 #+begin_src clojure 1.81 (defn debug-experience 1.82 [experiences text] 1.83 @@ -683,7 +691,8 @@ 1.84 #+caption: Program to convert an experience vector into a 1.85 #+caption: proprioceptively binned lookup function. 1.86 #+name: bin 1.87 - #+begin_listing clojure 1.88 + #+attr_latex: [htpb] 1.89 +#+begin_listing clojure 1.90 #+begin_src clojure 1.91 (defn bin [digits] 1.92 (fn [angles] 1.93 @@ -741,7 +750,8 @@ 1.94 #+caption: and finding the longest (ie. most coherent) interpretation 1.95 #+caption: of the data. 1.96 #+name: longest-thread 1.97 - #+begin_listing clojure 1.98 + #+attr_latex: [htpb] 1.99 +#+begin_listing clojure 1.100 #+begin_src clojure 1.101 (defn longest-thread 1.102 "Find the longest thread from phi-index-sets. The index sets should 1.103 @@ -783,7 +793,8 @@ 1.104 #+caption: Fill in blanks in sensory experience by replicating the most 1.105 #+caption: recent experience. 1.106 #+name: infer-nils 1.107 - #+begin_listing clojure 1.108 + #+attr_latex: [htpb] 1.109 +#+begin_listing clojure 1.110 #+begin_src clojure 1.111 (defn infer-nils 1.112 "Replace nils with the next available non-nil element in the 1.113 @@ -804,19 +815,21 @@ 1.114 1.115 To use =EMPATH= with the worm, I first need to gather a set of 1.116 experiences from the worm that includes the actions I want to 1.117 - recognize. The =generate-phi-space= program (listint 1.118 + recognize. The =generate-phi-space= program (listing 1.119 \ref{generate-phi-space} runs the worm through a series of 1.120 exercices and gatheres those experiences into a vector. The 1.121 =do-all-the-things= program is a routine expressed in a simple 1.122 - muscle contraction script language for automated worm control. 1.123 + muscle contraction script language for automated worm control. It 1.124 + causes the worm to rest, curl, and wiggle over about 700 frames 1.125 + (approx. 11 seconds). 1.126 1.127 #+caption: Program to gather the worm's experiences into a vector for 1.128 #+caption: further processing. The =motor-control-program= line uses 1.129 #+caption: a motor control script that causes the worm to execute a series 1.130 #+caption: of ``exercices'' that include all the action predicates. 1.131 #+name: generate-phi-space 1.132 - #+attr_latex: [!H] 1.133 - #+begin_listing clojure 1.134 + #+attr_latex: [htpb] 1.135 +#+begin_listing clojure 1.136 #+begin_src clojure 1.137 (def do-all-the-things 1.138 (concat 1.139 @@ -843,7 +856,8 @@ 1.140 #+caption: Use longest thread and a phi-space generated from a short 1.141 #+caption: exercise routine to interpret actions during free play. 1.142 #+name: empathy-debug 1.143 - #+begin_listing clojure 1.144 + #+attr_latex: [htpb] 1.145 +#+begin_listing clojure 1.146 #+begin_src clojure 1.147 (defn init [] 1.148 (def phi-space (generate-phi-space)) 1.149 @@ -891,7 +905,8 @@ 1.150 #+caption: Determine how closely empathy approximates actual 1.151 #+caption: sensory data. 1.152 #+name: test-empathy-accuracy 1.153 - #+begin_listing clojure 1.154 + #+attr_latex: [htpb] 1.155 +#+begin_listing clojure 1.156 #+begin_src clojure 1.157 (def worm-action-label 1.158 (juxt grand-circle? curled? wiggling?)) 1.159 @@ -931,6 +946,7 @@ 1.160 1.161 #+caption: Program to generate \Phi-space using manual training. 1.162 #+name: manual-phi-space 1.163 + #+attr_latex: [htpb] 1.164 #+begin_listing clojure 1.165 #+begin_src clojure 1.166 (defn init-interactive [] 1.167 @@ -949,11 +965,174 @@ 1.168 1.169 After about 1 minute of manual training, I was able to achieve 95% 1.170 accuracy on manual testing of the worm using =init-interactive= and 1.171 - =test-empathy-accuracy=. The ability of the system to infer sensory 1.172 - states is truly impressive. 1.173 + =test-empathy-accuracy=. The majority of errors are near the 1.174 + boundaries of transitioning from one type of action to another. 1.175 + During these transitions the exact label for the action is more open 1.176 + to interpretation, and dissaggrement between empathy and experience 1.177 + is more excusable. 1.178 1.179 ** Digression: bootstrapping touch using free exploration 1.180 1.181 + In the previous section I showed how to compute actions in terms of 1.182 + body-centered predicates which relied averate touch activation of 1.183 + pre-defined regions of the worm's skin. What if, instead of recieving 1.184 + touch pre-grouped into the six faces of each worm segment, the true 1.185 + topology of the worm's skin was unknown? This is more similiar to how 1.186 + a nerve fiber bundle might be arranged. While two fibers that are 1.187 + close in a nerve bundle /might/ correspond to two touch sensors that 1.188 + are close together on the skin, the process of taking a complicated 1.189 + surface and forcing it into essentially a circle requires some cuts 1.190 + and rerragenments. 1.191 + 1.192 + In this section I show how to automatically learn the skin-topology of 1.193 + a worm segment by free exploration. As the worm rolls around on the 1.194 + floor, large sections of its surface get activated. If the worm has 1.195 + stopped moving, then whatever region of skin that is touching the 1.196 + floor is probably an important region, and should be recorded. 1.197 + 1.198 + #+caption: Program to detect whether the worm is in a resting state 1.199 + #+caption: with one face touching the floor. 1.200 + #+name: pure-touch 1.201 + #+begin_listing clojure 1.202 + #+begin_src clojure 1.203 +(def full-contact [(float 0.0) (float 0.1)]) 1.204 + 1.205 +(defn pure-touch? 1.206 + "This is worm specific code to determine if a large region of touch 1.207 + sensors is either all on or all off." 1.208 + [[coords touch :as touch-data]] 1.209 + (= (set (map first touch)) (set full-contact))) 1.210 + #+end_src 1.211 + #+end_listing 1.212 + 1.213 + After collecting these important regions, there will many nearly 1.214 + similiar touch regions. While for some purposes the subtle 1.215 + differences between these regions will be important, for my 1.216 + purposes I colapse them into mostly non-overlapping sets using 1.217 + =remove-similiar= in listing \ref{remove-similiar} 1.218 + 1.219 + #+caption: Program to take a lits of set of points and ``collapse them'' 1.220 + #+caption: so that the remaining sets in the list are siginificantly 1.221 + #+caption: different from each other. Prefer smaller sets to larger ones. 1.222 + #+name: remove-similiar 1.223 + #+begin_listing clojure 1.224 + #+begin_src clojure 1.225 +(defn remove-similar 1.226 + [coll] 1.227 + (loop [result () coll (sort-by (comp - count) coll)] 1.228 + (if (empty? coll) result 1.229 + (let [[x & xs] coll 1.230 + c (count x)] 1.231 + (if (some 1.232 + (fn [other-set] 1.233 + (let [oc (count other-set)] 1.234 + (< (- (count (union other-set x)) c) (* oc 0.1)))) 1.235 + xs) 1.236 + (recur result xs) 1.237 + (recur (cons x result) xs)))))) 1.238 + #+end_src 1.239 + #+end_listing 1.240 + 1.241 + Actually running this simulation is easy given =CORTEX='s facilities. 1.242 + 1.243 + #+caption: Collect experiences while the worm moves around. Filter the touch 1.244 + #+caption: sensations by stable ones, collapse similiar ones together, 1.245 + #+caption: and report the regions learned. 1.246 + #+name: learn-touch 1.247 + #+begin_listing clojure 1.248 + #+begin_src clojure 1.249 +(defn learn-touch-regions [] 1.250 + (let [experiences (atom []) 1.251 + world (apply-map 1.252 + worm-world 1.253 + (assoc (worm-segment-defaults) 1.254 + :experiences experiences))] 1.255 + (run-world world) 1.256 + (->> 1.257 + @experiences 1.258 + (drop 175) 1.259 + ;; access the single segment's touch data 1.260 + (map (comp first :touch)) 1.261 + ;; only deal with "pure" touch data to determine surfaces 1.262 + (filter pure-touch?) 1.263 + ;; associate coordinates with touch values 1.264 + (map (partial apply zipmap)) 1.265 + ;; select those regions where contact is being made 1.266 + (map (partial group-by second)) 1.267 + (map #(get % full-contact)) 1.268 + (map (partial map first)) 1.269 + ;; remove redundant/subset regions 1.270 + (map set) 1.271 + remove-similar))) 1.272 + 1.273 +(defn learn-and-view-touch-regions [] 1.274 + (map view-touch-region 1.275 + (learn-touch-regions))) 1.276 + #+end_src 1.277 + #+end_listing 1.278 + 1.279 + The only thing remining to define is the particular motion the worm 1.280 + must take. I accomplish this with a simple motor control program. 1.281 + 1.282 + #+caption: Motor control program for making the worm roll on the ground. 1.283 + #+caption: This could also be replaced with random motion. 1.284 + #+name: worm-roll 1.285 + #+begin_listing clojure 1.286 + #+begin_src clojure 1.287 +(defn touch-kinesthetics [] 1.288 + [[170 :lift-1 40] 1.289 + [190 :lift-1 19] 1.290 + [206 :lift-1 0] 1.291 + 1.292 + [400 :lift-2 40] 1.293 + [410 :lift-2 0] 1.294 + 1.295 + [570 :lift-2 40] 1.296 + [590 :lift-2 21] 1.297 + [606 :lift-2 0] 1.298 + 1.299 + [800 :lift-1 30] 1.300 + [809 :lift-1 0] 1.301 + 1.302 + [900 :roll-2 40] 1.303 + [905 :roll-2 20] 1.304 + [910 :roll-2 0] 1.305 + 1.306 + [1000 :roll-2 40] 1.307 + [1005 :roll-2 20] 1.308 + [1010 :roll-2 0] 1.309 + 1.310 + [1100 :roll-2 40] 1.311 + [1105 :roll-2 20] 1.312 + [1110 :roll-2 0] 1.313 + ]) 1.314 + #+end_src 1.315 + #+end_listing 1.316 + 1.317 + 1.318 + #+caption: The small worm rolls around on the floor, driven 1.319 + #+caption: by the motor control program in listing \ref{worm-roll}. 1.320 + #+name: worm-roll 1.321 + #+ATTR_LaTeX: :width 12cm 1.322 + [[./images/worm-roll.png]] 1.323 + 1.324 + 1.325 + #+caption: After completing its adventures, the worm now knows 1.326 + #+caption: how its touch sensors are arranged along its skin. These 1.327 + #+caption: are the regions that were deemed important by 1.328 + #+caption: =learn-touch-regions=. Note that the worm has discovered 1.329 + #+caption: that it has six sides. 1.330 + #+name: worm-touch-map 1.331 + #+ATTR_LaTeX: :width 12cm 1.332 + [[./images/touch-learn.png]] 1.333 + 1.334 + While simple, =learn-touch-regions= exploits regularities in both 1.335 + the worm's physiology and the worm's environment to correctly 1.336 + deduce that the worm has six sides. Note that =learn-touch-regions= 1.337 + would work just as well even if the worm's touch sense data were 1.338 + completely scrambled. The cross shape is just for convienence. This 1.339 + example justifies the use of pre-defined touch regions in =EMPATH=. 1.340 + 1.341 * Contributions 1.342 1.343