Mercurial > cortex

comparison thesis/cortex.org @ 440:b01c070b03d4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
save for tonight.
author Robert McIntyre <rlm@mit.edu>
date Sun, 23 Mar 2014 23:43:20 -0400
parents 97dc719fd1ac
children c20de2267d39
comparison
equal deleted inserted replaced
439:97dc719fd1ac 440:b01c070b03d4
39 hard problem! What features can you describe in terms of low level 39 hard problem! What features can you describe in terms of low level
40 functions of pixels that can even begin to describe what is 40 functions of pixels that can even begin to describe what is
41 happening here? 41 happening here?
42 42
43 Or suppose that you are building a program that recognizes 43 Or suppose that you are building a program that recognizes
44 chairs. How could you ``see'' the chair in the following picture? 44 chairs. How could you ``see'' the chair in the following pictures?
45 45
46 #+caption: When you look at this, do you think ``chair''? I certainly do. 46 #+caption: When you look at this, do you think ``chair''? I certainly do.
47 #+ATTR_LaTeX: :width 10cm 47 #+ATTR_LaTeX: :width 10cm
48 [[./images/invisible-chair.png]] 48 [[./images/invisible-chair.png]]
49 49
50 #+caption: The chair in this image is quite obvious to humans, but I 50 #+caption: The chair in this image is quite obvious to humans, but I
51 #+caption: doubt that any computer program can find it. 51 #+caption: doubt that any computer program can find it.
52 #+ATTR_LaTeX: :width 10cm 52 #+ATTR_LaTeX: :width 10cm
53 [[./images/fat-person-sitting-at-desk.jpg]] 53 [[./images/fat-person-sitting-at-desk.jpg]]
54 54
55 Finally, how is it that you can easily tell the difference between
56 how the girls /muscles/ are working in \ref{girl}?
55 57
56 I think humans are able to label 58 #+caption: The mysterious ``common sense'' appears here as you are able
57 such video as "drinking" because they imagine /themselves/ as the 59 #+caption: to ``see'' the difference in how the girl's arm muscles
58 cat, and imagine putting their face up against a stream of water and 60 #+caption: are activated differently in the two images.
59 sticking out their tongue. In that imagined world, they can feel the 61 #+name: girl
60 cool water hitting their tongue, and feel the water entering their 62 #+ATTR_LaTeX: :width 10cm
61 body, and are able to recognize that /feeling/ as drinking. So, the 63 [[./images/wall-push.png]]
62 label of the action is not really in the pixels of the image, but is 64
63 found clearly in a simulation inspired by those pixels. An 65
64 imaginative system, having been trained on drinking and non-drinking 66 These problems are difficult because the language of pixels is far
65 examples and learning that the most important component of drinking 67 removed from what we would consider to be an acceptable description
66 is the feeling of water sliding down one's throat, would analyze a 68 of the events in these images. In order to process them, we must
67 video of a cat drinking in the following manner: 69 raise the images into some higher level of abstraction where their
70 descriptions become more similar to how we would describe them in
71 English. The question is, how can we raise
72
73
74 I think humans are able to label such video as "drinking" because
75 they imagine /themselves/ as the cat, and imagine putting their face
76 up against a stream of water and sticking out their tongue. In that
77 imagined world, they can feel the cool water hitting their tongue,
78 and feel the water entering their body, and are able to recognize
79 that /feeling/ as drinking. So, the label of the action is not
80 really in the pixels of the image, but is found clearly in a
81 simulation inspired by those pixels. An imaginative system, having
82 been trained on drinking and non-drinking examples and learning that
83 the most important component of drinking is the feeling of water
84 sliding down one's throat, would analyze a video of a cat drinking
85 in the following manner:
68 86
69 - Create a physical model of the video by putting a "fuzzy" model 87 - Create a physical model of the video by putting a "fuzzy" model
70 of its own body in place of the cat. Also, create a simulation of 88 of its own body in place of the cat. Also, create a simulation of
71 the stream of water. 89 the stream of water.
72 90
78 - The action is now easily identified as drinking by the sense of 96 - The action is now easily identified as drinking by the sense of
79 taste alone. The other senses (such as the tongue moving in and 97 taste alone. The other senses (such as the tongue moving in and
80 out) help to give plausibility to the simulated action. Note that 98 out) help to give plausibility to the simulated action. Note that
81 the sense of vision, while critical in creating the simulation, 99 the sense of vision, while critical in creating the simulation,
82 is not critical for identifying the action from the simulation. 100 is not critical for identifying the action from the simulation.
83
84
85
86
87
88
89 101
90 cat drinking, mimes, leaning, common sense 102 cat drinking, mimes, leaning, common sense
91 103
92 ** =EMPATH= neatly solves recognition problems 104 ** =EMPATH= neatly solves recognition problems
93 105
117 129
118 ** Hearing is hard; =CORTEX= does it right 130 ** Hearing is hard; =CORTEX= does it right
119 131
120 ** Touch uses hundreds of hair-like elements 132 ** Touch uses hundreds of hair-like elements
121 133
122 ** Proprioception is the force that makes everything ``real'' 134 ** Proprioception is the sense that makes everything ``real''
123 135
124 ** Muscles are both effectors and sensors 136 ** Muscles are both effectors and sensors
125 137
126 ** =CORTEX= brings complex creatures to life! 138 ** =CORTEX= brings complex creatures to life!
127 139
137 149
138 ** \Phi-space describes the worm's experiences 150 ** \Phi-space describes the worm's experiences
139 151
140 ** Empathy is the process of tracing though \Phi-space 152 ** Empathy is the process of tracing though \Phi-space
141 153
142 ** Efficient action recognition via empathy 154 ** Efficient action recognition =EMPATH=
143 155
144 * Contributions 156 * Contributions
145 - Built =CORTEX=, a comprehensive platform for embodied AI 157 - Built =CORTEX=, a comprehensive platform for embodied AI
146 experiments. Has many new features lacking in other systems, such 158 experiments. Has many new features lacking in other systems, such
147 as sound. Easy to model/create new creatures. 159 as sound. Easy to model/create new creatures.