Mercurial > cortex

comparison thesis/cortex.org @ 521:2529c34caa1a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
changes from mom.
author rlm
date Mon, 31 Mar 2014 08:29:50 -0400
parents 1803144ec9ae
children 1e51263afdc0
comparison
equal deleted inserted replaced
520:1803144ec9ae 521:2529c34caa1a
184 factors the action recognition problem into two easier problems. To 184 factors the action recognition problem into two easier problems. To
185 use empathy, you need an /aligner/, which takes the video and a 185 use empathy, you need an /aligner/, which takes the video and a
186 model of your body, and aligns the model with the video. Then, you 186 model of your body, and aligns the model with the video. Then, you
187 need a /recognizer/, which uses the aligned model to interpret the 187 need a /recognizer/, which uses the aligned model to interpret the
188 action. The power in this method lies in the fact that you describe 188 action. The power in this method lies in the fact that you describe
189 all actions form a body-centered viewpoint. You are less tied to 189 all actions from a body-centered viewpoint. You are less tied to
190 the particulars of any visual representation of the actions. If you 190 the particulars of any visual representation of the actions. If you
191 teach the system what ``running'' is, and you have a good enough 191 teach the system what ``running'' is, and you have a good enough
192 aligner, the system will from then on be able to recognize running 192 aligner, the system will from then on be able to recognize running
193 from any point of view, even strange points of view like above or 193 from any point of view, even strange points of view like above or
194 underneath the runner. This is in contrast to action recognition 194 underneath the runner. This is in contrast to action recognition
294 #+ATTR_LaTeX: :width 15cm 294 #+ATTR_LaTeX: :width 15cm
295 [[./images/worm-poses.png]] 295 [[./images/worm-poses.png]]
296 296
297 *** Main Results 297 *** Main Results
298 298
299 - After one-shot supervised training, =EMPATH= was able recognize a 299 - After one-shot supervised training, =EMPATH= was able to
300 wide variety of static poses and dynamic actions---ranging from 300 recognize a wide variety of static poses and dynamic
301 curling in a circle to wiggling with a particular frequency --- 301 actions---ranging from curling in a circle to wiggling with a
302 with 95\% accuracy. 302 particular frequency --- with 95\% accuracy.
303 303
304 - These results were completely independent of viewing angle 304 - These results were completely independent of viewing angle
305 because the underlying body-centered language fundamentally is 305 because the underlying body-centered language fundamentally is
306 independent; once an action is learned, it can be recognized 306 independent; once an action is learned, it can be recognized
307 equally well from any viewing angle. 307 equally well from any viewing angle.
379 #+caption: joints are described using special nodes in Blender. 379 #+caption: joints are described using special nodes in Blender.
380 #+name: worm-recognition-intro-2 380 #+name: worm-recognition-intro-2
381 #+ATTR_LaTeX: :width 12cm 381 #+ATTR_LaTeX: :width 12cm
382 [[./images/blender-worm.png]] 382 [[./images/blender-worm.png]]
383 383
384 Here are some thing I anticipate that =CORTEX= might be used for: 384 Here are some things I anticipate that =CORTEX= might be used for:
385 385
386 - exploring new ideas about sensory integration 386 - exploring new ideas about sensory integration
387 - distributed communication among swarm creatures 387 - distributed communication among swarm creatures
388 - self-learning using free exploration, 388 - self-learning using free exploration,
389 - evolutionary algorithms involving creature construction 389 - evolutionary algorithms involving creature construction
540 complicated surface of the skin onto a two dimensional image. 540 complicated surface of the skin onto a two dimensional image.
541 541
542 Most human senses consist of many discrete sensors of various 542 Most human senses consist of many discrete sensors of various
543 properties distributed along a surface at various densities. For 543 properties distributed along a surface at various densities. For
544 skin, it is Pacinian corpuscles, Meissner's corpuscles, Merkel's 544 skin, it is Pacinian corpuscles, Meissner's corpuscles, Merkel's
545 disks, and Ruffini's endings (\cite{9.01-textbook), which detect 545 disks, and Ruffini's endings \cite{textbook901}, which detect
546 pressure and vibration of various intensities. For ears, it is the 546 pressure and vibration of various intensities. For ears, it is the
547 stereocilia distributed along the basilar membrane inside the 547 stereocilia distributed along the basilar membrane inside the
548 cochlea; each one is sensitive to a slightly different frequency of 548 cochlea; each one is sensitive to a slightly different frequency of
549 sound. For eyes, it is rods and cones distributed along the surface 549 sound. For eyes, it is rods and cones distributed along the surface
550 of the retina. In each case, we can describe the sense with a 550 of the retina. In each case, we can describe the sense with a
558 Therefore, =CORTEX= must support the ability to create objects and 558 Therefore, =CORTEX= must support the ability to create objects and
559 then be able to ``paint'' points along their surfaces to describe 559 then be able to ``paint'' points along their surfaces to describe
560 each sense. 560 each sense.
561 561
562 Fortunately this idea is already a well known computer graphics 562 Fortunately this idea is already a well known computer graphics
563 technique called called /UV-mapping/. The three-dimensional surface 563 technique called /UV-mapping/. The three-dimensional surface of a
564 of a model is cut and smooshed until it fits on a two-dimensional 564 model is cut and smooshed until it fits on a two-dimensional
565 image. You paint whatever you want on that image, and when the 565 image. You paint whatever you want on that image, and when the
566 three-dimensional shape is rendered in a game the smooshing and 566 three-dimensional shape is rendered in a game the smooshing and
567 cutting is reversed and the image appears on the three-dimensional 567 cutting is reversed and the image appears on the three-dimensional
568 object. 568 object.
569 569
649 screengraphs to avoid drawing things that do not appear on the 649 screengraphs to avoid drawing things that do not appear on the
650 screen. It has an active community and several games in the 650 screen. It has an active community and several games in the
651 pipeline. The engine was not built to serve any particular 651 pipeline. The engine was not built to serve any particular
652 game but is instead meant to be used for any 3D game. 652 game but is instead meant to be used for any 3D game.
653 653
654 I chose jMonkeyEngine3 because it because it had the most features 654 I chose jMonkeyEngine3 because it had the most features out of all
655 out of all the free projects I looked at, and because I could then 655 the free projects I looked at, and because I could then write my
656 write my code in clojure, an implementation of =LISP= that runs on 656 code in clojure, an implementation of =LISP= that runs on the JVM.
657 the JVM.
658 657
659 ** =CORTEX= uses Blender to create creature models 658 ** =CORTEX= uses Blender to create creature models
660 659
661 For the simple worm-like creatures I will use later on in this 660 For the simple worm-like creatures I will use later on in this
662 thesis, I could define a simple API in =CORTEX= that would allow 661 thesis, I could define a simple API in =CORTEX= that would allow