comparison thesis/cortex.org @ 567:7837ca42d82c

final thesis printed!
author Robert McIntyre <rlm@mit.edu>
date Mon, 12 May 2014 16:33:34 -0400
parents b9b8567c14ee
children
comparison
equal deleted inserted replaced
566:b9b8567c14ee 567:7837ca42d82c
347 347
348 - You can design new creatures using Blender (\cite{blender}), a 348 - You can design new creatures using Blender (\cite{blender}), a
349 popular, free 3D modeling program. Each sense can be specified 349 popular, free 3D modeling program. Each sense can be specified
350 using special Blender nodes with biologically inspired 350 using special Blender nodes with biologically inspired
351 parameters. You need not write any code to create a creature, and 351 parameters. You need not write any code to create a creature, and
352 can use a wide library of pre-existing blender models as a base 352 can use a wide library of pre-existing Blender models as a base
353 for your own creatures. 353 for your own creatures.
354 354
355 - =CORTEX= implements a wide variety of senses: touch, 355 - =CORTEX= implements a wide variety of senses: touch,
356 proprioception, vision, hearing, and muscle tension. Complicated 356 proprioception, vision, hearing, and muscle tension. Complicated
357 senses like touch and vision involve multiple sensory elements 357 senses like touch and vision involve multiple sensory elements
766 766
767 =CORTEX= creates a creature in two steps: first, it traverses the 767 =CORTEX= creates a creature in two steps: first, it traverses the
768 nodes in the Blender file and creates physical representations for 768 nodes in the Blender file and creates physical representations for
769 any of them that have mass defined in their Blender meta-data. 769 any of them that have mass defined in their Blender meta-data.
770 770
771 #+caption: Program for iterating through the nodes in a blender file 771 #+caption: Program for iterating through the nodes in a Blender file
772 #+caption: and generating physical jMonkeyEngine3 objects with mass 772 #+caption: and generating physical jMonkeyEngine3 objects with mass
773 #+caption: and a matching physics shape. 773 #+caption: and a matching physics shape.
774 #+name: physical 774 #+name: physical
775 #+begin_listing clojure 775 #+begin_listing clojure
776 #+begin_src clojure 776 #+begin_src clojure
1142 (defn add-eye! 1142 (defn add-eye!
1143 "Create a Camera centered on the current position of 'eye which 1143 "Create a Camera centered on the current position of 'eye which
1144 follows the closest physical node in 'creature. The camera will 1144 follows the closest physical node in 'creature. The camera will
1145 point in the X direction and use the Z vector as up as determined 1145 point in the X direction and use the Z vector as up as determined
1146 by the rotation of these vectors in Blender coordinate space. Use 1146 by the rotation of these vectors in Blender coordinate space. Use
1147 XZY rotation for the node in blender." 1147 XZY rotation for the node in Blender."
1148 [#^Node creature #^Spatial eye] 1148 [#^Node creature #^Spatial eye]
1149 (let [target (closest-node creature eye) 1149 (let [target (closest-node creature eye)
1150 [cam-width cam-height] 1150 [cam-width cam-height]
1151 ;;[640 480] ;; graphics card on laptop doesn't support 1151 ;;[640 480] ;; graphics card on laptop doesn't support
1152 ;; arbitrary dimensions. 1152 ;; arbitrary dimensions.
1308 [[./images/worm-vision.png]] 1308 [[./images/worm-vision.png]]
1309 1309
1310 The vision code is not much more complicated than the body code, 1310 The vision code is not much more complicated than the body code,
1311 and enables multiple further paths for simulated vision. For 1311 and enables multiple further paths for simulated vision. For
1312 example, it is quite easy to create bifocal vision -- you just 1312 example, it is quite easy to create bifocal vision -- you just
1313 make two eyes next to each other in blender! It is also possible 1313 make two eyes next to each other in Blender! It is also possible
1314 to encode vision transforms in the retinal files. For example, the 1314 to encode vision transforms in the retinal files. For example, the
1315 human like retina file in figure \ref{retina} approximates a 1315 human like retina file in figure \ref{retina} approximates a
1316 log-polar transform. 1316 log-polar transform.
1317 1317
1318 This vision code has already been absorbed by the jMonkeyEngine 1318 This vision code has already been absorbed by the jMonkeyEngine
1509 1509
1510 With this special context-switching device, and some ugly JNI 1510 With this special context-switching device, and some ugly JNI
1511 bindings that are not worth mentioning, =CORTEX= gains the ability 1511 bindings that are not worth mentioning, =CORTEX= gains the ability
1512 to access multiple sound streams from =OpenAL=. 1512 to access multiple sound streams from =OpenAL=.
1513 1513
1514 #+caption: Program to create an ear from a blender empty node. The ear 1514 #+caption: Program to create an ear from a Blender empty node. The ear
1515 #+caption: follows around the nearest physical object and passes 1515 #+caption: follows around the nearest physical object and passes
1516 #+caption: all sensory data to a continuation function. 1516 #+caption: all sensory data to a continuation function.
1517 #+name: add-ear 1517 #+name: add-ear
1518 #+begin_listing clojure 1518 #+begin_listing clojure
1519 #+BEGIN_SRC clojure 1519 #+BEGIN_SRC clojure
2314 #+caption: file's metadata. 2314 #+caption: file's metadata.
2315 #+name: motor-pool 2315 #+name: motor-pool
2316 #+begin_listing clojure 2316 #+begin_listing clojure
2317 #+BEGIN_SRC clojure 2317 #+BEGIN_SRC clojure
2318 (defn muscle-profile-image 2318 (defn muscle-profile-image
2319 "Get the muscle-profile image from the node's blender meta-data." 2319 "Get the muscle-profile image from the node's Blender meta-data."
2320 [#^Node muscle] 2320 [#^Node muscle]
2321 (if-let [image (meta-data muscle "muscle")] 2321 (if-let [image (meta-data muscle "muscle")]
2322 (load-image image))) 2322 (load-image image)))
2323 2323
2324 (defn muscle-strength 2324 (defn muscle-strength
2862 2862
2863 The worm's previous experience of lying on the ground and lifting 2863 The worm's previous experience of lying on the ground and lifting
2864 its head generates possible interpretations for each frame (the 2864 its head generates possible interpretations for each frame (the
2865 numbers are experience-indices): 2865 numbers are experience-indices):
2866 2866
2867 \clearpage
2868
2867 #+BEGIN_EXAMPLE 2869 #+BEGIN_EXAMPLE
2868 [ flat, flat, flat, flat, flat, flat, flat, lift-head ] 2870 [ flat, flat, flat, flat, flat, flat, flat, lift-head ]
2869 1 1 1 1 1 1 1 4 2871 1 1 1 1 1 1 1 4
2870 2 2 2 2 2 2 2 2872 2 2 2 2 2 2 2
2871 3 3 3 3 3 3 3 2873 3 3 3 3 3 3 3
3142 #+caption: From only proprioceptive data, =EMPATH= was able to infer 3144 #+caption: From only proprioceptive data, =EMPATH= was able to infer
3143 #+caption: the complete sensory experience and classify four poses 3145 #+caption: the complete sensory experience and classify four poses
3144 #+caption: (The last panel shows a composite image of /wiggling/, 3146 #+caption: (The last panel shows a composite image of /wiggling/,
3145 #+caption: a dynamic pose.) 3147 #+caption: a dynamic pose.)
3146 #+name: empathy-debug-image 3148 #+name: empathy-debug-image
3147 #+ATTR_LaTeX: :width 10cm :placement [H] 3149 #+ATTR_LaTeX: :width 10cm
3148 [[./images/empathy-1.png]] 3150 [[./images/empathy-1.png]]
3149 3151
3150 One way to measure the performance of =EMPATH= is to compare the 3152 One way to measure the performance of =EMPATH= is to compare the
3151 suitability of the imagined sense experience to trigger the same 3153 suitability of the imagined sense experience to trigger the same
3152 action predicates as the real sensory experience. 3154 action predicates as the real sensory experience.
3706 #+caption: This is the tactile-sensor-profile for the upper segment 3708 #+caption: This is the tactile-sensor-profile for the upper segment
3707 #+caption: of a fingertip. It defines regions of high touch sensitivity 3709 #+caption: of a fingertip. It defines regions of high touch sensitivity
3708 #+caption: (where there are many white pixels) and regions of low 3710 #+caption: (where there are many white pixels) and regions of low
3709 #+caption: sensitivity (where white pixels are sparse). 3711 #+caption: sensitivity (where white pixels are sparse).
3710 #+name: guide-fingertip-UV 3712 #+name: guide-fingertip-UV
3711 #+ATTR_LaTeX: :width 9cm :placement [H] 3713 #+ATTR_LaTeX: :width 9cm
3712 [[./images/finger-UV.png]] 3714 [[./images/finger-UV.png]]
3713 3715
3714 #+caption: The fingertip UV-image form above applied to a simple 3716 #+caption: The fingertip UV-image form above applied to a simple
3715 #+caption: model of a fingertip. 3717 #+caption: model of a fingertip.
3716 #+name: guide-fingertip 3718 #+name: guide-fingertip
3717 #+ATTR_LaTeX: :width 9cm :placement [H] 3719 #+ATTR_LaTeX: :width 9cm
3718 [[./images/finger-1.png]] 3720 [[./images/finger-1.png]]
3719 3721
3720 *** Proprioception 3722 *** Proprioception
3721 3723
3722 Proprioception is tied to each joint node -- nothing special must 3724 Proprioception is tied to each joint node -- nothing special must
3753 #+caption: of each motor neuron in a muscle. White is weakest 3755 #+caption: of each motor neuron in a muscle. White is weakest
3754 #+caption: and dark red is strongest. This particular pattern 3756 #+caption: and dark red is strongest. This particular pattern
3755 #+caption: has weaker motor neurons at the beginning, just 3757 #+caption: has weaker motor neurons at the beginning, just
3756 #+caption: like human muscle. 3758 #+caption: like human muscle.
3757 #+name: muscle-recruit 3759 #+name: muscle-recruit
3758 #+ATTR_LaTeX: :width 7cm :placement [H] 3760 #+ATTR_LaTeX: :width 7cm
3759 [[./images/basic-muscle.png]] 3761 [[./images/basic-muscle.png]]
3760 3762
3761 Muscles twist the nearest physical object about the muscle node's 3763 Muscles twist the nearest physical object about the muscle node's
3762 Z-axis. I recommend using the ``Single Arrow'' display mode for 3764 Z-axis. I recommend using the ``Single Arrow'' display mode for
3763 muscles and using the right hand rule to determine which way the 3765 muscles and using the right hand rule to determine which way the