Mercurial > cortex
comparison thesis/cortex.org @ 567:7837ca42d82c
final thesis printed!
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Mon, 12 May 2014 16:33:34 -0400 |
parents | b9b8567c14ee |
children |
comparison
equal
deleted
inserted
replaced
566:b9b8567c14ee | 567:7837ca42d82c |
---|---|
347 | 347 |
348 - You can design new creatures using Blender (\cite{blender}), a | 348 - You can design new creatures using Blender (\cite{blender}), a |
349 popular, free 3D modeling program. Each sense can be specified | 349 popular, free 3D modeling program. Each sense can be specified |
350 using special Blender nodes with biologically inspired | 350 using special Blender nodes with biologically inspired |
351 parameters. You need not write any code to create a creature, and | 351 parameters. You need not write any code to create a creature, and |
352 can use a wide library of pre-existing blender models as a base | 352 can use a wide library of pre-existing Blender models as a base |
353 for your own creatures. | 353 for your own creatures. |
354 | 354 |
355 - =CORTEX= implements a wide variety of senses: touch, | 355 - =CORTEX= implements a wide variety of senses: touch, |
356 proprioception, vision, hearing, and muscle tension. Complicated | 356 proprioception, vision, hearing, and muscle tension. Complicated |
357 senses like touch and vision involve multiple sensory elements | 357 senses like touch and vision involve multiple sensory elements |
766 | 766 |
767 =CORTEX= creates a creature in two steps: first, it traverses the | 767 =CORTEX= creates a creature in two steps: first, it traverses the |
768 nodes in the Blender file and creates physical representations for | 768 nodes in the Blender file and creates physical representations for |
769 any of them that have mass defined in their Blender meta-data. | 769 any of them that have mass defined in their Blender meta-data. |
770 | 770 |
771 #+caption: Program for iterating through the nodes in a blender file | 771 #+caption: Program for iterating through the nodes in a Blender file |
772 #+caption: and generating physical jMonkeyEngine3 objects with mass | 772 #+caption: and generating physical jMonkeyEngine3 objects with mass |
773 #+caption: and a matching physics shape. | 773 #+caption: and a matching physics shape. |
774 #+name: physical | 774 #+name: physical |
775 #+begin_listing clojure | 775 #+begin_listing clojure |
776 #+begin_src clojure | 776 #+begin_src clojure |
1142 (defn add-eye! | 1142 (defn add-eye! |
1143 "Create a Camera centered on the current position of 'eye which | 1143 "Create a Camera centered on the current position of 'eye which |
1144 follows the closest physical node in 'creature. The camera will | 1144 follows the closest physical node in 'creature. The camera will |
1145 point in the X direction and use the Z vector as up as determined | 1145 point in the X direction and use the Z vector as up as determined |
1146 by the rotation of these vectors in Blender coordinate space. Use | 1146 by the rotation of these vectors in Blender coordinate space. Use |
1147 XZY rotation for the node in blender." | 1147 XZY rotation for the node in Blender." |
1148 [#^Node creature #^Spatial eye] | 1148 [#^Node creature #^Spatial eye] |
1149 (let [target (closest-node creature eye) | 1149 (let [target (closest-node creature eye) |
1150 [cam-width cam-height] | 1150 [cam-width cam-height] |
1151 ;;[640 480] ;; graphics card on laptop doesn't support | 1151 ;;[640 480] ;; graphics card on laptop doesn't support |
1152 ;; arbitrary dimensions. | 1152 ;; arbitrary dimensions. |
1308 [[./images/worm-vision.png]] | 1308 [[./images/worm-vision.png]] |
1309 | 1309 |
1310 The vision code is not much more complicated than the body code, | 1310 The vision code is not much more complicated than the body code, |
1311 and enables multiple further paths for simulated vision. For | 1311 and enables multiple further paths for simulated vision. For |
1312 example, it is quite easy to create bifocal vision -- you just | 1312 example, it is quite easy to create bifocal vision -- you just |
1313 make two eyes next to each other in blender! It is also possible | 1313 make two eyes next to each other in Blender! It is also possible |
1314 to encode vision transforms in the retinal files. For example, the | 1314 to encode vision transforms in the retinal files. For example, the |
1315 human like retina file in figure \ref{retina} approximates a | 1315 human like retina file in figure \ref{retina} approximates a |
1316 log-polar transform. | 1316 log-polar transform. |
1317 | 1317 |
1318 This vision code has already been absorbed by the jMonkeyEngine | 1318 This vision code has already been absorbed by the jMonkeyEngine |
1509 | 1509 |
1510 With this special context-switching device, and some ugly JNI | 1510 With this special context-switching device, and some ugly JNI |
1511 bindings that are not worth mentioning, =CORTEX= gains the ability | 1511 bindings that are not worth mentioning, =CORTEX= gains the ability |
1512 to access multiple sound streams from =OpenAL=. | 1512 to access multiple sound streams from =OpenAL=. |
1513 | 1513 |
1514 #+caption: Program to create an ear from a blender empty node. The ear | 1514 #+caption: Program to create an ear from a Blender empty node. The ear |
1515 #+caption: follows around the nearest physical object and passes | 1515 #+caption: follows around the nearest physical object and passes |
1516 #+caption: all sensory data to a continuation function. | 1516 #+caption: all sensory data to a continuation function. |
1517 #+name: add-ear | 1517 #+name: add-ear |
1518 #+begin_listing clojure | 1518 #+begin_listing clojure |
1519 #+BEGIN_SRC clojure | 1519 #+BEGIN_SRC clojure |
2314 #+caption: file's metadata. | 2314 #+caption: file's metadata. |
2315 #+name: motor-pool | 2315 #+name: motor-pool |
2316 #+begin_listing clojure | 2316 #+begin_listing clojure |
2317 #+BEGIN_SRC clojure | 2317 #+BEGIN_SRC clojure |
2318 (defn muscle-profile-image | 2318 (defn muscle-profile-image |
2319 "Get the muscle-profile image from the node's blender meta-data." | 2319 "Get the muscle-profile image from the node's Blender meta-data." |
2320 [#^Node muscle] | 2320 [#^Node muscle] |
2321 (if-let [image (meta-data muscle "muscle")] | 2321 (if-let [image (meta-data muscle "muscle")] |
2322 (load-image image))) | 2322 (load-image image))) |
2323 | 2323 |
2324 (defn muscle-strength | 2324 (defn muscle-strength |
2862 | 2862 |
2863 The worm's previous experience of lying on the ground and lifting | 2863 The worm's previous experience of lying on the ground and lifting |
2864 its head generates possible interpretations for each frame (the | 2864 its head generates possible interpretations for each frame (the |
2865 numbers are experience-indices): | 2865 numbers are experience-indices): |
2866 | 2866 |
2867 \clearpage | |
2868 | |
2867 #+BEGIN_EXAMPLE | 2869 #+BEGIN_EXAMPLE |
2868 [ flat, flat, flat, flat, flat, flat, flat, lift-head ] | 2870 [ flat, flat, flat, flat, flat, flat, flat, lift-head ] |
2869 1 1 1 1 1 1 1 4 | 2871 1 1 1 1 1 1 1 4 |
2870 2 2 2 2 2 2 2 | 2872 2 2 2 2 2 2 2 |
2871 3 3 3 3 3 3 3 | 2873 3 3 3 3 3 3 3 |
3142 #+caption: From only proprioceptive data, =EMPATH= was able to infer | 3144 #+caption: From only proprioceptive data, =EMPATH= was able to infer |
3143 #+caption: the complete sensory experience and classify four poses | 3145 #+caption: the complete sensory experience and classify four poses |
3144 #+caption: (The last panel shows a composite image of /wiggling/, | 3146 #+caption: (The last panel shows a composite image of /wiggling/, |
3145 #+caption: a dynamic pose.) | 3147 #+caption: a dynamic pose.) |
3146 #+name: empathy-debug-image | 3148 #+name: empathy-debug-image |
3147 #+ATTR_LaTeX: :width 10cm :placement [H] | 3149 #+ATTR_LaTeX: :width 10cm |
3148 [[./images/empathy-1.png]] | 3150 [[./images/empathy-1.png]] |
3149 | 3151 |
3150 One way to measure the performance of =EMPATH= is to compare the | 3152 One way to measure the performance of =EMPATH= is to compare the |
3151 suitability of the imagined sense experience to trigger the same | 3153 suitability of the imagined sense experience to trigger the same |
3152 action predicates as the real sensory experience. | 3154 action predicates as the real sensory experience. |
3706 #+caption: This is the tactile-sensor-profile for the upper segment | 3708 #+caption: This is the tactile-sensor-profile for the upper segment |
3707 #+caption: of a fingertip. It defines regions of high touch sensitivity | 3709 #+caption: of a fingertip. It defines regions of high touch sensitivity |
3708 #+caption: (where there are many white pixels) and regions of low | 3710 #+caption: (where there are many white pixels) and regions of low |
3709 #+caption: sensitivity (where white pixels are sparse). | 3711 #+caption: sensitivity (where white pixels are sparse). |
3710 #+name: guide-fingertip-UV | 3712 #+name: guide-fingertip-UV |
3711 #+ATTR_LaTeX: :width 9cm :placement [H] | 3713 #+ATTR_LaTeX: :width 9cm |
3712 [[./images/finger-UV.png]] | 3714 [[./images/finger-UV.png]] |
3713 | 3715 |
3714 #+caption: The fingertip UV-image form above applied to a simple | 3716 #+caption: The fingertip UV-image form above applied to a simple |
3715 #+caption: model of a fingertip. | 3717 #+caption: model of a fingertip. |
3716 #+name: guide-fingertip | 3718 #+name: guide-fingertip |
3717 #+ATTR_LaTeX: :width 9cm :placement [H] | 3719 #+ATTR_LaTeX: :width 9cm |
3718 [[./images/finger-1.png]] | 3720 [[./images/finger-1.png]] |
3719 | 3721 |
3720 *** Proprioception | 3722 *** Proprioception |
3721 | 3723 |
3722 Proprioception is tied to each joint node -- nothing special must | 3724 Proprioception is tied to each joint node -- nothing special must |
3753 #+caption: of each motor neuron in a muscle. White is weakest | 3755 #+caption: of each motor neuron in a muscle. White is weakest |
3754 #+caption: and dark red is strongest. This particular pattern | 3756 #+caption: and dark red is strongest. This particular pattern |
3755 #+caption: has weaker motor neurons at the beginning, just | 3757 #+caption: has weaker motor neurons at the beginning, just |
3756 #+caption: like human muscle. | 3758 #+caption: like human muscle. |
3757 #+name: muscle-recruit | 3759 #+name: muscle-recruit |
3758 #+ATTR_LaTeX: :width 7cm :placement [H] | 3760 #+ATTR_LaTeX: :width 7cm |
3759 [[./images/basic-muscle.png]] | 3761 [[./images/basic-muscle.png]] |
3760 | 3762 |
3761 Muscles twist the nearest physical object about the muscle node's | 3763 Muscles twist the nearest physical object about the muscle node's |
3762 Z-axis. I recommend using the ``Single Arrow'' display mode for | 3764 Z-axis. I recommend using the ``Single Arrow'' display mode for |
3763 muscles and using the right hand rule to determine which way the | 3765 muscles and using the right hand rule to determine which way the |