1.1 --- a/thesis/cortex.bib	Sat Mar 29 16:22:49 2014 -0400
     1.2 +++ b/thesis/cortex.bib	Sat Mar 29 20:12:52 2014 -0400
     1.3 @@ -23,4 +23,6 @@
     1.4  
     1.5  @COMMENT sussman's HACKER
     1.6  
     1.7 -@comment motion as a 4D manifold
     1.8 \ No newline at end of file
     1.9 +@comment motion as a 4D manifold
    1.10 +
    1.11 +@comment winston multiple modality white paper
    1.12 \ No newline at end of file

     2.1 --- a/thesis/cortex.org	Sat Mar 29 16:22:49 2014 -0400
     2.2 +++ b/thesis/cortex.org	Sat Mar 29 20:12:52 2014 -0400
     2.3 @@ -1627,7 +1627,7 @@
     2.4      #+caption: of a fingertip. It defines regions of high touch sensitivity 
     2.5      #+caption: (where there are many white pixels) and regions of low 
     2.6      #+caption: sensitivity (where white pixels are sparse).
     2.7 -    #+name: fimgertip-UV
     2.8 +    #+name: fingertip-UV
     2.9      #+ATTR_LaTeX: :width 13cm
    2.10      [[./images/finger-UV.png]]
    2.11  

     3.1 Binary file thesis/images/finger-2.png has changed

     4.1 --- a/thesis/rlm-cortex-meng.tex	Sat Mar 29 16:22:49 2014 -0400
     4.2 +++ b/thesis/rlm-cortex-meng.tex	Sat Mar 29 20:12:52 2014 -0400
     4.3 @@ -58,7 +58,7 @@
     4.4  \usepackage{xcolor}
     4.5  \definecolor{dark-red}{rgb}{0.4,0.15,0.15}
     4.6  \definecolor{dark-blue}{rgb}{0.15,0.4,0.15}
     4.7 -\definecolor{medium-blue}{rgb}{0,0,0.5}
     4.8 +\definecolor{medium-blue}{rgb}{0.15,0.4,0.4}
     4.9  \hypersetup{
    4.10      colorlinks, linkcolor={dark-red},
    4.11      citecolor={dark-blue}, urlcolor={medium-blue}

     5.1 --- a/thesis/user-guide.org	Sat Mar 29 16:22:49 2014 -0400
     5.2 +++ b/thesis/user-guide.org	Sat Mar 29 20:12:52 2014 -0400
     5.3 @@ -1,6 +1,338 @@
     5.4  * Appendix: =CORTEX= User Guide
     5.5  
     5.6 -  For future students who whould like to use =CORTEX= in their own
     5.7 -  projects.
     5.8 +  Those who write a thesis should endeavor to make their code not only
     5.9 +  accessable, but actually useable, as a way to pay back the community
    5.10 +  that made the thesis possible in the first place. This thesis would
    5.11 +  not be possible without Free Software such as jMonkeyEngine3,
    5.12 +  Blender, clojure, emacs, ffmpeg, and many other tools. That is why I
    5.13 +  have included this user guide, in the hope that someone else might
    5.14 +  find =CORTEX= useful.
    5.15  
    5.16 +** Obtaining =CORTEX= 
    5.17  
    5.18 +   You can get cortex from its mercurial repository at
    5.19 +   http://hg.bortreb.com/cortex. You may also download =CORTEX=
    5.20 +   releases at http://aurellem.org/cortex/releases/. As a condition of
    5.21 +   making this thesis, I have also provided Professor Winston the
    5.22 +   =CORTEX= source, and he knows how to run the demos and get started.
    5.23 +   You may also email me at =cortex@aurellem.org= and I may help where
    5.24 +   I can.
    5.25 +
    5.26 +** Running =CORTEX= 
    5.27 +   
    5.28 +   =CORTEX= comes with README and INSTALL files that will guide you
    5.29 +   through installation and running the test suite. In particular you
    5.30 +   should look at test =cortex.test= which contains test suites that
    5.31 +   run through all senses and multiple creatures.
    5.32 +
    5.33 +** Creating creatures
    5.34 +
    5.35 +   Creatures are created using /Blender/, a free 3D modeling program.
    5.36 +   You will need Blender version 2.6 when using the =CORTEX= included
    5.37 +   in this thesis. You create a =CORTEX= creature in a similiar manner
    5.38 +   to modeling anything in Blender, except that you also create
    5.39 +   several trees of empty nodes which define the creature's senses.
    5.40 +
    5.41 +*** Mass 
    5.42 +    
    5.43 +    To give an object mass in =CORTEX=, add a ``mass'' metadata label
    5.44 +    to the object with the mass in jMonkeyEngine units. Note that
    5.45 +    setting the mass to 0 causes the object to be immovable.
    5.46 +
    5.47 +*** Joints
    5.48 +
    5.49 +    Joints are created by creating an empty node named =joints= and
    5.50 +    then creating any number of empty child nodes to represent your
    5.51 +    creature's joints. The joint will automatically connect the
    5.52 +    closest two physical objects. It will help to set the empty node's
    5.53 +    display mode to ``Arrows'' so that you can clearly see the
    5.54 +    direction of the axes.
    5.55 +   
    5.56 +    Joint nodes should have the following metadata under the ``joint''
    5.57 +    label:
    5.58 +
    5.59 +    #+BEGIN_SRC clojure
    5.60 +;; ONE OF the following, under the label "joint":
    5.61 +{:type :point}
    5.62 +
    5.63 +;; OR
    5.64 +
    5.65 +{:type :hinge
    5.66 + :limit [<limit-low> <limit-high>]
    5.67 + :axis (Vector3f. <x> <y> <z>)}
    5.68 +;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
    5.69 +
    5.70 +;; OR
    5.71 +
    5.72 +{:type :cone
    5.73 + :limit-xz <lim-xz>
    5.74 + :limit-xy <lim-xy>
    5.75 + :twist    <lim-twist>}   ;(use XZY rotation mode in blender!)
    5.76 +    #+END_SRC
    5.77 +
    5.78 +*** Eyes
    5.79 +
    5.80 +    Eyes are created by creating an empty node named =eyes= and then
    5.81 +    creating any number of empty child nodes to represent your
    5.82 +    creature's eyes.
    5.83 +
    5.84 +    Eye nodes should have the following metadata under the ``eye''
    5.85 +    label:
    5.86 +
    5.87 +#+BEGIN_SRC clojure
    5.88 +{:red    <red-retina-definition>
    5.89 + :blue   <blue-retina-definition>
    5.90 + :green  <green-retina-definition>
    5.91 + :all    <all-retina-definition>
    5.92 + (<0xrrggbb> <custom-retina-image>)...
    5.93 +}
    5.94 +#+END_SRC
    5.95 +
    5.96 +    Any of the color channels may be omitted. You may also include
    5.97 +    your own color selectors, and in fact :red is equivalent to
    5.98 +    0xFF0000 and so forth. The eye will be placed at the same position
    5.99 +    as the empty node and will bind to the neatest physical object.
   5.100 +    The eye will point outward from the X-axis of the node, and ``up''
   5.101 +    will be in the direction of the X-axis of the node. It will help
   5.102 +    to set the empty node's display mode to ``Arrows'' so that you can
   5.103 +    clearly see the direction of the axes.
   5.104 +
   5.105 +    Each retina file should contain white pixels whever you want to be
   5.106 +    sensitive to your chosen color. If you want the entire field of
   5.107 +    view, specify :all of 0xFFFFFF and a retinal map that is entirely
   5.108 +    white. 
   5.109 +
   5.110 +    Here is a sample retinal map:
   5.111 +
   5.112 +    #+caption: An example retinal profile image. White pixels are 
   5.113 +    #+caption: photo-sensitive elements. The distribution of white 
   5.114 +    #+caption: pixels is denser in the middle and falls off at the 
   5.115 +    #+caption: edges and is inspired by the human retina.
   5.116 +    #+name: retina
   5.117 +    #+ATTR_LaTeX: :width 7cm :placement [H]
   5.118 +    [[./images/retina-small.png]]
   5.119 +
   5.120 +*** Hearing
   5.121 +
   5.122 +    Ears are created by creating an empty node named =ears= and then
   5.123 +    creating any number of empty child nodes to represent your
   5.124 +    creature's ears. 
   5.125 +
   5.126 +    Ear nodes do not require any metadata.
   5.127 +
   5.128 +    The ear will bind to and follow the closest physical node.
   5.129 +
   5.130 +*** Touch
   5.131 +
   5.132 +    Touch is handled similarly to mass. To make a particular object
   5.133 +    touch sensitive, add metadata of the following form under the
   5.134 +    object's ``touch'' metadata field:
   5.135 +    
   5.136 +    #+BEGIN_EXAMPLE
   5.137 +    <touch-UV-map-file-name>    
   5.138 +    #+END_EXAMPLE
   5.139 +
   5.140 +    You may also include an optional ``scale'' metadata number to
   5.141 +    specifiy the length of the touch feelers. The default is $0.1$,
   5.142 +    and this is generally sufficient.
   5.143 +
   5.144 +    The touch UV should contain white pixels for each touch sensor.
   5.145 +
   5.146 +    Here is an example touch-uv map that approximates a human finger,
   5.147 +    and its corresponding model.
   5.148 +
   5.149 +    #+caption: This is the tactile-sensor-profile for the upper segment 
   5.150 +    #+caption: of a fingertip. It defines regions of high touch sensitivity 
   5.151 +    #+caption: (where there are many white pixels) and regions of low 
   5.152 +    #+caption: sensitivity (where white pixels are sparse).
   5.153 +    #+name: guide-fingertip-UV
   5.154 +    #+ATTR_LaTeX: :width 9cm :placement [H]
   5.155 +    [[./images/finger-UV.png]]
   5.156 +
   5.157 +    #+caption: The fingertip UV-image form above applied to a simple
   5.158 +    #+caption: model of a fingertip.
   5.159 +    #+name: guide-fingertip
   5.160 +    #+ATTR_LaTeX: :width 9cm :placement [H]
   5.161 +    [[./images/finger-2.png]]
   5.162 +
   5.163 +*** Propriocepotion
   5.164 +
   5.165 +    Proprioception is tied to each joint node -- nothing special must
   5.166 +    be done in a blender model to enable proprioception other than
   5.167 +    creating joint nodes.
   5.168 +
   5.169 +*** Muscles
   5.170 +
   5.171 +    Muscles are created by creating an empty node named =muscles= and
   5.172 +    then creating any number of empty child nodes to represent your
   5.173 +    creature's muscles.
   5.174 +
   5.175 +    
   5.176 +    Muscle nodes should have the following metadata under the
   5.177 +    ``muscle'' label:
   5.178 +
   5.179 +    #+BEGIN_EXAMPLE
   5.180 +    <muscle-profile-file-name>
   5.181 +    #+END_EXAMPLE
   5.182 +
   5.183 +    Muscles should also have a ``strength'' metadata entry describing
   5.184 +    the muscle's total strength at full activation. 
   5.185 +
   5.186 +    Muscle profiles are simple images that contain the relative amount
   5.187 +    of muscle power in each simulated alpha motor neuron. The width of
   5.188 +    the image is the total size of the motor pool, and the redness of
   5.189 +    each neuron is the relative power of that motor pool.
   5.190 +
   5.191 +    While the profile image can have any dimensions, only the first
   5.192 +    line of pixels is used to define the muscle. Here is a sample
   5.193 +    muscle profile image that defines a human-like muscle.
   5.194 +
   5.195 +    #+caption: A muscle profile image that describes the strengths
   5.196 +    #+caption: of each motor neuron in a muscle. White is weakest 
   5.197 +    #+caption: and dark red is strongest. This particular pattern 
   5.198 +    #+caption: has weaker motor neurons at the beginning, just 
   5.199 +    #+caption: like human muscle.
   5.200 +    #+name: muscle-recruit
   5.201 +    #+ATTR_LaTeX: :width 7cm :placement [H]
   5.202 +    [[./images/basic-muscle.png]]
   5.203 +    
   5.204 +    Muscles twist the nearest physical object about the muscle node's
   5.205 +    Z-axis. I recommend using the ``Single Arrow'' display mode for
   5.206 +    muscles and using the right hand rule to determine which way the
   5.207 +    muscle will twist. To make a segment that can twist in multiple
   5.208 +    directions, create multiple, differently aligned muscles.
   5.209 +
   5.210 +** =CORTEX= API
   5.211 +
   5.212 +   These are the some functions exposed by =CORTEX= for creating
   5.213 +   worlds and simulating creatures. These are in addition to
   5.214 +   jMonkeyEngine3's extensive library, which is documented elsewhere.
   5.215 +
   5.216 +*** Simulation
   5.217 +   - =(world root-node key-map setup-fn update-fn)= :: create
   5.218 +        a simulation.
   5.219 +     - /root-node/     :: a =com.jme3.scene.Node= object which
   5.220 +          contains all of the objects that should be in the
   5.221 +          simulation.
   5.222 +
   5.223 +     - /key-map/       :: a map from strings describing keys to
   5.224 +          functions that should be executed whenever that key is
   5.225 +          pressed. the functions should take a SimpleApplication
   5.226 +          object and a boolean value. The SimpleApplication is the
   5.227 +          current simulation that is running, and the boolean is true
   5.228 +          if the key is being pressed, and false if it is being
   5.229 +          released. As an example,
   5.230 +          #+BEGIN_SRC clojure
   5.231 +       {"key-j" (fn [game value] (if value (println "key j pressed")))}		  
   5.232 +	  #+END_SRC
   5.233 +	  is a valid key-map which will cause the simulation to print
   5.234 +          a message whenever the 'j' key on the keyboard is pressed.
   5.235 +
   5.236 +     - /setup-fn/      :: a function that takes a =SimpleApplication=
   5.237 +          object. It is called once when initializing the simulation.
   5.238 +          Use it to create things like lights, change the gravity,
   5.239 +          initialize debug nodes, etc.
   5.240 +
   5.241 +     - /update-fn/     :: this function takes a =SimpleApplication=
   5.242 +          object and a float and is called every frame of the
   5.243 +          simulation. The float tells how many seconds is has been
   5.244 +          since the last frame was rendered, according to whatever
   5.245 +          clock jme is currently using. The default is to use IsoTimer
   5.246 +          which will result in this value always being the same.
   5.247 + 
   5.248 +
   5.249 +   - =(position-camera world position rotation)= :: set the position
   5.250 +        of the simulation's main camera.
   5.251 +   - =(enable-debug world)= :: turn on debug wireframes for each
   5.252 +        simulated object.
   5.253 +   - =(set-gravity world gravity)= :: set the gravity of a running
   5.254 +        simulation.
   5.255 +   - =(box length width height & {options})= :: create a box in the
   5.256 +        simulation. Options is a hash map specifying texture, mass,
   5.257 +        etc. Possible options are =:name=, =:color=, =:mass=,
   5.258 +        =:friction=, =:texture=, =:material=, =:position=,
   5.259 +        =:rotation=, =:shape=, and =:physical?=.
   5.260 +   - =(sphere radius & {options})= :: create a sphere in the simulation.
   5.261 +        Options are the same as in =box=.
   5.262 +   - =(load-blender-model file-name)= :: create a node structure
   5.263 +        representing that described in a blender file.
   5.264 +   - =(light-up-everything world)= :: distribute a standard compliment
   5.265 +        of lights throught the simulation. Should be adequate for most
   5.266 +        purposes.
   5.267 +   - =(node-seq node)= :: return a recursuve list of the node's
   5.268 +        children.
   5.269 +   - =(nodify name children)= :: construct a node given a node-name and
   5.270 +        desired children.
   5.271 +   - =(add-element world element)= :: add an object to a running world
   5.272 +        simulation.
   5.273 +   - =(set-accuracy world accuracy)= :: change the accuracy of the
   5.274 +        world's physics simulator.
   5.275 +   - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine
   5.276 +        construct that is useful for loading textures and is required
   5.277 +        for smooth interaction with jMonkeyEngine library functions.
   5.278 +	
   5.279 +
   5.280 +*** Creature Manipulation / Import
   5.281 +
   5.282 +   - =(body! creature)= :: give the creature a physical body.
   5.283 +   - =(vision! creature)= :: give the creature a sense of vision.
   5.284 +        Returns a list of functions which will each, when called
   5.285 +        during a simulation, return the vision data for the channel of
   5.286 +        one of the eyes. The functions are ordered depending on the
   5.287 +        alphabetical order of the names of the eye nodes in the
   5.288 +        blender file. The data returned by the functions is a vector
   5.289 +        containing the eye's /topology/, a vector of coordinates, and
   5.290 +        the eye's /data/, a vector of RGB values filtered by the eye's
   5.291 +        sensitivity. 
   5.292 +   - =(hearing! creature)= :: give the creature a sense of hearing.
   5.293 +        Returns a list of functions, one for each ear, that when
   5.294 +        called will return a frame's worth of hearing data for that
   5.295 +        ear. The functions are ordered depending on the alphabetical
   5.296 +        order of the names of the ear nodes in the blender file. The
   5.297 +        data returned by the functions is an array PCM encoded wav
   5.298 +        data. 
   5.299 +   - =(touch! creature)= :: give the creature a sense of touch. Returns
   5.300 +        a single function that must be called with the /root node/ of
   5.301 +        the world, and which will return a vector of /touch-data/
   5.302 +        one entry for each touch sensitive component, each entry of
   5.303 +        which contains a /topology/ that specifies the distribution of
   5.304 +        touch sensors, and the /data/, which is a vector of
   5.305 +        =[activation, length]= pairs for each touch hair.
   5.306 +   - =(proprioception! creature)= :: give the creature the sense of
   5.307 +        proprioception. Returns a list of functions, one for each
   5.308 +        joint, that when called during a running simulation will
   5.309 +        report the =[headnig, pitch, roll]= of the joint.
   5.310 +   - =(movement! creature)= :: give the creature the power of movement.
   5.311 +        Creates a list of functions, one for each muscle, that when
   5.312 +        called with an integer, will set the recruitment of that
   5.313 +        muscle to that integer, and will report the current power
   5.314 +        being exerted by the muscle. Order of muscles is determined by
   5.315 +        the alphabetical sort order of the names of the muscle nodes.
   5.316 +	
   5.317 +*** Visualization/Debug
   5.318 +
   5.319 +   - =(view-vision)= :: create a function that when called with a list
   5.320 +        of visual data returned from the functions made by =vision!=, 
   5.321 +        will display that visual data on the screen.
   5.322 +   - =(view-hearing)= :: same as =view-vision= but for hearing.
   5.323 +   - =(view-touch)= :: same as =view-vision= but for touch.
   5.324 +   - =(view-proprioception)= :: same as =view-vision= but for
   5.325 +        proprioception.
   5.326 +   - =(view-movement)= :: same as =view-vision= but for
   5.327 +        proprioception.
   5.328 +   - =(view anything)= :: =view= is a polymorphic function that allows
   5.329 +        you to inspect almost anything you could reasonably expect to
   5.330 +        be able to ``see'' in =CORTEX=.
   5.331 +   - =(text anything)= :: =text= is a polymorphic function that allows
   5.332 +        you to convert practically anything into a text string.	
   5.333 +   - =(println-repl anything)= :: print messages to clojure's repl
   5.334 +        instead of the simulation's terminal window.
   5.335 +
   5.336 +*** Misc
   5.337 +
   5.338 +   - =(load-bullet)=   :: unpack native libraries and initialize
   5.339 +        blender. This function is required before other world building
   5.340 +        functions are called.
   5.341 +   - =(mega-import-jme3)= :: for experimenting at the REPL. This
   5.342 +        function will import all jMonkeyEngine3 classes for immediate
   5.343 +        use.