1.1 --- a/thesis/cortex.org	Sat Mar 29 20:17:58 2014 -0400
     1.2 +++ b/thesis/cortex.org	Sat Mar 29 20:33:35 2014 -0400
     1.3 @@ -2325,7 +2325,7 @@
     1.4      is currently being employed. This is analogous to muscle tension
     1.5      in humans and completes the sense of proprioception begun in the
     1.6      last section.
     1.7 -
     1.8 +    
     1.9  ** COMMENT =CORTEX= brings complex creatures to life!
    1.10     
    1.11     The ultimate test of =CORTEX= is to create a creature with the full
    1.12 @@ -3246,3 +3246,371 @@
    1.13  # - Skeletal imitation
    1.14  # - Sensory fleshing-out
    1.15  # - Classification
    1.16 +#+BEGIN_LaTeX
    1.17 +\appendix
    1.18 +#+END_LaTeX
    1.19 +* COMMENT Appendix: =CORTEX= User Guide
    1.20 +
    1.21 +  Those who write a thesis should endeavor to make their code not only
    1.22 +  accessable, but actually useable, as a way to pay back the community
    1.23 +  that made the thesis possible in the first place. This thesis would
    1.24 +  not be possible without Free Software such as jMonkeyEngine3,
    1.25 +  Blender, clojure, emacs, ffmpeg, and many other tools. That is why I
    1.26 +  have included this user guide, in the hope that someone else might
    1.27 +  find =CORTEX= useful.
    1.28 +
    1.29 +** Obtaining =CORTEX= 
    1.30 +
    1.31 +   You can get cortex from its mercurial repository at
    1.32 +   http://hg.bortreb.com/cortex. You may also download =CORTEX=
    1.33 +   releases at http://aurellem.org/cortex/releases/. As a condition of
    1.34 +   making this thesis, I have also provided Professor Winston the
    1.35 +   =CORTEX= source, and he knows how to run the demos and get started.
    1.36 +   You may also email me at =cortex@aurellem.org= and I may help where
    1.37 +   I can.
    1.38 +
    1.39 +** Running =CORTEX= 
    1.40 +   
    1.41 +   =CORTEX= comes with README and INSTALL files that will guide you
    1.42 +   through installation and running the test suite. In particular you
    1.43 +   should look at test =cortex.test= which contains test suites that
    1.44 +   run through all senses and multiple creatures.
    1.45 +
    1.46 +** Creating creatures
    1.47 +
    1.48 +   Creatures are created using /Blender/, a free 3D modeling program.
    1.49 +   You will need Blender version 2.6 when using the =CORTEX= included
    1.50 +   in this thesis. You create a =CORTEX= creature in a similiar manner
    1.51 +   to modeling anything in Blender, except that you also create
    1.52 +   several trees of empty nodes which define the creature's senses.
    1.53 +
    1.54 +*** Mass 
    1.55 +    
    1.56 +    To give an object mass in =CORTEX=, add a ``mass'' metadata label
    1.57 +    to the object with the mass in jMonkeyEngine units. Note that
    1.58 +    setting the mass to 0 causes the object to be immovable.
    1.59 +
    1.60 +*** Joints
    1.61 +
    1.62 +    Joints are created by creating an empty node named =joints= and
    1.63 +    then creating any number of empty child nodes to represent your
    1.64 +    creature's joints. The joint will automatically connect the
    1.65 +    closest two physical objects. It will help to set the empty node's
    1.66 +    display mode to ``Arrows'' so that you can clearly see the
    1.67 +    direction of the axes.
    1.68 +   
    1.69 +    Joint nodes should have the following metadata under the ``joint''
    1.70 +    label:
    1.71 +
    1.72 +    #+BEGIN_SRC clojure
    1.73 +;; ONE OF the following, under the label "joint":
    1.74 +{:type :point}
    1.75 +
    1.76 +;; OR
    1.77 +
    1.78 +{:type :hinge
    1.79 + :limit [<limit-low> <limit-high>]
    1.80 + :axis (Vector3f. <x> <y> <z>)}
    1.81 +;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
    1.82 +
    1.83 +;; OR
    1.84 +
    1.85 +{:type :cone
    1.86 + :limit-xz <lim-xz>
    1.87 + :limit-xy <lim-xy>
    1.88 + :twist    <lim-twist>}   ;(use XZY rotation mode in blender!)
    1.89 +    #+END_SRC
    1.90 +
    1.91 +*** Eyes
    1.92 +
    1.93 +    Eyes are created by creating an empty node named =eyes= and then
    1.94 +    creating any number of empty child nodes to represent your
    1.95 +    creature's eyes.
    1.96 +
    1.97 +    Eye nodes should have the following metadata under the ``eye''
    1.98 +    label:
    1.99 +
   1.100 +#+BEGIN_SRC clojure
   1.101 +{:red    <red-retina-definition>
   1.102 + :blue   <blue-retina-definition>
   1.103 + :green  <green-retina-definition>
   1.104 + :all    <all-retina-definition>
   1.105 + (<0xrrggbb> <custom-retina-image>)...
   1.106 +}
   1.107 +#+END_SRC
   1.108 +
   1.109 +    Any of the color channels may be omitted. You may also include
   1.110 +    your own color selectors, and in fact :red is equivalent to
   1.111 +    0xFF0000 and so forth. The eye will be placed at the same position
   1.112 +    as the empty node and will bind to the neatest physical object.
   1.113 +    The eye will point outward from the X-axis of the node, and ``up''
   1.114 +    will be in the direction of the X-axis of the node. It will help
   1.115 +    to set the empty node's display mode to ``Arrows'' so that you can
   1.116 +    clearly see the direction of the axes.
   1.117 +
   1.118 +    Each retina file should contain white pixels whever you want to be
   1.119 +    sensitive to your chosen color. If you want the entire field of
   1.120 +    view, specify :all of 0xFFFFFF and a retinal map that is entirely
   1.121 +    white. 
   1.122 +
   1.123 +    Here is a sample retinal map:
   1.124 +
   1.125 +    #+caption: An example retinal profile image. White pixels are 
   1.126 +    #+caption: photo-sensitive elements. The distribution of white 
   1.127 +    #+caption: pixels is denser in the middle and falls off at the 
   1.128 +    #+caption: edges and is inspired by the human retina.
   1.129 +    #+name: retina
   1.130 +    #+ATTR_LaTeX: :width 7cm :placement [H]
   1.131 +    [[./images/retina-small.png]]
   1.132 +
   1.133 +*** Hearing
   1.134 +
   1.135 +    Ears are created by creating an empty node named =ears= and then
   1.136 +    creating any number of empty child nodes to represent your
   1.137 +    creature's ears. 
   1.138 +
   1.139 +    Ear nodes do not require any metadata.
   1.140 +
   1.141 +    The ear will bind to and follow the closest physical node.
   1.142 +
   1.143 +*** Touch
   1.144 +
   1.145 +    Touch is handled similarly to mass. To make a particular object
   1.146 +    touch sensitive, add metadata of the following form under the
   1.147 +    object's ``touch'' metadata field:
   1.148 +    
   1.149 +    #+BEGIN_EXAMPLE
   1.150 +    <touch-UV-map-file-name>    
   1.151 +    #+END_EXAMPLE
   1.152 +
   1.153 +    You may also include an optional ``scale'' metadata number to
   1.154 +    specifiy the length of the touch feelers. The default is $0.1$,
   1.155 +    and this is generally sufficient.
   1.156 +
   1.157 +    The touch UV should contain white pixels for each touch sensor.
   1.158 +
   1.159 +    Here is an example touch-uv map that approximates a human finger,
   1.160 +    and its corresponding model.
   1.161 +
   1.162 +    #+caption: This is the tactile-sensor-profile for the upper segment 
   1.163 +    #+caption: of a fingertip. It defines regions of high touch sensitivity 
   1.164 +    #+caption: (where there are many white pixels) and regions of low 
   1.165 +    #+caption: sensitivity (where white pixels are sparse).
   1.166 +    #+name: guide-fingertip-UV
   1.167 +    #+ATTR_LaTeX: :width 9cm :placement [H]
   1.168 +    [[./images/finger-UV.png]]
   1.169 +
   1.170 +    #+caption: The fingertip UV-image form above applied to a simple
   1.171 +    #+caption: model of a fingertip.
   1.172 +    #+name: guide-fingertip
   1.173 +    #+ATTR_LaTeX: :width 9cm :placement [H]
   1.174 +    [[./images/finger-2.png]]
   1.175 +
   1.176 +*** Propriocepotion
   1.177 +
   1.178 +    Proprioception is tied to each joint node -- nothing special must
   1.179 +    be done in a blender model to enable proprioception other than
   1.180 +    creating joint nodes.
   1.181 +
   1.182 +*** Muscles
   1.183 +
   1.184 +    Muscles are created by creating an empty node named =muscles= and
   1.185 +    then creating any number of empty child nodes to represent your
   1.186 +    creature's muscles.
   1.187 +
   1.188 +    
   1.189 +    Muscle nodes should have the following metadata under the
   1.190 +    ``muscle'' label:
   1.191 +
   1.192 +    #+BEGIN_EXAMPLE
   1.193 +    <muscle-profile-file-name>
   1.194 +    #+END_EXAMPLE
   1.195 +
   1.196 +    Muscles should also have a ``strength'' metadata entry describing
   1.197 +    the muscle's total strength at full activation. 
   1.198 +
   1.199 +    Muscle profiles are simple images that contain the relative amount
   1.200 +    of muscle power in each simulated alpha motor neuron. The width of
   1.201 +    the image is the total size of the motor pool, and the redness of
   1.202 +    each neuron is the relative power of that motor pool.
   1.203 +
   1.204 +    While the profile image can have any dimensions, only the first
   1.205 +    line of pixels is used to define the muscle. Here is a sample
   1.206 +    muscle profile image that defines a human-like muscle.
   1.207 +
   1.208 +    #+caption: A muscle profile image that describes the strengths
   1.209 +    #+caption: of each motor neuron in a muscle. White is weakest 
   1.210 +    #+caption: and dark red is strongest. This particular pattern 
   1.211 +    #+caption: has weaker motor neurons at the beginning, just 
   1.212 +    #+caption: like human muscle.
   1.213 +    #+name: muscle-recruit
   1.214 +    #+ATTR_LaTeX: :width 7cm :placement [H]
   1.215 +    [[./images/basic-muscle.png]]
   1.216 +    
   1.217 +    Muscles twist the nearest physical object about the muscle node's
   1.218 +    Z-axis. I recommend using the ``Single Arrow'' display mode for
   1.219 +    muscles and using the right hand rule to determine which way the
   1.220 +    muscle will twist. To make a segment that can twist in multiple
   1.221 +    directions, create multiple, differently aligned muscles.
   1.222 +
   1.223 +** =CORTEX= API
   1.224 +
   1.225 +   These are the some functions exposed by =CORTEX= for creating
   1.226 +   worlds and simulating creatures. These are in addition to
   1.227 +   jMonkeyEngine3's extensive library, which is documented elsewhere.
   1.228 +
   1.229 +*** Simulation
   1.230 +   - =(world root-node key-map setup-fn update-fn)= :: create
   1.231 +        a simulation.
   1.232 +     - /root-node/     :: a =com.jme3.scene.Node= object which
   1.233 +          contains all of the objects that should be in the
   1.234 +          simulation.
   1.235 +
   1.236 +     - /key-map/       :: a map from strings describing keys to
   1.237 +          functions that should be executed whenever that key is
   1.238 +          pressed. the functions should take a SimpleApplication
   1.239 +          object and a boolean value. The SimpleApplication is the
   1.240 +          current simulation that is running, and the boolean is true
   1.241 +          if the key is being pressed, and false if it is being
   1.242 +          released. As an example,
   1.243 +          #+BEGIN_SRC clojure
   1.244 +       {"key-j" (fn [game value] (if value (println "key j pressed")))}		  
   1.245 +	  #+END_SRC
   1.246 +	  is a valid key-map which will cause the simulation to print
   1.247 +          a message whenever the 'j' key on the keyboard is pressed.
   1.248 +
   1.249 +     - /setup-fn/      :: a function that takes a =SimpleApplication=
   1.250 +          object. It is called once when initializing the simulation.
   1.251 +          Use it to create things like lights, change the gravity,
   1.252 +          initialize debug nodes, etc.
   1.253 +
   1.254 +     - /update-fn/     :: this function takes a =SimpleApplication=
   1.255 +          object and a float and is called every frame of the
   1.256 +          simulation. The float tells how many seconds is has been
   1.257 +          since the last frame was rendered, according to whatever
   1.258 +          clock jme is currently using. The default is to use IsoTimer
   1.259 +          which will result in this value always being the same.
   1.260 +
   1.261 +   - =(position-camera world position rotation)= :: set the position
   1.262 +        of the simulation's main camera.
   1.263 +
   1.264 +   - =(enable-debug world)= :: turn on debug wireframes for each
   1.265 +        simulated object.
   1.266 +
   1.267 +   - =(set-gravity world gravity)= :: set the gravity of a running
   1.268 +        simulation.
   1.269 +
   1.270 +   - =(box length width height & {options})= :: create a box in the
   1.271 +        simulation. Options is a hash map specifying texture, mass,
   1.272 +        etc. Possible options are =:name=, =:color=, =:mass=,
   1.273 +        =:friction=, =:texture=, =:material=, =:position=,
   1.274 +        =:rotation=, =:shape=, and =:physical?=.
   1.275 +
   1.276 +   - =(sphere radius & {options})= :: create a sphere in the simulation.
   1.277 +        Options are the same as in =box=.
   1.278 +
   1.279 +   - =(load-blender-model file-name)= :: create a node structure
   1.280 +        representing that described in a blender file.
   1.281 +
   1.282 +   - =(light-up-everything world)= :: distribute a standard compliment
   1.283 +        of lights throught the simulation. Should be adequate for most
   1.284 +        purposes.
   1.285 +
   1.286 +   - =(node-seq node)= :: return a recursuve list of the node's
   1.287 +        children.
   1.288 +
   1.289 +   - =(nodify name children)= :: construct a node given a node-name and
   1.290 +        desired children.
   1.291 +
   1.292 +   - =(add-element world element)= :: add an object to a running world
   1.293 +        simulation.
   1.294 +
   1.295 +   - =(set-accuracy world accuracy)= :: change the accuracy of the
   1.296 +        world's physics simulator.
   1.297 +
   1.298 +   - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine
   1.299 +        construct that is useful for loading textures and is required
   1.300 +        for smooth interaction with jMonkeyEngine library functions.
   1.301 +
   1.302 +   - =(load-bullet)=   :: unpack native libraries and initialize
   1.303 +        blender. This function is required before other world building
   1.304 +        functions are called.
   1.305 +	
   1.306 +
   1.307 +*** Creature Manipulation / Import
   1.308 +
   1.309 +   - =(body! creature)= :: give the creature a physical body.
   1.310 +
   1.311 +   - =(vision! creature)= :: give the creature a sense of vision.
   1.312 +        Returns a list of functions which will each, when called
   1.313 +        during a simulation, return the vision data for the channel of
   1.314 +        one of the eyes. The functions are ordered depending on the
   1.315 +        alphabetical order of the names of the eye nodes in the
   1.316 +        blender file. The data returned by the functions is a vector
   1.317 +        containing the eye's /topology/, a vector of coordinates, and
   1.318 +        the eye's /data/, a vector of RGB values filtered by the eye's
   1.319 +        sensitivity. 
   1.320 +
   1.321 +   - =(hearing! creature)= :: give the creature a sense of hearing.
   1.322 +        Returns a list of functions, one for each ear, that when
   1.323 +        called will return a frame's worth of hearing data for that
   1.324 +        ear. The functions are ordered depending on the alphabetical
   1.325 +        order of the names of the ear nodes in the blender file. The
   1.326 +        data returned by the functions is an array PCM encoded wav
   1.327 +        data. 
   1.328 +
   1.329 +   - =(touch! creature)= :: give the creature a sense of touch. Returns
   1.330 +        a single function that must be called with the /root node/ of
   1.331 +        the world, and which will return a vector of /touch-data/
   1.332 +        one entry for each touch sensitive component, each entry of
   1.333 +        which contains a /topology/ that specifies the distribution of
   1.334 +        touch sensors, and the /data/, which is a vector of
   1.335 +        =[activation, length]= pairs for each touch hair.
   1.336 +
   1.337 +   - =(proprioception! creature)= :: give the creature the sense of
   1.338 +        proprioception. Returns a list of functions, one for each
   1.339 +        joint, that when called during a running simulation will
   1.340 +        report the =[headnig, pitch, roll]= of the joint.
   1.341 +
   1.342 +   - =(movement! creature)= :: give the creature the power of movement.
   1.343 +        Creates a list of functions, one for each muscle, that when
   1.344 +        called with an integer, will set the recruitment of that
   1.345 +        muscle to that integer, and will report the current power
   1.346 +        being exerted by the muscle. Order of muscles is determined by
   1.347 +        the alphabetical sort order of the names of the muscle nodes.
   1.348 +	
   1.349 +*** Visualization/Debug
   1.350 +
   1.351 +   - =(view-vision)= :: create a function that when called with a list
   1.352 +        of visual data returned from the functions made by =vision!=, 
   1.353 +        will display that visual data on the screen.
   1.354 +
   1.355 +   - =(view-hearing)= :: same as =view-vision= but for hearing.
   1.356 +
   1.357 +   - =(view-touch)= :: same as =view-vision= but for touch.
   1.358 +
   1.359 +   - =(view-proprioception)= :: same as =view-vision= but for
   1.360 +        proprioception.
   1.361 +
   1.362 +   - =(view-movement)= :: same as =view-vision= but for
   1.363 +        proprioception.
   1.364 +
   1.365 +   - =(view anything)= :: =view= is a polymorphic function that allows
   1.366 +        you to inspect almost anything you could reasonably expect to
   1.367 +        be able to ``see'' in =CORTEX=.
   1.368 +
   1.369 +   - =(text anything)= :: =text= is a polymorphic function that allows
   1.370 +        you to convert practically anything into a text string.	
   1.371 +
   1.372 +   - =(println-repl anything)= :: print messages to clojure's repl
   1.373 +        instead of the simulation's terminal window.
   1.374 +
   1.375 +   - =(mega-import-jme3)= :: for experimenting at the REPL. This
   1.376 +        function will import all jMonkeyEngine3 classes for immediate
   1.377 +        use.
   1.378 +
   1.379 +   - =(display-dialated-time world timer)= :: Shows the time as it is
   1.380 +        flowing in the simulation on a HUD display.
   1.381 +
   1.382 +
   1.383 +