* Appendix: =CORTEX= User Guide

 

   Those who write a thesis should endeavor to make their code not only

   accessable, but actually useable, as a way to pay back the community

   that made the thesis possible in the first place. This thesis would

   not be possible without Free Software such as jMonkeyEngine3,

   Blender, clojure, emacs, ffmpeg, and many other tools. That is why I

   have included this user guide, in the hope that someone else might

   find =CORTEX= useful.

 

 ** Obtaining =CORTEX= 

 

    You can get cortex from its mercurial repository at

    http://hg.bortreb.com/cortex. You may also download =CORTEX=

    releases at http://aurellem.org/cortex/releases/. As a condition of

    making this thesis, I have also provided Professor Winston the

    =CORTEX= source, and he knows how to run the demos and get started.

    You may also email me at =cortex@aurellem.org= and I may help where

    I can.

 

 ** Running =CORTEX= 

    

    =CORTEX= comes with README and INSTALL files that will guide you

    through installation and running the test suite. In particular you

    should look at test =cortex.test= which contains test suites that

    run through all senses and multiple creatures.

 

 ** Creating creatures

 

    Creatures are created using /Blender/, a free 3D modeling program.

    You will need Blender version 2.6 when using the =CORTEX= included

    in this thesis. You create a =CORTEX= creature in a similiar manner

    to modeling anything in Blender, except that you also create

    several trees of empty nodes which define the creature's senses.

 

 *** Mass 

     

     To give an object mass in =CORTEX=, add a ``mass'' metadata label

     to the object with the mass in jMonkeyEngine units. Note that

     setting the mass to 0 causes the object to be immovable.

 

 *** Joints

 

     Joints are created by creating an empty node named =joints= and

     then creating any number of empty child nodes to represent your

     creature's joints. The joint will automatically connect the

     closest two physical objects. It will help to set the empty node's

     display mode to ``Arrows'' so that you can clearly see the

     direction of the axes.

    

     Joint nodes should have the following metadata under the ``joint''

     label:

 

     #+BEGIN_SRC clojure

 ;; ONE OF the following, under the label "joint":

 {:type :point}

 

 ;; OR

 

 {:type :hinge

  :limit [<limit-low> <limit-high>]

  :axis (Vector3f. <x> <y> <z>)}

 ;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)

 

 ;; OR

 

 {:type :cone

  :limit-xz <lim-xz>

  :limit-xy <lim-xy>

  :twist    <lim-twist>}   ;(use XZY rotation mode in blender!)

     #+END_SRC

 

 *** Eyes

 

     Eyes are created by creating an empty node named =eyes= and then

     creating any number of empty child nodes to represent your

     creature's eyes.

 

     Eye nodes should have the following metadata under the ``eye''

     label:

 

 #+BEGIN_SRC clojure

 {:red    <red-retina-definition>

  :blue   <blue-retina-definition>

  :green  <green-retina-definition>

  :all    <all-retina-definition>

  (<0xrrggbb> <custom-retina-image>)...

 }

 #+END_SRC

 

     Any of the color channels may be omitted. You may also include

     your own color selectors, and in fact :red is equivalent to

     0xFF0000 and so forth. The eye will be placed at the same position

     as the empty node and will bind to the neatest physical object.

     The eye will point outward from the X-axis of the node, and ``up''

     will be in the direction of the X-axis of the node. It will help

     to set the empty node's display mode to ``Arrows'' so that you can

     clearly see the direction of the axes.

 

     Each retina file should contain white pixels whever you want to be

     sensitive to your chosen color. If you want the entire field of

     view, specify :all of 0xFFFFFF and a retinal map that is entirely

     white. 

 

     Here is a sample retinal map:

 

     #+caption: An example retinal profile image. White pixels are 

     #+caption: photo-sensitive elements. The distribution of white 

     #+caption: pixels is denser in the middle and falls off at the 

     #+caption: edges and is inspired by the human retina.

     #+name: retina

     #+ATTR_LaTeX: :width 7cm :placement [H]

     [[./images/retina-small.png]]

 

 *** Hearing

 

     Ears are created by creating an empty node named =ears= and then

     creating any number of empty child nodes to represent your

     creature's ears. 

 

     Ear nodes do not require any metadata.

 

     The ear will bind to and follow the closest physical node.

 

 *** Touch

 

     Touch is handled similarly to mass. To make a particular object

     touch sensitive, add metadata of the following form under the

     object's ``touch'' metadata field:

     

     #+BEGIN_EXAMPLE

     <touch-UV-map-file-name>    

     #+END_EXAMPLE

 

     You may also include an optional ``scale'' metadata number to

     specifiy the length of the touch feelers. The default is $0.1$,

     and this is generally sufficient.

 

     The touch UV should contain white pixels for each touch sensor.

 

     Here is an example touch-uv map that approximates a human finger,

     and its corresponding model.

 

     #+caption: This is the tactile-sensor-profile for the upper segment 

     #+caption: of a fingertip. It defines regions of high touch sensitivity 

     #+caption: (where there are many white pixels) and regions of low 

     #+caption: sensitivity (where white pixels are sparse).

     #+name: guide-fingertip-UV

     #+ATTR_LaTeX: :width 9cm :placement [H]

     [[./images/finger-UV.png]]

 

     #+caption: The fingertip UV-image form above applied to a simple

     #+caption: model of a fingertip.

     #+name: guide-fingertip

     #+ATTR_LaTeX: :width 9cm :placement [H]

     [[./images/finger-2.png]]

 

 *** Propriocepotion

 

     Proprioception is tied to each joint node -- nothing special must

     be done in a blender model to enable proprioception other than

     creating joint nodes.

 

 *** Muscles

 

     Muscles are created by creating an empty node named =muscles= and

     then creating any number of empty child nodes to represent your

     creature's muscles.

 

     

     Muscle nodes should have the following metadata under the

     ``muscle'' label:

 

     #+BEGIN_EXAMPLE

     <muscle-profile-file-name>

     #+END_EXAMPLE

 

     Muscles should also have a ``strength'' metadata entry describing

     the muscle's total strength at full activation. 

 

     Muscle profiles are simple images that contain the relative amount

     of muscle power in each simulated alpha motor neuron. The width of

     the image is the total size of the motor pool, and the redness of

     each neuron is the relative power of that motor pool.

 

     While the profile image can have any dimensions, only the first

     line of pixels is used to define the muscle. Here is a sample

     muscle profile image that defines a human-like muscle.

 

     #+caption: A muscle profile image that describes the strengths

     #+caption: of each motor neuron in a muscle. White is weakest 

     #+caption: and dark red is strongest. This particular pattern 

     #+caption: has weaker motor neurons at the beginning, just 

     #+caption: like human muscle.

     #+name: muscle-recruit

     #+ATTR_LaTeX: :width 7cm :placement [H]

     [[./images/basic-muscle.png]]

     

     Muscles twist the nearest physical object about the muscle node's

     Z-axis. I recommend using the ``Single Arrow'' display mode for

     muscles and using the right hand rule to determine which way the

     muscle will twist. To make a segment that can twist in multiple

     directions, create multiple, differently aligned muscles.

 

 ** =CORTEX= API

 

    These are the some functions exposed by =CORTEX= for creating

    worlds and simulating creatures. These are in addition to

    jMonkeyEngine3's extensive library, which is documented elsewhere.

 

 *** Simulation

    - =(world root-node key-map setup-fn update-fn)= :: create

         a simulation.

      - /root-node/     :: a =com.jme3.scene.Node= object which

           contains all of the objects that should be in the

           simulation.

 

      - /key-map/       :: a map from strings describing keys to

           functions that should be executed whenever that key is

           pressed. the functions should take a SimpleApplication

           object and a boolean value. The SimpleApplication is the

           current simulation that is running, and the boolean is true

           if the key is being pressed, and false if it is being

           released. As an example,

           #+BEGIN_SRC clojure

        {"key-j" (fn [game value] (if value (println "key j pressed")))}		  

 	  #+END_SRC

 	  is a valid key-map which will cause the simulation to print

           a message whenever the 'j' key on the keyboard is pressed.

 

      - /setup-fn/      :: a function that takes a =SimpleApplication=

           object. It is called once when initializing the simulation.

           Use it to create things like lights, change the gravity,

           initialize debug nodes, etc.

 

      - /update-fn/     :: this function takes a =SimpleApplication=

           object and a float and is called every frame of the

           simulation. The float tells how many seconds is has been

           since the last frame was rendered, according to whatever

           clock jme is currently using. The default is to use IsoTimer

           which will result in this value always being the same.

  

 

    - =(position-camera world position rotation)= :: set the position

         of the simulation's main camera.

    - =(enable-debug world)= :: turn on debug wireframes for each

         simulated object.

    - =(set-gravity world gravity)= :: set the gravity of a running

         simulation.

    - =(box length width height & {options})= :: create a box in the

         simulation. Options is a hash map specifying texture, mass,

         etc. Possible options are =:name=, =:color=, =:mass=,

         =:friction=, =:texture=, =:material=, =:position=,

         =:rotation=, =:shape=, and =:physical?=.

    - =(sphere radius & {options})= :: create a sphere in the simulation.

         Options are the same as in =box=.

    - =(load-blender-model file-name)= :: create a node structure

         representing that described in a blender file.

    - =(light-up-everything world)= :: distribute a standard compliment

         of lights throught the simulation. Should be adequate for most

         purposes.

    - =(node-seq node)= :: return a recursuve list of the node's

         children.

    - =(nodify name children)= :: construct a node given a node-name and

         desired children.

    - =(add-element world element)= :: add an object to a running world

         simulation.

    - =(set-accuracy world accuracy)= :: change the accuracy of the

         world's physics simulator.

    - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine

         construct that is useful for loading textures and is required

         for smooth interaction with jMonkeyEngine library functions.

 	

 

 *** Creature Manipulation / Import

 

    - =(body! creature)= :: give the creature a physical body.

    - =(vision! creature)= :: give the creature a sense of vision.

         Returns a list of functions which will each, when called

         during a simulation, return the vision data for the channel of

         one of the eyes. The functions are ordered depending on the

         alphabetical order of the names of the eye nodes in the

         blender file. The data returned by the functions is a vector

         containing the eye's /topology/, a vector of coordinates, and

         the eye's /data/, a vector of RGB values filtered by the eye's

         sensitivity. 

    - =(hearing! creature)= :: give the creature a sense of hearing.

         Returns a list of functions, one for each ear, that when

         called will return a frame's worth of hearing data for that

         ear. The functions are ordered depending on the alphabetical

         order of the names of the ear nodes in the blender file. The

         data returned by the functions is an array PCM encoded wav

         data. 

    - =(touch! creature)= :: give the creature a sense of touch. Returns

         a single function that must be called with the /root node/ of

         the world, and which will return a vector of /touch-data/

         one entry for each touch sensitive component, each entry of

         which contains a /topology/ that specifies the distribution of

         touch sensors, and the /data/, which is a vector of

         =[activation, length]= pairs for each touch hair.

    - =(proprioception! creature)= :: give the creature the sense of

         proprioception. Returns a list of functions, one for each

         joint, that when called during a running simulation will

         report the =[headnig, pitch, roll]= of the joint.

    - =(movement! creature)= :: give the creature the power of movement.

         Creates a list of functions, one for each muscle, that when

         called with an integer, will set the recruitment of that

         muscle to that integer, and will report the current power

         being exerted by the muscle. Order of muscles is determined by

         the alphabetical sort order of the names of the muscle nodes.

 	

 *** Visualization/Debug

 

    - =(view-vision)= :: create a function that when called with a list

         of visual data returned from the functions made by =vision!=, 

         will display that visual data on the screen.

    - =(view-hearing)= :: same as =view-vision= but for hearing.

    - =(view-touch)= :: same as =view-vision= but for touch.

    - =(view-proprioception)= :: same as =view-vision= but for

         proprioception.

    - =(view-movement)= :: same as =view-vision= but for

         proprioception.

    - =(view anything)= :: =view= is a polymorphic function that allows

         you to inspect almost anything you could reasonably expect to

         be able to ``see'' in =CORTEX=.

    - =(text anything)= :: =text= is a polymorphic function that allows

         you to convert practically anything into a text string.	

    - =(println-repl anything)= :: print messages to clojure's repl

         instead of the simulation's terminal window.

 

 *** Misc

 

    - =(load-bullet)=   :: unpack native libraries and initialize

         blender. This function is required before other world building

         functions are called.

    - =(mega-import-jme3)= :: for experimenting at the REPL. This

         function will import all jMonkeyEngine3 classes for immediate

         use.