rlm@437: * Appendix: =CORTEX= User Guide rlm@437: rlm@486: Those who write a thesis should endeavor to make their code not only rlm@486: accessable, but actually useable, as a way to pay back the community rlm@486: that made the thesis possible in the first place. This thesis would rlm@486: not be possible without Free Software such as jMonkeyEngine3, rlm@486: Blender, clojure, emacs, ffmpeg, and many other tools. That is why I rlm@486: have included this user guide, in the hope that someone else might rlm@486: find =CORTEX= useful. rlm@437: rlm@486: ** Obtaining =CORTEX= rlm@437: rlm@486: You can get cortex from its mercurial repository at rlm@486: http://hg.bortreb.com/cortex. You may also download =CORTEX= rlm@486: releases at http://aurellem.org/cortex/releases/. As a condition of rlm@486: making this thesis, I have also provided Professor Winston the rlm@486: =CORTEX= source, and he knows how to run the demos and get started. rlm@486: You may also email me at =cortex@aurellem.org= and I may help where rlm@486: I can. rlm@486: rlm@486: ** Running =CORTEX= rlm@486: rlm@486: =CORTEX= comes with README and INSTALL files that will guide you rlm@486: through installation and running the test suite. In particular you rlm@486: should look at test =cortex.test= which contains test suites that rlm@486: run through all senses and multiple creatures. rlm@486: rlm@486: ** Creating creatures rlm@486: rlm@486: Creatures are created using /Blender/, a free 3D modeling program. rlm@486: You will need Blender version 2.6 when using the =CORTEX= included rlm@486: in this thesis. You create a =CORTEX= creature in a similiar manner rlm@486: to modeling anything in Blender, except that you also create rlm@486: several trees of empty nodes which define the creature's senses. rlm@486: rlm@486: *** Mass rlm@486: rlm@486: To give an object mass in =CORTEX=, add a ``mass'' metadata label rlm@486: to the object with the mass in jMonkeyEngine units. Note that rlm@486: setting the mass to 0 causes the object to be immovable. rlm@486: rlm@486: *** Joints rlm@486: rlm@486: Joints are created by creating an empty node named =joints= and rlm@486: then creating any number of empty child nodes to represent your rlm@486: creature's joints. The joint will automatically connect the rlm@486: closest two physical objects. It will help to set the empty node's rlm@486: display mode to ``Arrows'' so that you can clearly see the rlm@486: direction of the axes. rlm@486: rlm@486: Joint nodes should have the following metadata under the ``joint'' rlm@486: label: rlm@486: rlm@486: #+BEGIN_SRC clojure rlm@486: ;; ONE OF the following, under the label "joint": rlm@486: {:type :point} rlm@486: rlm@486: ;; OR rlm@486: rlm@486: {:type :hinge rlm@486: :limit [ ] rlm@486: :axis (Vector3f. )} rlm@486: ;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints) rlm@486: rlm@486: ;; OR rlm@486: rlm@486: {:type :cone rlm@486: :limit-xz rlm@486: :limit-xy rlm@486: :twist } ;(use XZY rotation mode in blender!) rlm@486: #+END_SRC rlm@486: rlm@486: *** Eyes rlm@486: rlm@486: Eyes are created by creating an empty node named =eyes= and then rlm@486: creating any number of empty child nodes to represent your rlm@486: creature's eyes. rlm@486: rlm@486: Eye nodes should have the following metadata under the ``eye'' rlm@486: label: rlm@486: rlm@486: #+BEGIN_SRC clojure rlm@486: {:red rlm@486: :blue rlm@486: :green rlm@486: :all rlm@486: (<0xrrggbb> )... rlm@486: } rlm@486: #+END_SRC rlm@486: rlm@486: Any of the color channels may be omitted. You may also include rlm@486: your own color selectors, and in fact :red is equivalent to rlm@486: 0xFF0000 and so forth. The eye will be placed at the same position rlm@486: as the empty node and will bind to the neatest physical object. rlm@486: The eye will point outward from the X-axis of the node, and ``up'' rlm@486: will be in the direction of the X-axis of the node. It will help rlm@486: to set the empty node's display mode to ``Arrows'' so that you can rlm@486: clearly see the direction of the axes. rlm@486: rlm@486: Each retina file should contain white pixels whever you want to be rlm@486: sensitive to your chosen color. If you want the entire field of rlm@486: view, specify :all of 0xFFFFFF and a retinal map that is entirely rlm@486: white. rlm@486: rlm@486: Here is a sample retinal map: rlm@486: rlm@486: #+caption: An example retinal profile image. White pixels are rlm@486: #+caption: photo-sensitive elements. The distribution of white rlm@486: #+caption: pixels is denser in the middle and falls off at the rlm@486: #+caption: edges and is inspired by the human retina. rlm@486: #+name: retina rlm@486: #+ATTR_LaTeX: :width 7cm :placement [H] rlm@486: [[./images/retina-small.png]] rlm@486: rlm@486: *** Hearing rlm@486: rlm@486: Ears are created by creating an empty node named =ears= and then rlm@486: creating any number of empty child nodes to represent your rlm@486: creature's ears. rlm@486: rlm@486: Ear nodes do not require any metadata. rlm@486: rlm@486: The ear will bind to and follow the closest physical node. rlm@486: rlm@486: *** Touch rlm@486: rlm@486: Touch is handled similarly to mass. To make a particular object rlm@486: touch sensitive, add metadata of the following form under the rlm@486: object's ``touch'' metadata field: rlm@486: rlm@486: #+BEGIN_EXAMPLE rlm@486: rlm@486: #+END_EXAMPLE rlm@486: rlm@486: You may also include an optional ``scale'' metadata number to rlm@486: specifiy the length of the touch feelers. The default is $0.1$, rlm@486: and this is generally sufficient. rlm@486: rlm@486: The touch UV should contain white pixels for each touch sensor. rlm@486: rlm@486: Here is an example touch-uv map that approximates a human finger, rlm@486: and its corresponding model. rlm@486: rlm@486: #+caption: This is the tactile-sensor-profile for the upper segment rlm@486: #+caption: of a fingertip. It defines regions of high touch sensitivity rlm@486: #+caption: (where there are many white pixels) and regions of low rlm@486: #+caption: sensitivity (where white pixels are sparse). rlm@486: #+name: guide-fingertip-UV rlm@486: #+ATTR_LaTeX: :width 9cm :placement [H] rlm@486: [[./images/finger-UV.png]] rlm@486: rlm@486: #+caption: The fingertip UV-image form above applied to a simple rlm@486: #+caption: model of a fingertip. rlm@486: #+name: guide-fingertip rlm@486: #+ATTR_LaTeX: :width 9cm :placement [H] rlm@486: [[./images/finger-2.png]] rlm@486: rlm@486: *** Propriocepotion rlm@486: rlm@486: Proprioception is tied to each joint node -- nothing special must rlm@486: be done in a blender model to enable proprioception other than rlm@486: creating joint nodes. rlm@486: rlm@486: *** Muscles rlm@486: rlm@486: Muscles are created by creating an empty node named =muscles= and rlm@486: then creating any number of empty child nodes to represent your rlm@486: creature's muscles. rlm@486: rlm@486: rlm@486: Muscle nodes should have the following metadata under the rlm@486: ``muscle'' label: rlm@486: rlm@486: #+BEGIN_EXAMPLE rlm@486: rlm@486: #+END_EXAMPLE rlm@486: rlm@486: Muscles should also have a ``strength'' metadata entry describing rlm@486: the muscle's total strength at full activation. rlm@486: rlm@486: Muscle profiles are simple images that contain the relative amount rlm@486: of muscle power in each simulated alpha motor neuron. The width of rlm@486: the image is the total size of the motor pool, and the redness of rlm@486: each neuron is the relative power of that motor pool. rlm@486: rlm@486: While the profile image can have any dimensions, only the first rlm@486: line of pixels is used to define the muscle. Here is a sample rlm@486: muscle profile image that defines a human-like muscle. rlm@486: rlm@486: #+caption: A muscle profile image that describes the strengths rlm@486: #+caption: of each motor neuron in a muscle. White is weakest rlm@486: #+caption: and dark red is strongest. This particular pattern rlm@486: #+caption: has weaker motor neurons at the beginning, just rlm@486: #+caption: like human muscle. rlm@486: #+name: muscle-recruit rlm@486: #+ATTR_LaTeX: :width 7cm :placement [H] rlm@486: [[./images/basic-muscle.png]] rlm@486: rlm@486: Muscles twist the nearest physical object about the muscle node's rlm@486: Z-axis. I recommend using the ``Single Arrow'' display mode for rlm@486: muscles and using the right hand rule to determine which way the rlm@486: muscle will twist. To make a segment that can twist in multiple rlm@486: directions, create multiple, differently aligned muscles. rlm@486: rlm@486: ** =CORTEX= API rlm@486: rlm@486: These are the some functions exposed by =CORTEX= for creating rlm@486: worlds and simulating creatures. These are in addition to rlm@486: jMonkeyEngine3's extensive library, which is documented elsewhere. rlm@486: rlm@486: *** Simulation rlm@486: - =(world root-node key-map setup-fn update-fn)= :: create rlm@486: a simulation. rlm@486: - /root-node/ :: a =com.jme3.scene.Node= object which rlm@486: contains all of the objects that should be in the rlm@486: simulation. rlm@486: rlm@486: - /key-map/ :: a map from strings describing keys to rlm@486: functions that should be executed whenever that key is rlm@486: pressed. the functions should take a SimpleApplication rlm@486: object and a boolean value. The SimpleApplication is the rlm@486: current simulation that is running, and the boolean is true rlm@486: if the key is being pressed, and false if it is being rlm@486: released. As an example, rlm@486: #+BEGIN_SRC clojure rlm@486: {"key-j" (fn [game value] (if value (println "key j pressed")))} rlm@486: #+END_SRC rlm@486: is a valid key-map which will cause the simulation to print rlm@486: a message whenever the 'j' key on the keyboard is pressed. rlm@486: rlm@486: - /setup-fn/ :: a function that takes a =SimpleApplication= rlm@486: object. It is called once when initializing the simulation. rlm@486: Use it to create things like lights, change the gravity, rlm@486: initialize debug nodes, etc. rlm@486: rlm@486: - /update-fn/ :: this function takes a =SimpleApplication= rlm@486: object and a float and is called every frame of the rlm@486: simulation. The float tells how many seconds is has been rlm@486: since the last frame was rendered, according to whatever rlm@486: clock jme is currently using. The default is to use IsoTimer rlm@486: which will result in this value always being the same. rlm@486: rlm@486: - =(position-camera world position rotation)= :: set the position rlm@486: of the simulation's main camera. rlm@487: rlm@486: - =(enable-debug world)= :: turn on debug wireframes for each rlm@486: simulated object. rlm@487: rlm@486: - =(set-gravity world gravity)= :: set the gravity of a running rlm@486: simulation. rlm@487: rlm@486: - =(box length width height & {options})= :: create a box in the rlm@486: simulation. Options is a hash map specifying texture, mass, rlm@486: etc. Possible options are =:name=, =:color=, =:mass=, rlm@486: =:friction=, =:texture=, =:material=, =:position=, rlm@486: =:rotation=, =:shape=, and =:physical?=. rlm@487: rlm@486: - =(sphere radius & {options})= :: create a sphere in the simulation. rlm@486: Options are the same as in =box=. rlm@487: rlm@486: - =(load-blender-model file-name)= :: create a node structure rlm@486: representing that described in a blender file. rlm@487: rlm@486: - =(light-up-everything world)= :: distribute a standard compliment rlm@486: of lights throught the simulation. Should be adequate for most rlm@486: purposes. rlm@487: rlm@486: - =(node-seq node)= :: return a recursuve list of the node's rlm@486: children. rlm@487: rlm@486: - =(nodify name children)= :: construct a node given a node-name and rlm@486: desired children. rlm@487: rlm@486: - =(add-element world element)= :: add an object to a running world rlm@486: simulation. rlm@487: rlm@486: - =(set-accuracy world accuracy)= :: change the accuracy of the rlm@486: world's physics simulator. rlm@487: rlm@486: - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine rlm@486: construct that is useful for loading textures and is required rlm@486: for smooth interaction with jMonkeyEngine library functions. rlm@487: rlm@487: - =(load-bullet)= :: unpack native libraries and initialize rlm@487: blender. This function is required before other world building rlm@487: functions are called. rlm@486: rlm@486: rlm@486: *** Creature Manipulation / Import rlm@486: rlm@486: - =(body! creature)= :: give the creature a physical body. rlm@487: rlm@486: - =(vision! creature)= :: give the creature a sense of vision. rlm@486: Returns a list of functions which will each, when called rlm@486: during a simulation, return the vision data for the channel of rlm@486: one of the eyes. The functions are ordered depending on the rlm@486: alphabetical order of the names of the eye nodes in the rlm@486: blender file. The data returned by the functions is a vector rlm@486: containing the eye's /topology/, a vector of coordinates, and rlm@486: the eye's /data/, a vector of RGB values filtered by the eye's rlm@486: sensitivity. rlm@487: rlm@486: - =(hearing! creature)= :: give the creature a sense of hearing. rlm@486: Returns a list of functions, one for each ear, that when rlm@486: called will return a frame's worth of hearing data for that rlm@486: ear. The functions are ordered depending on the alphabetical rlm@486: order of the names of the ear nodes in the blender file. The rlm@486: data returned by the functions is an array PCM encoded wav rlm@486: data. rlm@487: rlm@486: - =(touch! creature)= :: give the creature a sense of touch. Returns rlm@486: a single function that must be called with the /root node/ of rlm@486: the world, and which will return a vector of /touch-data/ rlm@486: one entry for each touch sensitive component, each entry of rlm@486: which contains a /topology/ that specifies the distribution of rlm@486: touch sensors, and the /data/, which is a vector of rlm@486: =[activation, length]= pairs for each touch hair. rlm@487: rlm@486: - =(proprioception! creature)= :: give the creature the sense of rlm@486: proprioception. Returns a list of functions, one for each rlm@486: joint, that when called during a running simulation will rlm@486: report the =[headnig, pitch, roll]= of the joint. rlm@487: rlm@486: - =(movement! creature)= :: give the creature the power of movement. rlm@486: Creates a list of functions, one for each muscle, that when rlm@486: called with an integer, will set the recruitment of that rlm@486: muscle to that integer, and will report the current power rlm@486: being exerted by the muscle. Order of muscles is determined by rlm@486: the alphabetical sort order of the names of the muscle nodes. rlm@486: rlm@486: *** Visualization/Debug rlm@486: rlm@486: - =(view-vision)= :: create a function that when called with a list rlm@486: of visual data returned from the functions made by =vision!=, rlm@486: will display that visual data on the screen. rlm@487: rlm@486: - =(view-hearing)= :: same as =view-vision= but for hearing. rlm@487: rlm@486: - =(view-touch)= :: same as =view-vision= but for touch. rlm@487: rlm@486: - =(view-proprioception)= :: same as =view-vision= but for rlm@486: proprioception. rlm@487: rlm@486: - =(view-movement)= :: same as =view-vision= but for rlm@486: proprioception. rlm@487: rlm@486: - =(view anything)= :: =view= is a polymorphic function that allows rlm@486: you to inspect almost anything you could reasonably expect to rlm@486: be able to ``see'' in =CORTEX=. rlm@487: rlm@486: - =(text anything)= :: =text= is a polymorphic function that allows rlm@486: you to convert practically anything into a text string. rlm@487: rlm@486: - =(println-repl anything)= :: print messages to clojure's repl rlm@486: instead of the simulation's terminal window. rlm@486: rlm@486: - =(mega-import-jme3)= :: for experimenting at the REPL. This rlm@486: function will import all jMonkeyEngine3 classes for immediate rlm@486: use. rlm@487: rlm@487: - =(display-dialated-time world timer)= :: Shows the time as it is rlm@487: flowing in the simulation on a HUD display. rlm@487: rlm@487: rlm@487: