# HG changeset patch # User Robert McIntyre # Date 1396138372 14400 # Node ID 6d460ac3f5d028f743156c500ec59b9abeb287fe # Parent ac953b562eabd6a9d2445c71262a026ca2e93b84 working on user guide. diff -r ac953b562eab -r 6d460ac3f5d0 thesis/cortex.bib --- a/thesis/cortex.bib Sat Mar 29 16:22:49 2014 -0400 +++ b/thesis/cortex.bib Sat Mar 29 20:12:52 2014 -0400 @@ -23,4 +23,6 @@ @COMMENT sussman's HACKER -@comment motion as a 4D manifold \ No newline at end of file +@comment motion as a 4D manifold + +@comment winston multiple modality white paper \ No newline at end of file diff -r ac953b562eab -r 6d460ac3f5d0 thesis/cortex.org --- a/thesis/cortex.org Sat Mar 29 16:22:49 2014 -0400 +++ b/thesis/cortex.org Sat Mar 29 20:12:52 2014 -0400 @@ -1627,7 +1627,7 @@ #+caption: of a fingertip. It defines regions of high touch sensitivity #+caption: (where there are many white pixels) and regions of low #+caption: sensitivity (where white pixels are sparse). - #+name: fimgertip-UV + #+name: fingertip-UV #+ATTR_LaTeX: :width 13cm [[./images/finger-UV.png]] diff -r ac953b562eab -r 6d460ac3f5d0 thesis/images/finger-2.png Binary file thesis/images/finger-2.png has changed diff -r ac953b562eab -r 6d460ac3f5d0 thesis/rlm-cortex-meng.tex --- a/thesis/rlm-cortex-meng.tex Sat Mar 29 16:22:49 2014 -0400 +++ b/thesis/rlm-cortex-meng.tex Sat Mar 29 20:12:52 2014 -0400 @@ -58,7 +58,7 @@ \usepackage{xcolor} \definecolor{dark-red}{rgb}{0.4,0.15,0.15} \definecolor{dark-blue}{rgb}{0.15,0.4,0.15} -\definecolor{medium-blue}{rgb}{0,0,0.5} +\definecolor{medium-blue}{rgb}{0.15,0.4,0.4} \hypersetup{ colorlinks, linkcolor={dark-red}, citecolor={dark-blue}, urlcolor={medium-blue} diff -r ac953b562eab -r 6d460ac3f5d0 thesis/user-guide.org --- a/thesis/user-guide.org Sat Mar 29 16:22:49 2014 -0400 +++ b/thesis/user-guide.org Sat Mar 29 20:12:52 2014 -0400 @@ -1,6 +1,338 @@ * Appendix: =CORTEX= User Guide - For future students who whould like to use =CORTEX= in their own - projects. + Those who write a thesis should endeavor to make their code not only + accessable, but actually useable, as a way to pay back the community + that made the thesis possible in the first place. This thesis would + not be possible without Free Software such as jMonkeyEngine3, + Blender, clojure, emacs, ffmpeg, and many other tools. That is why I + have included this user guide, in the hope that someone else might + find =CORTEX= useful. +** Obtaining =CORTEX= + You can get cortex from its mercurial repository at + http://hg.bortreb.com/cortex. You may also download =CORTEX= + releases at http://aurellem.org/cortex/releases/. As a condition of + making this thesis, I have also provided Professor Winston the + =CORTEX= source, and he knows how to run the demos and get started. + You may also email me at =cortex@aurellem.org= and I may help where + I can. + +** Running =CORTEX= + + =CORTEX= comes with README and INSTALL files that will guide you + through installation and running the test suite. In particular you + should look at test =cortex.test= which contains test suites that + run through all senses and multiple creatures. + +** Creating creatures + + Creatures are created using /Blender/, a free 3D modeling program. + You will need Blender version 2.6 when using the =CORTEX= included + in this thesis. You create a =CORTEX= creature in a similiar manner + to modeling anything in Blender, except that you also create + several trees of empty nodes which define the creature's senses. + +*** Mass + + To give an object mass in =CORTEX=, add a ``mass'' metadata label + to the object with the mass in jMonkeyEngine units. Note that + setting the mass to 0 causes the object to be immovable. + +*** Joints + + Joints are created by creating an empty node named =joints= and + then creating any number of empty child nodes to represent your + creature's joints. The joint will automatically connect the + closest two physical objects. It will help to set the empty node's + display mode to ``Arrows'' so that you can clearly see the + direction of the axes. + + Joint nodes should have the following metadata under the ``joint'' + label: + + #+BEGIN_SRC clojure +;; ONE OF the following, under the label "joint": +{:type :point} + +;; OR + +{:type :hinge + :limit [ ] + :axis (Vector3f. )} +;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints) + +;; OR + +{:type :cone + :limit-xz + :limit-xy + :twist } ;(use XZY rotation mode in blender!) + #+END_SRC + +*** Eyes + + Eyes are created by creating an empty node named =eyes= and then + creating any number of empty child nodes to represent your + creature's eyes. + + Eye nodes should have the following metadata under the ``eye'' + label: + +#+BEGIN_SRC clojure +{:red + :blue + :green + :all + (<0xrrggbb> )... +} +#+END_SRC + + Any of the color channels may be omitted. You may also include + your own color selectors, and in fact :red is equivalent to + 0xFF0000 and so forth. The eye will be placed at the same position + as the empty node and will bind to the neatest physical object. + The eye will point outward from the X-axis of the node, and ``up'' + will be in the direction of the X-axis of the node. It will help + to set the empty node's display mode to ``Arrows'' so that you can + clearly see the direction of the axes. + + Each retina file should contain white pixels whever you want to be + sensitive to your chosen color. If you want the entire field of + view, specify :all of 0xFFFFFF and a retinal map that is entirely + white. + + Here is a sample retinal map: + + #+caption: An example retinal profile image. White pixels are + #+caption: photo-sensitive elements. The distribution of white + #+caption: pixels is denser in the middle and falls off at the + #+caption: edges and is inspired by the human retina. + #+name: retina + #+ATTR_LaTeX: :width 7cm :placement [H] + [[./images/retina-small.png]] + +*** Hearing + + Ears are created by creating an empty node named =ears= and then + creating any number of empty child nodes to represent your + creature's ears. + + Ear nodes do not require any metadata. + + The ear will bind to and follow the closest physical node. + +*** Touch + + Touch is handled similarly to mass. To make a particular object + touch sensitive, add metadata of the following form under the + object's ``touch'' metadata field: + + #+BEGIN_EXAMPLE + + #+END_EXAMPLE + + You may also include an optional ``scale'' metadata number to + specifiy the length of the touch feelers. The default is $0.1$, + and this is generally sufficient. + + The touch UV should contain white pixels for each touch sensor. + + Here is an example touch-uv map that approximates a human finger, + and its corresponding model. + + #+caption: This is the tactile-sensor-profile for the upper segment + #+caption: of a fingertip. It defines regions of high touch sensitivity + #+caption: (where there are many white pixels) and regions of low + #+caption: sensitivity (where white pixels are sparse). + #+name: guide-fingertip-UV + #+ATTR_LaTeX: :width 9cm :placement [H] + [[./images/finger-UV.png]] + + #+caption: The fingertip UV-image form above applied to a simple + #+caption: model of a fingertip. + #+name: guide-fingertip + #+ATTR_LaTeX: :width 9cm :placement [H] + [[./images/finger-2.png]] + +*** Propriocepotion + + Proprioception is tied to each joint node -- nothing special must + be done in a blender model to enable proprioception other than + creating joint nodes. + +*** Muscles + + Muscles are created by creating an empty node named =muscles= and + then creating any number of empty child nodes to represent your + creature's muscles. + + + Muscle nodes should have the following metadata under the + ``muscle'' label: + + #+BEGIN_EXAMPLE + + #+END_EXAMPLE + + Muscles should also have a ``strength'' metadata entry describing + the muscle's total strength at full activation. + + Muscle profiles are simple images that contain the relative amount + of muscle power in each simulated alpha motor neuron. The width of + the image is the total size of the motor pool, and the redness of + each neuron is the relative power of that motor pool. + + While the profile image can have any dimensions, only the first + line of pixels is used to define the muscle. Here is a sample + muscle profile image that defines a human-like muscle. + + #+caption: A muscle profile image that describes the strengths + #+caption: of each motor neuron in a muscle. White is weakest + #+caption: and dark red is strongest. This particular pattern + #+caption: has weaker motor neurons at the beginning, just + #+caption: like human muscle. + #+name: muscle-recruit + #+ATTR_LaTeX: :width 7cm :placement [H] + [[./images/basic-muscle.png]] + + Muscles twist the nearest physical object about the muscle node's + Z-axis. I recommend using the ``Single Arrow'' display mode for + muscles and using the right hand rule to determine which way the + muscle will twist. To make a segment that can twist in multiple + directions, create multiple, differently aligned muscles. + +** =CORTEX= API + + These are the some functions exposed by =CORTEX= for creating + worlds and simulating creatures. These are in addition to + jMonkeyEngine3's extensive library, which is documented elsewhere. + +*** Simulation + - =(world root-node key-map setup-fn update-fn)= :: create + a simulation. + - /root-node/ :: a =com.jme3.scene.Node= object which + contains all of the objects that should be in the + simulation. + + - /key-map/ :: a map from strings describing keys to + functions that should be executed whenever that key is + pressed. the functions should take a SimpleApplication + object and a boolean value. The SimpleApplication is the + current simulation that is running, and the boolean is true + if the key is being pressed, and false if it is being + released. As an example, + #+BEGIN_SRC clojure + {"key-j" (fn [game value] (if value (println "key j pressed")))} + #+END_SRC + is a valid key-map which will cause the simulation to print + a message whenever the 'j' key on the keyboard is pressed. + + - /setup-fn/ :: a function that takes a =SimpleApplication= + object. It is called once when initializing the simulation. + Use it to create things like lights, change the gravity, + initialize debug nodes, etc. + + - /update-fn/ :: this function takes a =SimpleApplication= + object and a float and is called every frame of the + simulation. The float tells how many seconds is has been + since the last frame was rendered, according to whatever + clock jme is currently using. The default is to use IsoTimer + which will result in this value always being the same. + + + - =(position-camera world position rotation)= :: set the position + of the simulation's main camera. + - =(enable-debug world)= :: turn on debug wireframes for each + simulated object. + - =(set-gravity world gravity)= :: set the gravity of a running + simulation. + - =(box length width height & {options})= :: create a box in the + simulation. Options is a hash map specifying texture, mass, + etc. Possible options are =:name=, =:color=, =:mass=, + =:friction=, =:texture=, =:material=, =:position=, + =:rotation=, =:shape=, and =:physical?=. + - =(sphere radius & {options})= :: create a sphere in the simulation. + Options are the same as in =box=. + - =(load-blender-model file-name)= :: create a node structure + representing that described in a blender file. + - =(light-up-everything world)= :: distribute a standard compliment + of lights throught the simulation. Should be adequate for most + purposes. + - =(node-seq node)= :: return a recursuve list of the node's + children. + - =(nodify name children)= :: construct a node given a node-name and + desired children. + - =(add-element world element)= :: add an object to a running world + simulation. + - =(set-accuracy world accuracy)= :: change the accuracy of the + world's physics simulator. + - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine + construct that is useful for loading textures and is required + for smooth interaction with jMonkeyEngine library functions. + + +*** Creature Manipulation / Import + + - =(body! creature)= :: give the creature a physical body. + - =(vision! creature)= :: give the creature a sense of vision. + Returns a list of functions which will each, when called + during a simulation, return the vision data for the channel of + one of the eyes. The functions are ordered depending on the + alphabetical order of the names of the eye nodes in the + blender file. The data returned by the functions is a vector + containing the eye's /topology/, a vector of coordinates, and + the eye's /data/, a vector of RGB values filtered by the eye's + sensitivity. + - =(hearing! creature)= :: give the creature a sense of hearing. + Returns a list of functions, one for each ear, that when + called will return a frame's worth of hearing data for that + ear. The functions are ordered depending on the alphabetical + order of the names of the ear nodes in the blender file. The + data returned by the functions is an array PCM encoded wav + data. + - =(touch! creature)= :: give the creature a sense of touch. Returns + a single function that must be called with the /root node/ of + the world, and which will return a vector of /touch-data/ + one entry for each touch sensitive component, each entry of + which contains a /topology/ that specifies the distribution of + touch sensors, and the /data/, which is a vector of + =[activation, length]= pairs for each touch hair. + - =(proprioception! creature)= :: give the creature the sense of + proprioception. Returns a list of functions, one for each + joint, that when called during a running simulation will + report the =[headnig, pitch, roll]= of the joint. + - =(movement! creature)= :: give the creature the power of movement. + Creates a list of functions, one for each muscle, that when + called with an integer, will set the recruitment of that + muscle to that integer, and will report the current power + being exerted by the muscle. Order of muscles is determined by + the alphabetical sort order of the names of the muscle nodes. + +*** Visualization/Debug + + - =(view-vision)= :: create a function that when called with a list + of visual data returned from the functions made by =vision!=, + will display that visual data on the screen. + - =(view-hearing)= :: same as =view-vision= but for hearing. + - =(view-touch)= :: same as =view-vision= but for touch. + - =(view-proprioception)= :: same as =view-vision= but for + proprioception. + - =(view-movement)= :: same as =view-vision= but for + proprioception. + - =(view anything)= :: =view= is a polymorphic function that allows + you to inspect almost anything you could reasonably expect to + be able to ``see'' in =CORTEX=. + - =(text anything)= :: =text= is a polymorphic function that allows + you to convert practically anything into a text string. + - =(println-repl anything)= :: print messages to clojure's repl + instead of the simulation's terminal window. + +*** Misc + + - =(load-bullet)= :: unpack native libraries and initialize + blender. This function is required before other world building + functions are called. + - =(mega-import-jme3)= :: for experimenting at the REPL. This + function will import all jMonkeyEngine3 classes for immediate + use.