In order for this to be a reasonable thesis that I can be proud of,

 what are the /minimum/ number of things I need to get done?

 

 

 * worm OR hand registration

   - training from a few examples (2 to start out)

   - aligning the body with the scene

   - generating sensory data

   - matching previous labeled examples using dot-products or some

     other basic thing

   - showing that it works with different views

 

 * first draft

   - draft of thesis without bibliography or formatting

   - should have basic experiment and have full description of

     framework with code

   - review with Winston

   

 * final draft

   - implement stretch goals from Winston if possible

   - complete final formatting and submit

 

 * CORTEX

   DEADLINE: <2014-05-09 Fri>

   SHIT THAT'S IN 67 DAYS!!!

 

 ** program simple feature matching code for the worm's segments

 

 Subgoals:

 *** DONE Get cortex working again, run tests, no jmonkeyengine updates

     CLOSED: [2014-03-03 Mon 22:07] SCHEDULED: <2014-03-03 Mon>

 *** DONE get blender working again

     CLOSED: [2014-03-03 Mon 22:43] SCHEDULED: <2014-03-03 Mon>

 *** DONE make sparce touch worm segment in blender

     CLOSED: [2014-03-03 Mon 23:16] SCHEDULED: <2014-03-03 Mon>

     CLOCK: [2014-03-03 Mon 22:44]--[2014-03-03 Mon 23:16] =>  0:32

 *** DONE make multi-segment touch worm with touch sensors and display

     CLOSED: [2014-03-03 Mon 23:54] SCHEDULED: <2014-03-03 Mon>

 

 *** DONE Make a worm wiggle and curl

     CLOSED: [2014-03-04 Tue 23:03] SCHEDULED: <2014-03-04 Tue>

 

 

 ** First draft

 

 Subgoals:

 *** Writeup new worm experiments.

 *** Triage implementation code and get it into chapter form.

 

 

 

  

 

 ** for today

 

 - guided worm :: control the worm with the keyboard. Useful for

                  testing the body-centered recog scripts, and for

                  preparing a cool demo video.

 

 - body-centered recognition :: detect actions using hard coded

      body-centered scripts. 

 

 - cool demo video of the worm being moved and recognizing things ::

      will be a neat part of the thesis.

 

 - thesis export :: refactoring and organization of code so that it

                    spits out a thesis in addition to the web page.

 

 - video alignment :: analyze the frames of a video in order to align

      the worm. Requires body-centered recognition. Can "cheat".

 

 - smoother actions :: use debugging controls to directly influence the

      demo actions, and to generate recoginition procedures.

 

 - degenerate video demonstration :: show the system recognizing a

      curled worm from dead on. Crowning achievement of thesis.

 

 ** Ordered from easiest to hardest

 

 Just report the positions of everything. I don't think that this

 necessairly shows anything usefull.

 

 Worm-segment vision -- you initialize a view of the worm, but instead

 of pixels you use labels via ray tracing. Has the advantage of still

 allowing for visual occlusion, but reliably identifies the objects,

 even without rainbow coloring. You can code this as an image. 

 

 Same as above, except just with worm/non-worm labels.

 

 Color code each worm segment and then recognize them using blob

 detectors. Then you solve for the perspective and the action

 simultaneously.

 

 The entire worm can be colored the same, high contrast color against a

 nearly black background.

 

 "Rooted" vision. You give the exact coordinates of ONE piece of the

 worm, but the algorithm figures out the rest.

 

 More rooted vision -- start off the entire worm with one posistion.

 

 The right way to do alignment is to use motion over multiple frames to

 snap individual pieces of the model into place sharing and

 propragating the individual alignments over the whole model. We also

 want to limit the alignment search to just those actions we are

 prepared to identify. This might mean that I need some small "micro

 actions" such as the individual movements of the worm pieces.

 

 Get just the centers of each segment projected onto the imaging

 plane. (best so far).

 

 

 Repertoire of actions  +  video frames -->

    directed multi-frame-search alg

 

 

 

 

 

 

 !! Could also have a bounding box around the worm provided by

 filtering the worm/non-worm render, and use bbbgs. As a bonus, I get

 to include bbbgs in my thesis! Could finally do that recursive things

 where I make bounding boxes be those things that give results that

 give good bounding boxes. If I did this I could use a disruptive

 pattern on the worm.

 

 Re imagining using default textures is very simple for this system,

 but hard for others.

 

 

 Want to demonstrate, at minimum, alignment of some model of the worm

 to the video, and a lookup of the action by simulated perception.

 

 note: the purple/white points is a very beautiful texture, because

 when it moves slightly, the white dots look like they're

 twinkling. Would look even better if it was a darker purple. Also

 would look better more spread out.

 

 

 embed assumption of one frame of view, search by moving around in

 simulated world.

 

 Allowed to limit search by setting limits to a hemisphere around the

 imagined worm! This limits scale also.

 

 

 

 

 

 !! Limited search with worm/non-worm rendering. 

 How much inverse kinematics do we have to do?

 What about cached (allowed state-space) paths, derived from labeled

 training. You have to lead from one to another.

 

 What about initial state? Could start the input videos at a specific

 state, then just match that explicitly.

 

 !! The training doesn't have to be labeled -- you can just move around

 for a while!!

 

 !! Limited search with motion based alignment.

 

 

 

 

 "play arounds" can establish a chain of linked sensoriums. Future

 matches must fall into one of the already experienced things, and once

 they do, it greatly limits the things that are possible in the future.

 

 

 frame differences help to detect muscle exertion.

 

 Can try to match on a few "representative" frames. Can also just have

 a few "bodies" in various states which we try to match.

 

 

 

 Paths through state-space have the exact same signature as

 simulation. BUT, these can be searched in parallel and don't interfere

 with each other.

 

 

 

 

 ** Final stretch up to First Draft

 

 *** DONE complete debug control of worm

     CLOSED: [2014-03-17 Mon 17:29] SCHEDULED: <2014-03-17 Mon>

     CLOCK: [2014-03-17 Mon 14:01]--[2014-03-17 Mon 17:29] =>  3:28

 *** DONE add phi-space output to debug control

     CLOSED: [2014-03-17 Mon 17:42] SCHEDULED: <2014-03-17 Mon>

     CLOCK: [2014-03-17 Mon 17:31]--[2014-03-17 Mon 17:42] =>  0:11

 

 *** DONE complete automatic touch partitioning

     CLOSED: [2014-03-18 Tue 21:43] SCHEDULED: <2014-03-18 Tue>

 *** TODO complete cyclic predicate

     SCHEDULED: <2014-03-18 Tue>

 *** TODO complete three phi-stream action predicatates; test them with debug control

     SCHEDULED: <2014-03-17 Mon>

     CLOCK: [2014-03-18 Tue 18:36]

     CLOCK: [2014-03-18 Tue 18:34]--[2014-03-18 Tue 18:36] =>  0:02

     CLOCK: [2014-03-17 Mon 19:19]--[2014-03-17 Mon 21:19] =>  2:00

 *** TODO create test videos, also record positions of worm segments

     SCHEDULED: <2014-03-17 Mon>

 *** TODO complete proprioception based movement lookup in phi-space

     SCHEDULED: <2014-03-17 Mon>

 

 

 *** TODO Collect intro, worm-learn and cortex creation into draft thesis.