cortex: thesis/org/roadmap.org comparison

comparison thesis/org/roadmap.org @ 430:5205535237fb

fix skew in self-organizing-touch, work on thesis.

author	Robert McIntyre <rlm@mit.edu>
date	Sat, 22 Mar 2014 16:10:34 -0400
parents	thesis/aux/org/roadmap.org@6b0f77df0e53
children	8e52a2802821

comparison

equal deleted inserted replaced

-:b5d0f0adf19f
+:5205535237fb
+In order for this to be a reasonable thesis that I can be proud of,
+what are the /minimum/ number of things I need to get done?
+* worm OR hand registration
+- training from a few examples (2 to start out)
+- aligning the body with the scene
+- generating sensory data
+- matching previous labeled examples using dot-products or some
+other basic thing
+- showing that it works with different views
+* first draft
+- draft of thesis without bibliography or formatting
+- should have basic experiment and have full description of
+framework with code
+- review with Winston
+* final draft
+- implement stretch goals from Winston if possible
+- complete final formatting and submit
+* CORTEX
+DEADLINE: <2014-05-09 Fri>
+SHIT THAT'S IN 67 DAYS!!!
+** program simple feature matching code for the worm's segments
+Subgoals:
+*** DONE Get cortex working again, run tests, no jmonkeyengine updates
+CLOSED: [2014-03-03 Mon 22:07] SCHEDULED: <2014-03-03 Mon>
+*** DONE get blender working again
+CLOSED: [2014-03-03 Mon 22:43] SCHEDULED: <2014-03-03 Mon>
+*** DONE make sparce touch worm segment in blender
+CLOSED: [2014-03-03 Mon 23:16] SCHEDULED: <2014-03-03 Mon>
+CLOCK: [2014-03-03 Mon 22:44]--[2014-03-03 Mon 23:16] =>  0:32
+*** DONE make multi-segment touch worm with touch sensors and display
+CLOSED: [2014-03-03 Mon 23:54] SCHEDULED: <2014-03-03 Mon>
+*** DONE Make a worm wiggle and curl
+CLOSED: [2014-03-04 Tue 23:03] SCHEDULED: <2014-03-04 Tue>
+** First draft
+Subgoals:
+*** Writeup new worm experiments.
+*** Triage implementation code and get it into chapter form.
+** for today
+- guided worm :: control the worm with the keyboard. Useful for
+testing the body-centered recog scripts, and for
+preparing a cool demo video.
+- body-centered recognition :: detect actions using hard coded
+body-centered scripts.
+- cool demo video of the worm being moved and recognizing things ::
+will be a neat part of the thesis.
+- thesis export :: refactoring and organization of code so that it
+spits out a thesis in addition to the web page.
+- video alignment :: analyze the frames of a video in order to align
+the worm. Requires body-centered recognition. Can "cheat".
+- smoother actions :: use debugging controls to directly influence the
+demo actions, and to generate recoginition procedures.
+- degenerate video demonstration :: show the system recognizing a
+curled worm from dead on. Crowning achievement of thesis.
+** Ordered from easiest to hardest
+Just report the positions of everything. I don't think that this
+necessairly shows anything usefull.
+Worm-segment vision -- you initialize a view of the worm, but instead
+of pixels you use labels via ray tracing. Has the advantage of still
+allowing for visual occlusion, but reliably identifies the objects,
+even without rainbow coloring. You can code this as an image.
+Same as above, except just with worm/non-worm labels.
+Color code each worm segment and then recognize them using blob
+detectors. Then you solve for the perspective and the action
+simultaneously.
+The entire worm can be colored the same, high contrast color against a
+nearly black background.
+"Rooted" vision. You give the exact coordinates of ONE piece of the
+worm, but the algorithm figures out the rest.
+More rooted vision -- start off the entire worm with one posistion.
+The right way to do alignment is to use motion over multiple frames to
+snap individual pieces of the model into place sharing and
+propragating the individual alignments over the whole model. We also
+want to limit the alignment search to just those actions we are
+prepared to identify. This might mean that I need some small "micro
+actions" such as the individual movements of the worm pieces.
+Get just the centers of each segment projected onto the imaging
+plane. (best so far).
+Repertoire of actions  +  video frames -->
+directed multi-frame-search alg
+!! Could also have a bounding box around the worm provided by
+filtering the worm/non-worm render, and use bbbgs. As a bonus, I get
+to include bbbgs in my thesis! Could finally do that recursive things
+where I make bounding boxes be those things that give results that
+give good bounding boxes. If I did this I could use a disruptive
+pattern on the worm.
+Re imagining using default textures is very simple for this system,
+but hard for others.
+Want to demonstrate, at minimum, alignment of some model of the worm
+to the video, and a lookup of the action by simulated perception.
+note: the purple/white points is a very beautiful texture, because
+when it moves slightly, the white dots look like they're
+twinkling. Would look even better if it was a darker purple. Also
+would look better more spread out.
+embed assumption of one frame of view, search by moving around in
+simulated world.
+Allowed to limit search by setting limits to a hemisphere around the
+imagined worm! This limits scale also.
+!! Limited search with worm/non-worm rendering.
+How much inverse kinematics do we have to do?
+What about cached (allowed state-space) paths, derived from labeled
+training. You have to lead from one to another.
+What about initial state? Could start the input videos at a specific
+state, then just match that explicitly.
+!! The training doesn't have to be labeled -- you can just move around
+for a while!!
+!! Limited search with motion based alignment.
+"play arounds" can establish a chain of linked sensoriums. Future
+matches must fall into one of the already experienced things, and once
+they do, it greatly limits the things that are possible in the future.
+frame differences help to detect muscle exertion.
+Can try to match on a few "representative" frames. Can also just have
+a few "bodies" in various states which we try to match.
+Paths through state-space have the exact same signature as
+simulation. BUT, these can be searched in parallel and don't interfere
+with each other.
+** Final stretch up to First Draft
+*** DONE complete debug control of worm
+CLOSED: [2014-03-17 Mon 17:29] SCHEDULED: <2014-03-17 Mon>
+CLOCK: [2014-03-17 Mon 14:01]--[2014-03-17 Mon 17:29] =>  3:28
+*** DONE add phi-space output to debug control
+CLOSED: [2014-03-17 Mon 17:42] SCHEDULED: <2014-03-17 Mon>
+CLOCK: [2014-03-17 Mon 17:31]--[2014-03-17 Mon 17:42] =>  0:11
+*** DONE complete automatic touch partitioning
+CLOSED: [2014-03-18 Tue 21:43] SCHEDULED: <2014-03-18 Tue>
+*** DONE complete cyclic predicate
+CLOSED: [2014-03-19 Wed 16:34] SCHEDULED: <2014-03-18 Tue>
+CLOCK: [2014-03-19 Wed 13:16]--[2014-03-19 Wed 16:34] =>  3:18
+*** DONE complete three phi-stream action predicatates; test them with debug control
+CLOSED: [2014-03-19 Wed 16:35] SCHEDULED: <2014-03-17 Mon>
+CLOCK: [2014-03-18 Tue 18:36]--[2014-03-18 Tue 21:43] =>  3:07
+CLOCK: [2014-03-18 Tue 18:34]--[2014-03-18 Tue 18:36] =>  0:02
+CLOCK: [2014-03-17 Mon 19:19]--[2014-03-17 Mon 21:19] =>  2:00
+*** DONE build an automatic "do all the things" sequence.
+CLOSED: [2014-03-19 Wed 16:55] SCHEDULED: <2014-03-19 Wed>
+CLOCK: [2014-03-19 Wed 16:53]--[2014-03-19 Wed 16:55] =>  0:02
+*** DONE implement proprioception based movement lookup in phi-space
+CLOSED: [2014-03-19 Wed 22:04] SCHEDULED: <2014-03-19 Wed>
+CLOCK: [2014-03-19 Wed 19:32]--[2014-03-19 Wed 22:04] =>  2:32
+*** DONE make proprioception reference phi-space indexes
+CLOSED: [2014-03-19 Wed 22:47] SCHEDULED: <2014-03-19 Wed>
+CLOCK: [2014-03-19 Wed 22:07]
+*** DONE create test videos, also record positions of worm segments
+CLOSED: [2014-03-20 Thu 22:02] SCHEDULED: <2014-03-19 Wed>
+*** TODO Collect intro, worm-learn and cortex creation into draft thesis.

Mercurial > cortex

comparison thesis/org/roadmap.org @ 430:5205535237fb