Mercurial > cortex
view thesis/org/roadmap.org @ 503:f648f8cc3b63
processing bib queue. num left: 0 (for now)
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Sat, 29 Mar 2014 23:55:47 -0400 |
parents | 5205535237fb |
children | 8e52a2802821 |
line wrap: on
line source
1 In order for this to be a reasonable thesis that I can be proud of,2 what are the /minimum/ number of things I need to get done?5 * worm OR hand registration6 - training from a few examples (2 to start out)7 - aligning the body with the scene8 - generating sensory data9 - matching previous labeled examples using dot-products or some10 other basic thing11 - showing that it works with different views13 * first draft14 - draft of thesis without bibliography or formatting15 - should have basic experiment and have full description of16 framework with code17 - review with Winston19 * final draft20 - implement stretch goals from Winston if possible21 - complete final formatting and submit23 * CORTEX24 DEADLINE: <2014-05-09 Fri>25 SHIT THAT'S IN 67 DAYS!!!27 ** program simple feature matching code for the worm's segments29 Subgoals:30 *** DONE Get cortex working again, run tests, no jmonkeyengine updates31 CLOSED: [2014-03-03 Mon 22:07] SCHEDULED: <2014-03-03 Mon>32 *** DONE get blender working again33 CLOSED: [2014-03-03 Mon 22:43] SCHEDULED: <2014-03-03 Mon>34 *** DONE make sparce touch worm segment in blender35 CLOSED: [2014-03-03 Mon 23:16] SCHEDULED: <2014-03-03 Mon>36 CLOCK: [2014-03-03 Mon 22:44]--[2014-03-03 Mon 23:16] => 0:3237 *** DONE make multi-segment touch worm with touch sensors and display38 CLOSED: [2014-03-03 Mon 23:54] SCHEDULED: <2014-03-03 Mon>40 *** DONE Make a worm wiggle and curl41 CLOSED: [2014-03-04 Tue 23:03] SCHEDULED: <2014-03-04 Tue>44 ** First draft46 Subgoals:47 *** Writeup new worm experiments.48 *** Triage implementation code and get it into chapter form.54 ** for today56 - guided worm :: control the worm with the keyboard. Useful for57 testing the body-centered recog scripts, and for58 preparing a cool demo video.60 - body-centered recognition :: detect actions using hard coded61 body-centered scripts.63 - cool demo video of the worm being moved and recognizing things ::64 will be a neat part of the thesis.66 - thesis export :: refactoring and organization of code so that it67 spits out a thesis in addition to the web page.69 - video alignment :: analyze the frames of a video in order to align70 the worm. Requires body-centered recognition. Can "cheat".72 - smoother actions :: use debugging controls to directly influence the73 demo actions, and to generate recoginition procedures.75 - degenerate video demonstration :: show the system recognizing a76 curled worm from dead on. Crowning achievement of thesis.78 ** Ordered from easiest to hardest80 Just report the positions of everything. I don't think that this81 necessairly shows anything usefull.83 Worm-segment vision -- you initialize a view of the worm, but instead84 of pixels you use labels via ray tracing. Has the advantage of still85 allowing for visual occlusion, but reliably identifies the objects,86 even without rainbow coloring. You can code this as an image.88 Same as above, except just with worm/non-worm labels.90 Color code each worm segment and then recognize them using blob91 detectors. Then you solve for the perspective and the action92 simultaneously.94 The entire worm can be colored the same, high contrast color against a95 nearly black background.97 "Rooted" vision. You give the exact coordinates of ONE piece of the98 worm, but the algorithm figures out the rest.100 More rooted vision -- start off the entire worm with one posistion.102 The right way to do alignment is to use motion over multiple frames to103 snap individual pieces of the model into place sharing and104 propragating the individual alignments over the whole model. We also105 want to limit the alignment search to just those actions we are106 prepared to identify. This might mean that I need some small "micro107 actions" such as the individual movements of the worm pieces.109 Get just the centers of each segment projected onto the imaging110 plane. (best so far).113 Repertoire of actions + video frames -->114 directed multi-frame-search alg121 !! Could also have a bounding box around the worm provided by122 filtering the worm/non-worm render, and use bbbgs. As a bonus, I get123 to include bbbgs in my thesis! Could finally do that recursive things124 where I make bounding boxes be those things that give results that125 give good bounding boxes. If I did this I could use a disruptive126 pattern on the worm.128 Re imagining using default textures is very simple for this system,129 but hard for others.132 Want to demonstrate, at minimum, alignment of some model of the worm133 to the video, and a lookup of the action by simulated perception.135 note: the purple/white points is a very beautiful texture, because136 when it moves slightly, the white dots look like they're137 twinkling. Would look even better if it was a darker purple. Also138 would look better more spread out.141 embed assumption of one frame of view, search by moving around in142 simulated world.144 Allowed to limit search by setting limits to a hemisphere around the145 imagined worm! This limits scale also.151 !! Limited search with worm/non-worm rendering.152 How much inverse kinematics do we have to do?153 What about cached (allowed state-space) paths, derived from labeled154 training. You have to lead from one to another.156 What about initial state? Could start the input videos at a specific157 state, then just match that explicitly.159 !! The training doesn't have to be labeled -- you can just move around160 for a while!!162 !! Limited search with motion based alignment.167 "play arounds" can establish a chain of linked sensoriums. Future168 matches must fall into one of the already experienced things, and once169 they do, it greatly limits the things that are possible in the future.172 frame differences help to detect muscle exertion.174 Can try to match on a few "representative" frames. Can also just have175 a few "bodies" in various states which we try to match.179 Paths through state-space have the exact same signature as180 simulation. BUT, these can be searched in parallel and don't interfere181 with each other.186 ** Final stretch up to First Draft188 *** DONE complete debug control of worm189 CLOSED: [2014-03-17 Mon 17:29] SCHEDULED: <2014-03-17 Mon>190 CLOCK: [2014-03-17 Mon 14:01]--[2014-03-17 Mon 17:29] => 3:28191 *** DONE add phi-space output to debug control192 CLOSED: [2014-03-17 Mon 17:42] SCHEDULED: <2014-03-17 Mon>193 CLOCK: [2014-03-17 Mon 17:31]--[2014-03-17 Mon 17:42] => 0:11195 *** DONE complete automatic touch partitioning196 CLOSED: [2014-03-18 Tue 21:43] SCHEDULED: <2014-03-18 Tue>197 *** DONE complete cyclic predicate198 CLOSED: [2014-03-19 Wed 16:34] SCHEDULED: <2014-03-18 Tue>199 CLOCK: [2014-03-19 Wed 13:16]--[2014-03-19 Wed 16:34] => 3:18200 *** DONE complete three phi-stream action predicatates; test them with debug control201 CLOSED: [2014-03-19 Wed 16:35] SCHEDULED: <2014-03-17 Mon>202 CLOCK: [2014-03-18 Tue 18:36]--[2014-03-18 Tue 21:43] => 3:07203 CLOCK: [2014-03-18 Tue 18:34]--[2014-03-18 Tue 18:36] => 0:02204 CLOCK: [2014-03-17 Mon 19:19]--[2014-03-17 Mon 21:19] => 2:00205 *** DONE build an automatic "do all the things" sequence.206 CLOSED: [2014-03-19 Wed 16:55] SCHEDULED: <2014-03-19 Wed>207 CLOCK: [2014-03-19 Wed 16:53]--[2014-03-19 Wed 16:55] => 0:02208 *** DONE implement proprioception based movement lookup in phi-space209 CLOSED: [2014-03-19 Wed 22:04] SCHEDULED: <2014-03-19 Wed>210 CLOCK: [2014-03-19 Wed 19:32]--[2014-03-19 Wed 22:04] => 2:32211 *** DONE make proprioception reference phi-space indexes212 CLOSED: [2014-03-19 Wed 22:47] SCHEDULED: <2014-03-19 Wed>213 CLOCK: [2014-03-19 Wed 22:07]216 *** DONE create test videos, also record positions of worm segments217 CLOSED: [2014-03-20 Thu 22:02] SCHEDULED: <2014-03-19 Wed>219 *** TODO Collect intro, worm-learn and cortex creation into draft thesis.