view thesis/org/roadmap.org @ 411:a331d5ff73e0

saving progress for the night. completed self-organizing touch, still working on stream predicates.
author Robert McIntyre <rlm@mit.edu>
date Tue, 18 Mar 2014 23:04:48 -0400
parents ea524e4d8f8d
children af7945c27474
line wrap: on
line source
1 In order for this to be a reasonable thesis that I can be proud of,
2 what are the /minimum/ number of things I need to get done?
5 * worm OR hand registration
6 - training from a few examples (2 to start out)
7 - aligning the body with the scene
8 - generating sensory data
9 - matching previous labeled examples using dot-products or some
10 other basic thing
11 - showing that it works with different views
13 * first draft
14 - draft of thesis without bibliography or formatting
15 - should have basic experiment and have full description of
16 framework with code
17 - review with Winston
19 * final draft
20 - implement stretch goals from Winston if possible
21 - complete final formatting and submit
23 * CORTEX
24 DEADLINE: <2014-05-09 Fri>
25 SHIT THAT'S IN 67 DAYS!!!
27 ** program simple feature matching code for the worm's segments
29 Subgoals:
30 *** DONE Get cortex working again, run tests, no jmonkeyengine updates
31 CLOSED: [2014-03-03 Mon 22:07] SCHEDULED: <2014-03-03 Mon>
32 *** DONE get blender working again
33 CLOSED: [2014-03-03 Mon 22:43] SCHEDULED: <2014-03-03 Mon>
34 *** DONE make sparce touch worm segment in blender
35 CLOSED: [2014-03-03 Mon 23:16] SCHEDULED: <2014-03-03 Mon>
36 CLOCK: [2014-03-03 Mon 22:44]--[2014-03-03 Mon 23:16] => 0:32
37 *** DONE make multi-segment touch worm with touch sensors and display
38 CLOSED: [2014-03-03 Mon 23:54] SCHEDULED: <2014-03-03 Mon>
40 *** DONE Make a worm wiggle and curl
41 CLOSED: [2014-03-04 Tue 23:03] SCHEDULED: <2014-03-04 Tue>
44 ** First draft
46 Subgoals:
47 *** Writeup new worm experiments.
48 *** Triage implementation code and get it into chapter form.
54 ** for today
56 - guided worm :: control the worm with the keyboard. Useful for
57 testing the body-centered recog scripts, and for
58 preparing a cool demo video.
60 - body-centered recognition :: detect actions using hard coded
61 body-centered scripts.
63 - cool demo video of the worm being moved and recognizing things ::
64 will be a neat part of the thesis.
66 - thesis export :: refactoring and organization of code so that it
67 spits out a thesis in addition to the web page.
69 - video alignment :: analyze the frames of a video in order to align
70 the worm. Requires body-centered recognition. Can "cheat".
72 - smoother actions :: use debugging controls to directly influence the
73 demo actions, and to generate recoginition procedures.
75 - degenerate video demonstration :: show the system recognizing a
76 curled worm from dead on. Crowning achievement of thesis.
78 ** Ordered from easiest to hardest
80 Just report the positions of everything. I don't think that this
81 necessairly shows anything usefull.
83 Worm-segment vision -- you initialize a view of the worm, but instead
84 of pixels you use labels via ray tracing. Has the advantage of still
85 allowing for visual occlusion, but reliably identifies the objects,
86 even without rainbow coloring. You can code this as an image.
88 Same as above, except just with worm/non-worm labels.
90 Color code each worm segment and then recognize them using blob
91 detectors. Then you solve for the perspective and the action
92 simultaneously.
94 The entire worm can be colored the same, high contrast color against a
95 nearly black background.
97 "Rooted" vision. You give the exact coordinates of ONE piece of the
98 worm, but the algorithm figures out the rest.
100 More rooted vision -- start off the entire worm with one posistion.
102 The right way to do alignment is to use motion over multiple frames to
103 snap individual pieces of the model into place sharing and
104 propragating the individual alignments over the whole model. We also
105 want to limit the alignment search to just those actions we are
106 prepared to identify. This might mean that I need some small "micro
107 actions" such as the individual movements of the worm pieces.
109 Get just the centers of each segment projected onto the imaging
110 plane. (best so far).
113 Repertoire of actions + video frames -->
114 directed multi-frame-search alg
121 !! Could also have a bounding box around the worm provided by
122 filtering the worm/non-worm render, and use bbbgs. As a bonus, I get
123 to include bbbgs in my thesis! Could finally do that recursive things
124 where I make bounding boxes be those things that give results that
125 give good bounding boxes. If I did this I could use a disruptive
126 pattern on the worm.
128 Re imagining using default textures is very simple for this system,
129 but hard for others.
132 Want to demonstrate, at minimum, alignment of some model of the worm
133 to the video, and a lookup of the action by simulated perception.
135 note: the purple/white points is a very beautiful texture, because
136 when it moves slightly, the white dots look like they're
137 twinkling. Would look even better if it was a darker purple. Also
138 would look better more spread out.
141 embed assumption of one frame of view, search by moving around in
142 simulated world.
144 Allowed to limit search by setting limits to a hemisphere around the
145 imagined worm! This limits scale also.
151 !! Limited search with worm/non-worm rendering.
152 How much inverse kinematics do we have to do?
153 What about cached (allowed state-space) paths, derived from labeled
154 training. You have to lead from one to another.
156 What about initial state? Could start the input videos at a specific
157 state, then just match that explicitly.
159 !! The training doesn't have to be labeled -- you can just move around
160 for a while!!
162 !! Limited search with motion based alignment.
167 "play arounds" can establish a chain of linked sensoriums. Future
168 matches must fall into one of the already experienced things, and once
169 they do, it greatly limits the things that are possible in the future.
172 frame differences help to detect muscle exertion.
174 Can try to match on a few "representative" frames. Can also just have
175 a few "bodies" in various states which we try to match.
179 Paths through state-space have the exact same signature as
180 simulation. BUT, these can be searched in parallel and don't interfere
181 with each other.
186 ** Final stretch up to First Draft
188 *** DONE complete debug control of worm
189 CLOSED: [2014-03-17 Mon 17:29] SCHEDULED: <2014-03-17 Mon>
190 CLOCK: [2014-03-17 Mon 14:01]--[2014-03-17 Mon 17:29] => 3:28
191 *** DONE add phi-space output to debug control
192 CLOSED: [2014-03-17 Mon 17:42] SCHEDULED: <2014-03-17 Mon>
193 CLOCK: [2014-03-17 Mon 17:31]--[2014-03-17 Mon 17:42] => 0:11
195 *** DONE complete automatic touch partitioning
196 CLOSED: [2014-03-18 Tue 21:43] SCHEDULED: <2014-03-18 Tue>
197 *** TODO complete cyclic predicate
198 SCHEDULED: <2014-03-18 Tue>
199 *** TODO complete three phi-stream action predicatates; test them with debug control
200 SCHEDULED: <2014-03-17 Mon>
201 CLOCK: [2014-03-18 Tue 18:36]
202 CLOCK: [2014-03-18 Tue 18:34]--[2014-03-18 Tue 18:36] => 0:02
203 CLOCK: [2014-03-17 Mon 19:19]--[2014-03-17 Mon 21:19] => 2:00
204 *** TODO create test videos, also record positions of worm segments
205 SCHEDULED: <2014-03-17 Mon>
206 *** TODO complete proprioception based movement lookup in phi-space
207 SCHEDULED: <2014-03-17 Mon>
210 *** TODO Collect intro, worm-learn and cortex creation into draft thesis.