Mercurial > cortex

comparison winston-intro.txt @ 305:19c43ec6958d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
modified cover letter, wonston-letter, and began uploading files to youtube for backup
author Robert McIntyre <rlm@mit.edu>
date Sat, 18 Feb 2012 10:28:14 -0700
parents 7e3938f40c52
children 71946ec07be9
comparison
equal deleted inserted replaced
304:2dfebf71053c 305:19c43ec6958d
1 Dear Professor Winston, 1 Dear Professor Winston,
2 2
3 I'm ready for you to look through the work that I've done so far. It's 3 I've finished the first part of my project, building a framework for
4 a sequence of posts describing the different simulated senses I've 4 virtual sensate creatures; I would like your help evaluating what I've
5 implemented, with videos. 5 done so far, and deciding what to do next.
6 6
7 It's "blocks world reloaded", because like you say, you need multiple 7 For the work I've done so far, I compiled the results into short
8 senses to enable intelligence. 8 articles that explain how I implemented each sense, with videos that
9 show each sense in action. Please look through the articles, in
10 particular the video showcase, and tell me what you think.
9 11
10 Please look through the videos and skim the text and tell me what 12 Video Showcase : http://aurellem.localhost/cortex/org/cover.html
11 you think:
12 13
13 Introduction: 14 Introduction:
14 http://aurellem.org/cortex/html/intro.html 15 http://aurellem.org/cortex/html/intro.html
15 http://aurellem.org/cortex/html/sense.html 16 http://aurellem.org/cortex/html/sense.html
16 17
17 http://aurellem.org/cortex/html/body.html -- simulated physical bodies 18 Physical Bodies : http://aurellem.org/cortex/html/body.html
18 http://aurellem.org/cortex/html/vision.html -- simulated eyes 19 Vision : http://aurellem.org/cortex/html/vision.html
19 http://aurellem.org/cortex/html/hearing.html -- simulated ears 20 Hearing : http://aurellem.org/cortex/html/hearing.html
20 http://aurellem.org/cortex/html/touch.html -- simulated skin/hairs 21 Touch : http://aurellem.org/cortex/html/touch.html
21 http://aurellem.org/cortex/html/proprioception.html -- simulated proprioception 22 Proprioception : http://aurellem.org/cortex/html/proprioception.html
22 http://aurellem.org/cortex/html/movement.html -- simulated muscles 23 Muscles : http://aurellem.org/cortex/html/movement.html
23 http://aurellem.org/cortex/html/integration.html -- full demonstration 24 Full Demonstration : http://aurellem.org/cortex/html/integration.html
24 25
25 In particular, look at the video at 26 I think this work could be a fruitful foundation for a Master's
26 http://aurellem.org/cortex/html/integration.html. It shows a 27 thesis, so in particular, I'd like critiques, suggestions, and project
27 simulated hand equipped with all of the senses I've built so far. 28 ideas. For example, here are some projects I think would be worthy, in
29 increasing order of complexity:
28 30
29 There's some more background information and full source code at 31 * Create a self-powered joint that can determine its range of
30 http://aurellem.org 32 motion and joint type (hinge, cone, point-to-point, etc.) by
33 making exploratory muscle movements and observing their effect.
31 34
32 If you can't see a video, let me know and I'll upload it to YouTube so 35 * Develop an agent that writes and debugs low-level motor control
33 you can see it. 36 programs to achieve simple goals like "look at the light" or
37 "extend all of your fingers". These simple "calisthenic"
38 programs could then be combined to form more elaborate
39 procedures of motion, which in turn could be the basic
40 instinctive reflexes in the "spinal cord" of some more advanced
41 creature. (like Sussman's HACKER program but in a richer world)
34 42
43 * Program a group of creatures that cooperate with each
44 other. Because the creatures would be simulated, I could
45 investigate computationally complex rules of behavior which
46 still, from the group's point of view, would happen in "real
47 time". Interactions could be as simple as cellular organisms
48 communicating via flashing lights, or as complex as humanoids
49 completing social tasks, etc.
35 50
51 * Simulated Imagination -- this would involve a creature with an
52 effector which creates an entire new sub-simulation where the
53 creature has direct control over placement/creation of objects
54 via simulated telekinesis. The creature observes this sub-world
55 through it's normal senses and uses its observations to make
56 predictions about it's top level world.
36 57
58 * Integrate the simulated world with Genesis, so that Genesis
59 could use the simulated world to answer questions about a
60 proposed physical scenario. For example "You stack two blocks
61 together, then hit the bottom block with your hand. Does the top
62 block move?". This project is complicated and very large in
63 scope, but it could be narrowed to focus on a single key
64 aspect. For example, one key aspect of turning a scenario into a
65 simulation is knowing when you're constructing "typical" or
66 "atypical" examples of the scenario. So, a narrower project
67 might simply learn about the edge cases of different scenarios
68 (e.g. "A block stacked on top of another block is usually
69 stable, provided the bottom block is large enough, and is not
70 moving, and is level, etc."). With this knowledge, this kind of
71 program could aid Genesis not only in answering common-sense
72 questions, but in refining them: "A block is stacked on top of
73 another block. Is it stable?"; "Usually, but do you know if the
74 bottom block is slanted?", etc.
37 75
38 Now, I need your help moving forward. Can I use this work as a base 76 These are some ideas, but I think you can come up with better ones. I
39 for a Masters thesis with you when I come back to MIT this coming Fall? 77 can't wait to hear your critiques and suggestions.
40 What critiques and project ideas do you have after looking through
41 what I've done so far?
42 78
43 I have some ideas on where I can go with this project but I think you 79 Finally, regarding next year at MIT, can I be considered for the
44 will have some better ones. 80 position of TA for 6.034 or 6.xxx? Also, do you want me to return Fall
81 or Summer?
45 82
46 Here are some possible projects I might do with this as a base that I 83 Sincerely,
47 think would be worthy Masters projects.
48 84
49 - HACKER for writing muscle-control programs : Presented with
50 low-level muscle control/ sense API, generate higher level programs
51 for accomplishing various stated goals. Example goals might be
52 "extend all your fingers" or "move your hand into the area with
53 blue light" or "decrease the angle of this joint". It would be
54 like Sussman's HACKER, except it would operate with much more data
55 in a more realistic world. Start off with "calisthenics" to
56 develop subroutines over the motor control API. This would be the
57 "spinal chord" of a more intelligent creature.
58
59 - Create hundreds of small creatures and have them do simple
60 simulated swarm tasks.
61
62 - A joint that learns what sort of joint it (cone, point-to-point,
63 hinge, etc.) is by correlating exploratory muscle movements with
64 vision.
65
66 - Something with cross-modal clustering using the rich sense
67 data. This might prove difficult due to the higher dimensionality
68 of my senses.
69
70 - Simulated Imagination --- this would involve a creature with an
71 effector which creates an /entire new sub-simulation/ where the
72 creature has direct control over placement/creation of objects via
73 simulated telekinesis. The creature observes this sub-world through
74 it's normal senses and uses its observations to make predictions
75 about it's top level world.
76
77 - Hook it up with Genesis --- I could make a "semantic effector"
78 which marks objects/sensory states with semantic information. In
79 conjunction with Simulated Imagination, and HACKER-like motor
80 control, Genesis might be able to ask simple questions like "stack
81 two blocks together and hit the top one with your hand; does the
82 bottom block move?" and the system could answer "yes". This would
83 be rather complicated to do and involves many of the above
84 projects, but there may be a way to scale it down to Master's
85 thesis size.
86
87 - Make a virtual computer in the virtual world which with which the
88 creature interacts using its fingers to press keys on a virtual
89 keyboard. The creature can access the internet, watch videos, take
90 over the world, anything it wants. (This is probably not worthy of
91 a Masters project, I just thought it was a neat idea. It's possible
92 to use videos/etc in the simulated world at any rate.)
93
94
95 I can't wait to hear your critiques and ideas. If you think I
96 shouldn't use this system as a base and should instead do something
97 else, that's fine too.
98
99 On a related note, can I be considered for the position of TA next
100 year for 6.034 or 6.xxx?
101
102 sincerely,
103 --Robert McIntyre 85 --Robert McIntyre