Mercurial > cortex

comparison winston-intro.txt @ 302:7e3938f40c52

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added winston letter, corrected source listing for integration
author Robert McIntyre <rlm@mit.edu>
date Fri, 17 Feb 2012 13:03:58 -0700
parents
children 19c43ec6958d
comparison
equal deleted inserted replaced
301:4203c2140b95 302:7e3938f40c52
1 Dear Professor Winston,
2
3 I'm ready for you to look through the work that I've done so far. It's
4 a sequence of posts describing the different simulated senses I've
5 implemented, with videos.
6
7 It's "blocks world reloaded", because like you say, you need multiple
8 senses to enable intelligence.
9
10 Please look through the videos and skim the text and tell me what
11 you think:
12
13 Introduction:
14 http://aurellem.org/cortex/html/intro.html
15 http://aurellem.org/cortex/html/sense.html
16
17 http://aurellem.org/cortex/html/body.html -- simulated physical bodies
18 http://aurellem.org/cortex/html/vision.html -- simulated eyes
19 http://aurellem.org/cortex/html/hearing.html -- simulated ears
20 http://aurellem.org/cortex/html/touch.html -- simulated skin/hairs
21 http://aurellem.org/cortex/html/proprioception.html -- simulated proprioception
22 http://aurellem.org/cortex/html/movement.html -- simulated muscles
23 http://aurellem.org/cortex/html/integration.html -- full demonstration
24
25 In particular, look at the video at
26 http://aurellem.org/cortex/html/integration.html. It shows a
27 simulated hand equipped with all of the senses I've built so far.
28
29 There's some more background information and full source code at
30 http://aurellem.org
31
32 If you can't see a video, let me know and I'll upload it to YouTube so
33 you can see it.
34
35
36
37
38 Now, I need your help moving forward. Can I use this work as a base
39 for a Masters thesis with you when I come back to MIT this coming Fall?
40 What critiques and project ideas do you have after looking through
41 what I've done so far?
42
43 I have some ideas on where I can go with this project but I think you
44 will have some better ones.
45
46 Here are some possible projects I might do with this as a base that I
47 think would be worthy Masters projects.
48
49 - HACKER for writing muscle-control programs : Presented with
50 low-level muscle control/ sense API, generate higher level programs
51 for accomplishing various stated goals. Example goals might be
52 "extend all your fingers" or "move your hand into the area with
53 blue light" or "decrease the angle of this joint". It would be
54 like Sussman's HACKER, except it would operate with much more data
55 in a more realistic world. Start off with "calisthenics" to
56 develop subroutines over the motor control API. This would be the
57 "spinal chord" of a more intelligent creature.
58
59 - Create hundreds of small creatures and have them do simple
60 simulated swarm tasks.
61
62 - A joint that learns what sort of joint it (cone, point-to-point,
63 hinge, etc.) is by correlating exploratory muscle movements with
64 vision.
65
66 - Something with cross-modal clustering using the rich sense
67 data. This might prove difficult due to the higher dimensionality
68 of my senses.
69
70 - Simulated Imagination --- this would involve a creature with an
71 effector which creates an /entire new sub-simulation/ where the
72 creature has direct control over placement/creation of objects via
73 simulated telekinesis. The creature observes this sub-world through
74 it's normal senses and uses its observations to make predictions
75 about it's top level world.
76
77 - Hook it up with Genesis --- I could make a "semantic effector"
78 which marks objects/sensory states with semantic information. In
79 conjunction with Simulated Imagination, and HACKER-like motor
80 control, Genesis might be able to ask simple questions like "stack
81 two blocks together and hit the top one with your hand; does the
82 bottom block move?" and the system could answer "yes". This would
83 be rather complicated to do and involves many of the above
84 projects, but there may be a way to scale it down to Master's
85 thesis size.
86
87 - Make a virtual computer in the virtual world which with which the
88 creature interacts using its fingers to press keys on a virtual
89 keyboard. The creature can access the internet, watch videos, take
90 over the world, anything it wants. (This is probably not worthy of
91 a Masters project, I just thought it was a neat idea. It's possible
92 to use videos/etc in the simulated world at any rate.)
93
94
95 I can't wait to hear your critiques and ideas. If you think I
96 shouldn't use this system as a base and should instead do something
97 else, that's fine too.
98
99 On a related note, can I be considered for the position of TA next
100 year for 6.034 or 6.xxx?
101
102 sincerely,
103 --Robert McIntyre