Mercurial > thoughts

     1 <?xml version="1.0" encoding="utf-8"?>

     2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

     3                "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

     4 <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">

     5 <head>

     6 <title>Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview</title>

     7 <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>

     8 <meta name="title" content="Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview"/>

     9 <meta name="generator" content="Org-mode"/>

    10 <meta name="generated" content="2013-10-04 18:49:53 UTC"/>

    11 <meta name="author" content="Dylan Holmes"/>

    12 <meta name="description" content=""/>

    13 <meta name="keywords" content=""/>

    14 <style type="text/css">

    15  <!--/*--><![CDATA[/*><!--*/

    16   html { font-family: Times, serif; font-size: 12pt; }

    17   .title  { text-align: center; }

    18   .todo   { color: red; }

    19   .done   { color: green; }

    20   .tag    { background-color: #add8e6; font-weight:normal }

    21   .target { }

    22   .timestamp { color: #bebebe; }

    23   .timestamp-kwd { color: #5f9ea0; }

    24   .right  {margin-left:auto; margin-right:0px;  text-align:right;}

    25   .left   {margin-left:0px;  margin-right:auto; text-align:left;}

    26   .center {margin-left:auto; margin-right:auto; text-align:center;}

    27   p.verse { margin-left: 3% }

    28   pre {

    29 	border: 1pt solid #AEBDCC;

    30 	background-color: #F3F5F7;

    31 	padding: 5pt;

    32 	font-family: courier, monospace;

    33         font-size: 90%;

    34         overflow:auto;

    35   }

    36   table { border-collapse: collapse; }

    37   td, th { vertical-align: top;  }

    38   th.right  { text-align:center;  }

    39   th.left   { text-align:center;   }

    40   th.center { text-align:center; }

    41   td.right  { text-align:right;  }

    42   td.left   { text-align:left;   }

    43   td.center { text-align:center; }

    44   dt { font-weight: bold; }

    45   div.figure { padding: 0.5em; }

    46   div.figure p { text-align: center; }

    47   div.inlinetask {

    48     padding:10px;

    49     border:2px solid gray;

    50     margin:10px;

    51     background: #ffffcc;

    52   }

    53   textarea { overflow-x: auto; }

    54   .linenr { font-size:smaller }

    55   .code-highlighted {background-color:#ffff00;}

    56   .org-info-js_info-navigation { border-style:none; }

    57   #org-info-js_console-label { font-size:10px; font-weight:bold;

    58                                white-space:nowrap; }

    59   .org-info-js_search-highlight {background-color:#ffff00; color:#000000;

    60                                  font-weight:bold; }

    61   /*]]>*/-->

    62 </style>

    63 <link rel="stylesheet" type="text/css" href="../css/sloman.css" /> 

    64 <script type="text/javascript">

    65 <!--/*--><![CDATA[/*><!--*/

    66  function CodeHighlightOn(elem, id)

    67  {

    68    var target = document.getElementById(id);

    69    if(null != target) {

    70      elem.cacheClassElem = elem.className;

    71      elem.cacheClassTarget = target.className;

    72      target.className = "code-highlighted";

    73      elem.className   = "code-highlighted";

    74    }

    75  }

    76  function CodeHighlightOff(elem, id)

    77  {

    78    var target = document.getElementById(id);

    79    if(elem.cacheClassElem)

    80      elem.className = elem.cacheClassElem;

    81    if(elem.cacheClassTarget)

    82      target.className = elem.cacheClassTarget;

    83  }

    84 /*]]>*///-->

    85 </script>

    86 

    87 </head>

    88 <body>

    89 

    90 

    91 <div id="content">

    92 <h1 class="title">Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview</h1>

    93 

    94 

    95 <blockquote>

    96 

    97 

    98 

    99 

   100 

   101 

   102 

   103 

   104 

   105 

   106 

   107 

   108 

   109 

   110 

   111 

   112 <p>

   113 <b>Editor's note:</b> This is a working draft transcript which I made of

   114 <a href="http://www.youtube.com/watch?feature=player_detailpage&amp;v=iuH8dC7Snno">this nice interview</a> of Aaron Sloman. Having just finished one

   115 iteration of transcription, I still need to go in and clean up the

   116 formatting and fix the parts that I misheard, so you can expect the

   117 text to improve significantly in the near future.

   118 </p>

   119 <p>

   120 To the extent that this is my work, you have my permission to make

   121 copies of this transcript for your own purposes. Also, feel free to

   122 e-mail me with comments or corrections.

   123 </p>

   124 <p>

   125 You can send mail to <code>transcript@aurellem.org</code>.

   126 </p>

   127 <p>

   128 Cheers,

   129 </p>

   130 <p>

   131 &mdash;Dylan

   132 </p>

   133 </blockquote>

   134 

   135 

   136 

   137 

   138 

   139 <div id="table-of-contents">

   140 <h2>Table of Contents</h2>

   141 <div id="text-table-of-contents">

   142 <ul>

   143 <li><a href="#sec-1">1 Introduction</a>

   144 <ul>

   145 <li><a href="#sec-1-1">1.1 Aaron Sloman evolves into a philosopher of AI</a></li>

   146 <li><a href="#sec-1-2">1.2 AI is hard, in part because there are tempting non-problems.</a></li>

   147 </ul>

   148 </li>

   149 <li><a href="#sec-2">2 What problems of intelligence did evolution solve?</a>

   150 <ul>

   151 <li><a href="#sec-2-1">2.1 Intelligence consists of solutions to many evolutionary problems; no single development (e.g. communication) was key to human-level intelligence.</a></li>

   152 <li><a href="#sec-2-2">2.2 Speculation about how communication might have evolved from internal lanagues.</a></li>

   153 </ul>

   154 </li>

   155 <li><a href="#sec-3">3 How do language and internal states relate to AI?</a>

   156 <ul>

   157 <li><a href="#sec-3-1">3.1 In AI, false assumptions can lead investigators astray.</a></li>

   158 <li><a href="#sec-3-2">3.2 Example: Vision is not just about finding surfaces, but about finding affordances.</a></li>

   159 <li><a href="#sec-3-3">3.3 Online and offline intelligence</a></li>

   160 <li><a href="#sec-3-4">3.4 Example: Even toddlers use sophisticated geometric knowledge</a></li>

   161 </ul>

   162 </li>

   163 <li><a href="#sec-4">4 Animal intelligence</a>

   164 <ul>

   165 <li><a href="#sec-4-1">4.1 The priority is <i>cataloguing</i> what competences have evolved, not ranking them.</a></li>

   166 <li><a href="#sec-4-2">4.2 AI can be used to test philosophical theories</a></li>

   167 </ul>

   168 </li>

   169 <li><a href="#sec-5">5 Is abstract general intelligence feasible?</a>

   170 <ul>

   171 <li><a href="#sec-5-1">5.1 It's misleading to compare the brain and its neurons to a computer made of transistors</a></li>

   172 <li><a href="#sec-5-2">5.2 For example, brains may rely heavily on chemical information processing</a></li>

   173 <li><a href="#sec-5-3">5.3 Brain algorithms may simply be optimized for certain kinds of information processing other than bit manipulations</a></li>

   174 <li><a href="#sec-5-4">5.4 Example: find the shortest path by dangling strings</a></li>

   175 <li><a href="#sec-5-5">5.5 In sum, we know surprisingly little about the kinds of problems that evolution solved, and the manner in which they were solved.</a></li>

   176 </ul>

   177 </li>

   178 <li><a href="#sec-6">6 A singularity of cognitive catch-up</a>

   179 <ul>

   180 <li><a href="#sec-6-1">6.1 What if it will take a lifetime to learn enough to make something new?</a></li>

   181 </ul>

   182 </li>

   183 <li><a href="#sec-7">7 Spatial reasoning: a difficult problem</a>

   184 <ul>

   185 <li><a href="#sec-7-1">7.1 Example: Spatial proof that the angles of any triangle add up to a half-circle</a></li>

   186 <li><a href="#sec-7-2">7.2 Geometric results are fundamentally different than experimental results in chemistry or physics.</a></li>

   187 </ul>

   188 </li>

   189 <li><a href="#sec-8">8 Is near-term artificial general intelligence likely?</a>

   190 <ul>

   191 <li><a href="#sec-8-1">8.1 Two interpretations: a single mechanism for all problems, or many mechanisms unified in one program.</a></li>

   192 </ul>

   193 </li>

   194 <li><a href="#sec-9">9 Abstract General Intelligence impacts</a></li>

   195 </ul>

   196 </div>

   197 </div>

   198 

   199 <div id="outline-container-1" class="outline-2">

   200 <h2 id="sec-1"><span class="section-number-2">1</span> Introduction</h2>

   201 <div class="outline-text-2" id="text-1">

   202 

   203 

   204 

   205 </div>

   206 

   207 <div id="outline-container-1-1" class="outline-3">

   208 <h3 id="sec-1-1"><span class="section-number-3">1.1</span> Aaron Sloman evolves into a philosopher of AI</h3>

   209 <div class="outline-text-3" id="text-1-1">

   210 

   211 <p>[0:09] My name is Aaron Sloman. My first degree many years ago in

   212 Capetown University was in Physics and Mathematics, and I intended to

   213 go and be a mathematician. I came to Oxford and encountered

   214 philosophers &mdash; I had started reading philosophy and discussing

   215 philosophy before then, and then I found that there were philosophers

   216 who said things about mathematics that I thought were wrong, so

   217 gradually got more and more involved in [philosophy] discussions and

   218 switched to doing philosophy DPhil. Then I became a philosophy

   219 lecturer and about six years later, I was introduced to artificial

   220 intelligence when I was a lecturer at Sussex University in philosophy

   221 and I very soon became convinced that the best way to make progress in

   222 both areas of philosophy (including philosophy of mathematics which I

   223 felt i hadn't dealt with adequately in my DPhil) about the philosophy

   224 of mathematics, philosophy of mind, philsophy of language and all

   225 those things&mdash;the best way was to try to design and test working

   226 fragments of mind and maybe eventually put them all together but

   227 initially just working fragments that would do various things.

   228 </p>

   229 <p>

   230 [1:12] And I learned to program and ~ with various other people

   231 including ~Margaret Boden whom you've interviewed, developed&mdash;helped

   232 develop an undergraduate degree in AI and other things and also began

   233 to do research in AI and so on which I thought of as doing philosophy,

   234 primarily.

   235 </p>

   236 <p>

   237 [1:29] And then I later moved to the University of Birmingham and I

   238 was there &mdash; I came in 1991 &mdash; and I've been retired for a while but

   239 I'm not interested in golf or gardening so I just go on doing full

   240 time research and my department is happy to keep me on without paying

   241 me and provide space and resources and I come, meeting bright people

   242 at conferences and try to learn and make progress if I can.

   243 </p>

   244 </div>

   245 

   246 </div>

   247 

   248 <div id="outline-container-1-2" class="outline-3">

   249 <h3 id="sec-1-2"><span class="section-number-3">1.2</span> AI is hard, in part because there are tempting non-problems.</h3>

   250 <div class="outline-text-3" id="text-1-2">

   251 

   252 

   253 <p>

   254 One of the things I learnt and understood more and more over the many

   255 years &mdash; forty years or so since I first encountered AI &mdash; is how

   256 hard the problems are, and in part that's because it's very often

   257 tempting to <i>think</i> the problem is something different from what it

   258 actually is, and then people design solutions to the non-problems, and

   259 I think of most of my work now as just helping to clarify what the

   260 problems are: what is it that we're trying to explain &mdash; and maybe

   261 this is leading into what you wanted to talk about:

   262 </p>

   263 <p>

   264 I now think that one of the ways of getting a deep understanding of

   265 that is to find out what were the problems that biological evolution

   266 solved, because we are a product of <i>many</i> solutions to <i>many</i>

   267 problems, and if we just try to go in and work out what the whole

   268 system is doing, we may get it all wrong, or badly wrong.

   269 </p>

   270 

   271 </div>

   272 </div>

   273 

   274 </div>

   275 

   276 <div id="outline-container-2" class="outline-2">

   277 <h2 id="sec-2"><span class="section-number-2">2</span> What problems of intelligence did evolution solve?</h2>

   278 <div class="outline-text-2" id="text-2">

   279 

   280 

   281 

   282 </div>

   283 

   284 <div id="outline-container-2-1" class="outline-3">

   285 <h3 id="sec-2-1"><span class="section-number-3">2.1</span> Intelligence consists of solutions to many evolutionary problems; no single development (e.g. communication) was key to human-level intelligence.</h3>

   286 <div class="outline-text-3" id="text-2-1">

   287 

   288 

   289 <p>

   290 [2:57] Well, first I would challenge that we are the dominant

   291 species. I know it looks like that but actually if you count biomass,

   292 if you count number of species, if you count number of individuals,

   293 the dominant species are microbes &mdash; maybe not one of them but anyway

   294 they're the ones who dominate in that sense, and furthermore we are

   295 mostly &mdash; we are largely composed of microbes, without which we

   296 wouldn't survive.

   297 </p>

   298 

   299 <p>

   300 [3:27] But there are things that make humans (you could say) best at

   301 those things, or worst at those things, but it's a combination.  And I

   302 think it was a collection of developments of which there isn't any

   303 single one. [] there might be, some people say, human language which

   304 changed everything. By our human language, they mean human

   305 communication in words, but I think that was a later development from

   306 what must have started as the use of <i>internal</i> forms of

   307 representation &mdash; which are there in nest-building birds, in

   308 pre-verbal children, in hunting mammals &mdash; because you can't take in

   309 information about a complex structured environment in which things can

   310 change and you may have to be able to work out what's possible and

   311 what isn't possible, without having some way of representing the

   312 components of the environment, their relationships, the kinds of

   313 things they can and can't do, the kinds of things you might or might

   314 not be able to do &mdash; and <i>that</i> kind of capability needs internal

   315 languages, and I and colleagues [at Birmingham] have been referring to

   316 them as generalized languages because some people object to

   317 referring&hellip;to using language to refer to something that isn't used

   318 for communication. But from that viewpoint, not only humans but many

   319 other animals developed abilities to do things to their environment to

   320 make them more friendly to themselves, which depended on being able to

   321 represent possible futures, possible actions, and work out what's the

   322 best thing to do.

   323 </p>

   324 <p>

   325 [5:13] And nest-building in corvids for instance&mdash;crows, magpies,

   326  [hawks], and so on &mdash; are way beyond what current robots can do, and

   327  in fact I think most humans would be challenged if they had to go and

   328  find a collection of twigs, one at a time, maybe bring them with just

   329  one hand &mdash; or with your mouth &mdash; and assemble them into a

   330  structure that, you know, is shaped like a nest, and is fairly rigid,

   331  and you could trust your eggs in them when wind blows. But they're

   332  doing it, and so &hellip; they're not our evolutionary ancestors, but

   333  they're an indication &mdash; and that example is an indication &mdash; of

   334  what must have evolved in order to provide control over the

   335  environment in <i>that</i> species.

   336 </p>

   337 </div>

   338 

   339 </div>

   340 

   341 <div id="outline-container-2-2" class="outline-3">

   342 <h3 id="sec-2-2"><span class="section-number-3">2.2</span> Speculation about how communication might have evolved from internal lanagues.</h3>

   343 <div class="outline-text-3" id="text-2-2">

   344 

   345 <p>[5:56] And I think hunting mammals, fruit-picking mammals, mammals

   346 that can rearrange parts of the environment, provide shelters, needed

   347 to have &hellip;. also needed to have ways of representing possible

   348 futures, not just what's there in the environment. I think at a later

   349 stage, that developed into a form of communication, or rather the

   350 <i>internal</i> forms of representation became usable as a basis for

   351 providing [context] to be communicated. And that happened, I think,

   352 initially through performing actions that expressed intentions, and

   353 probably led to situtations where an action (for instance, moving some

   354 large object) was performed more easily, or more successfully, or more

   355 accurately if it was done collaboratively. So someone who had worked

   356 out what to do might start doing it, and then a conspecific might be

   357 able to work out what the intention is, because that person has the

   358 <i>same</i> forms of representation and can build theories about what's

   359 going on, and might then be able to help.

   360 </p>

   361 <p>

   362 [7:11] You can imagine that if that started happening more (a lot of

   363 collaboration based on inferred intentions and plans) then sometimes

   364 the inferences might be obscure and difficult, so the <i>actions</i> might

   365 be enhanced to provide signals as to what the intention is, and what

   366 the best way is to help, and so on.

   367 </p>

   368 <p>

   369 [7:35] So, this is all handwaving and wild speculation, but I think

   370 it's consistent with a large collection of facts which one can look at

   371 &mdash; and find if one looks for them, but one won't know if [some]one

   372 doesn't look for them &mdash; about the way children, for instance, who

   373 can't yet talk, communicate, and the things they'll do, like going to

   374 the mother and turning the face to point in the direction where the

   375 child wants it to look and so on; that's an extreme version of action

   376 indicating intention.

   377 </p>

   378 <p>

   379 [8:03] Anyway. That's a very long roundabout answer to one conjecture

   380 that the use of communicative language is what gave humans their

   381 unique power to create and destroy and whatever, and I'm saying that

   382 if by that you mean <i>communicative</i> language, then I'm saying there

   383 was something before that which was <i>non</i>-communicative language, and I

   384 suspect that noncommunicative language continues to play a deep role

   385 in <i>all</i> human perception &mdash;in mathematical and scientific reasoning, in

   386 problem solving &mdash; and we don't understand very much about it.

   387 </p>

   388 <p>

   389 [8:48]

   390 I'm sure there's a lot more to be said about the development of

   391 different kinds of senses, the development of brain structures and

   392 mechanisms is above all that, but perhaps I've droned on long enough

   393 on that question.

   394 </p>

   395 

   396 </div>

   397 </div>

   398 

   399 </div>

   400 

   401 <div id="outline-container-3" class="outline-2">

   402 <h2 id="sec-3"><span class="section-number-2">3</span> How do language and internal states relate to AI?</h2>

   403 <div class="outline-text-2" id="text-3">

   404 

   405 

   406 <p>

   407 [9:09] Well, I think most of the human and animal capabilities that

   408 I've been referring to are not yet to be found in current robots or

   409 [computing] systems, and I think there are two reasons for that: one

   410 is that it's intrinsically very difficult; I think that in particular

   411 it may turn out that the forms of information processing that one can

   412 implement on digital computers as we currently know them may not be as

   413 well suited to performing some of these tasks as other kinds of

   414 computing about which we don't know so much &mdash; for example, I think

   415 there may be important special features about <i>chemical</i> computers

   416 which we might [talk about in a little bit? find out about]. 

   417 </p>

   418 

   419 </div>

   420 

   421 <div id="outline-container-3-1" class="outline-3">

   422 <h3 id="sec-3-1"><span class="section-number-3">3.1</span> In AI, false assumptions can lead investigators astray.</h3>

   423 <div class="outline-text-3" id="text-3-1">

   424 

   425 <p>[9:57] So, one of the problems then is that the tasks are hard &hellip; but

   426 there's a deeper problem as to why AI hasn't made a great deal of

   427 progress on these problems that I'm talking about, and that is that

   428 most AI researchers assume things&mdash;and this is not just AI

   429 researchers, but [also] philsophers, and psychologists, and people

   430 studying animal behavior&mdash;make assumptions about what it is that

   431 animals or humans do, for instance make assumptions about what vision

   432 is for, or assumptions about what motivation is and how motivation

   433 works, or assumptions about how learning works, and then they try ---

   434 the AI people try &mdash; to model [or] build systems that perform those

   435 assumed functions. So if you get the <i>functions</i> wrong, then even if

   436 you implement some of the functions that you're trying to implement,

   437 they won't necessarily perform the tasks that the initial objective

   438 was to imitate, for instance the tasks that humans, and nest-building

   439 birds, and monkeys and so on can perform. 

   440 </p>

   441 </div>

   442 

   443 </div>

   444 

   445 <div id="outline-container-3-2" class="outline-3">

   446 <h3 id="sec-3-2"><span class="section-number-3">3.2</span> Example: Vision is not just about finding surfaces, but about finding affordances.</h3>

   447 <div class="outline-text-3" id="text-3-2">

   448 

   449 <p>[11:09] I'll give you a simple example &mdash; well, maybe not so simple,

   450 but &mdash; It's often assumed that the function of vision in humans (and

   451 in other animals with good eyesight and so on) is to take in optical

   452 information that hits the retina, and form into the (maybe changing

   453 &mdash; or, really, in our case definitely changing) patterns of

   454 illumination where there are sensory receptors that detect those

   455 patterns, and then somehow from that information (plus maybe other

   456 information gained from head movement or from comparisons between two

   457 eyes) to work out what there was in the environment that produced

   458 those patterns, and that is often taken to mean &ldquo;where were the

   459 surfaces off which the light bounced before it came to me&rdquo;. So

   460 you essentially think of the task of the visual system as being to

   461 reverse the image formation process: so the 3D structure's there, the

   462 lens causes the image to form in the retina, and then the brain goes

   463 back to a model of that 3D structure there. That's a very plausible

   464 theory about vision, and it may be that that's a <i>subset</i> of what

   465 human vision does, but I think James Gibson pointed out that that kind

   466 of thing is not necessarily going to be very useful for an organism,

   467 and it's very unlikely that that's the main function of perception in

   468 general, namely to produce some physical description of what's out

   469 there.

   470 </p>

   471 <p>

   472 [12:37] What does an animal <i>need</i>? It needs to know what it can do,

   473 what it can't do, what the consequences of its actions will be

   474 &hellip;. so, he introduced the word <i>affordance</i>, so from his point of

   475 view, the function of vision, perception, are to inform the organism

   476 of what the <i>affordances</i> are for action, where that would mean what

   477 the animal, <i>given</i> its morphology (what it can do with its mouth, its

   478 limbs, and so on, and the ways it can move) what it can do, what its

   479 needs are, what the obstacles are, and how the environment supports or

   480 obstructs those possible actions.

   481 </p>

   482 <p>

   483 [13:15] And that's a very different collection of information

   484 structures that you need from, say, &ldquo;where are all the

   485 surfaces?&rdquo;: if you've got all the surfaces, <i>deriving</i> the

   486 affordances would still be a major task. So, if you think of the

   487 perceptual system as primarily (for biological organisms) being

   488 devices that provide information about affordances and so on, then the

   489 tasks look very different. And most of the people working, doing

   490 research on computer vision in robots, I think haven't taken all that

   491 on board, so they're trying to get machines to do things which, even

   492 if they were successful, would not make the robots very intelligent

   493 (and in fact, even the ones they're trying to do are not really easy

   494 to do, and they don't succeed very well&mdash; although, there's progress;

   495 I shouldn't disparage it too much.)

   496 </p>

   497 </div>

   498 

   499 </div>

   500 

   501 <div id="outline-container-3-3" class="outline-3">

   502 <h3 id="sec-3-3"><span class="section-number-3">3.3</span> Online and offline intelligence</h3>

   503 <div class="outline-text-3" id="text-3-3">

   504 

   505 

   506 <p>

   507 [14:10] It gets more complex as animals get more sophisticated. So, I

   508 like to make a distinction between online intelligence and offline

   509 intelligence. So, for example, if I want to pick something up &mdash; like

   510 this leaf &lt;he plucks a leaf from the table&gt; &mdash; I was able to select

   511 it from all the others in there, and while moving my hand towards it,

   512 I was able to guide its trajectory, making sure it was going roughly

   513 in the right direction &mdash; as opposed to going out there, which

   514 wouldn't have been able to pick it up &mdash; and these two fingers ended

   515 up with a portion of the leaf between them, so that I was able to tell

   516 when I'm ready to do that &lt;he clamps the leaf between two fingers&gt;

   517 and at that point, I clamped my fingers and then I could pick up the

   518 leaf. 

   519 </p>

   520 <p>

   521 [14:54] Whereas, &mdash; and that's an example of online intelligence:

   522 during the performance of an action (both from the stage where it's

   523 initiated, and during the intermediate stages, and where it's

   524 completed) I'm taking in information relevant to controlling all those

   525 stages, and that relevant information keeps changing. That means I

   526 need stores of transient information which gets discarded almost

   527 immediately and replaced or something. That's online intelligence. And

   528 there are many forms; that's just one example, and Gibson discussed

   529 quite a lot of examples which I won't try to replicate now.

   530 </p>

   531 <p>

   532 [15:30] But in offline intelligence, you're not necessarily actually

   533 <i>performing</i> the actions when you're using your intelligence; you're

   534 thinking about <i>possible</i> actions. So, for instance, I could think

   535 about how fast or by what route I would get back to the lecture room

   536 if I wanted to [get to the next talk] or something. And I know where

   537 the door is, roughly speaking, and I know roughly which route I would

   538 take, when I go out, I should go to the left or to the right, because

   539 I've stored information about where the spaces are, where the

   540 buildings are, where the door was that we came out &mdash; but in using

   541 that information to think about that route, I'm not actually

   542 performing the action. I'm not even <i>simulating</i> it in detail: the

   543 precise details of direction and speed and when to clamp my fingers,

   544 or when to contract my leg muscles when walking, are all irrelevant to

   545 thinking about a good route, or thinking about the potential things

   546 that might happen on the way. Or what would be a good place to meet

   547 someone who I think [for an acquaintance in particular] &mdash; [barber]

   548 or something &mdash; I don't necessarily have to work out exactly <i>where</i>

   549 the person's going to stand, or from what angle I would recognize

   550 them, and so on.

   551 </p>

   552 <p>

   553 [16:46] So, offline intelligence &mdash; which I think became not just a

   554 human competence; I think there are other animals that have aspects of

   555 it: Squirrels are very impressive as you watch them. Gray squirrels at

   556 any rate, as you watch them defeating squirrel-proof birdfeeders, seem

   557 to have a lot of that [offline intelligence], as well as the online

   558 intelligence when they eventually perform the action they've worked

   559 out [] that will get them to the nuts. 

   560 </p>

   561 <p>

   562 [17:16] And I think that what happened during our evolution is that

   563 mechanisms for acquiring and processing and storing and manipulating

   564 information that is more and more remote from the performance of

   565 actions developed. An example is taking in information about where

   566 locations are that you might need to go to infrequently: There's a

   567 store of a particular type of material that's good for building on

   568 roofs of houses or something out around there in some

   569 direction. There's a good place to get water somewhere in another

   570 direction. There are people that you'd like to go and visit in

   571 another place, and so on. 

   572 </p>

   573 <p>

   574 [17:59] So taking in information about an extended environment and

   575 building it into a structure that you can make use of for different

   576 purposes is another example of offline intelligence. And when we do

   577 that, we sometimes use only our brains, but in modern times, we also

   578 learned how to make maps on paper and walls and so on. And it's not

   579 clear whether the stuff inside our heads has the same structures as

   580 the maps we make on paper: the maps on paper have a different

   581 function; they may be used to communicate with others, or meant for

   582 <i>looking</i> at, whereas the stuff in your head you don't <i>look</i> at; you

   583 use it in some other way.

   584 </p>

   585 <p>

   586 [18:46] So, what I'm getting at is that there's a great deal of human

   587 intelligence (and animal intelligence) which is involved in what's

   588 possible in the future, what exists in distant places, what might have

   589 happened in the past (sometimes you need to know why something is as

   590 it is, because that might be relevant to what you should or shouldn't

   591 do in the future, and so on), and I think there was something about

   592 human evolution that extended that offline intelligence way beyond

   593 that of animals. And I don't think it was <i>just</i> human language, (but

   594 human language had something to do with it) but I think there was

   595 something else that came earlier than language which involves the

   596 ability to use your offline intelligence to discover something that

   597 has a rich mathematical structure. 

   598 </p>

   599 </div>

   600 

   601 </div>

   602 

   603 <div id="outline-container-3-4" class="outline-3">

   604 <h3 id="sec-3-4"><a name="example-gap" id="example-gap"></a><span class="section-number-3">3.4</span> Example: Even toddlers use sophisticated geometric knowledge</h3>

   605 <div class="outline-text-3" id="text-3-4">

   606 

   607 <p>[19:44] I'll give you a simple example: if you look through a gap, you

   608 can see something that's on the other side of the gap. Now, you

   609 <i>might</i> see what you want to see, or you might see only part of it. If

   610 you want to see more of it, which way would you move? Well, you could

   611 either move <i>sideways</i>, and see through the gap&mdash;and see it roughly

   612 the same amount but a different part of it [if it's a ????], or you

   613 could move <i>towards</i> the gap and then your view will widen as you

   614 approach the gap. Now, there's a bit of mathematics in there, insofar

   615 as you are implicitly assuming that information travels in straight

   616 lines, and as you go closer to a gap, the straight lines that you can

   617 draw from where you are through the gap, widen as you approach that

   618 gap. Now, there's a kind of theorem of Euclidean geometry in there

   619 which I'm not going to try to state very precisely (and as far as I

   620 know, wasn't stated explicitly in Euclidean geometry) but it's

   621 something every toddler&mdash; human toddler&mdash;learns. (Maybe other

   622 animals also know it, I don't know.) But there are many more things,

   623 actions to perform, to get you more information about things, actions

   624 to perform to conceal information from other people, actions that will

   625 enable you to operate, to act on a rigid object in one place in order

   626 to produce an effect on another place. So, there's a lot of stuff that

   627 involves lines and rotations and angles and speeds and so on that I

   628 think humans (maybe, to a lesser extent, other animals) develop the

   629 ability to think about in a generic way. That means that you could

   630 take out the generalizations from the particular contexts and then

   631 re-use them in a new contexts in ways that I think are not yet

   632 represented at all in AI and in theories of human learning in any []

   633 way &mdash; although some people are trying to study learning of mathematics.

   634 </p>

   635 </div>

   636 </div>

   637 

   638 </div>

   639 

   640 <div id="outline-container-4" class="outline-2">

   641 <h2 id="sec-4"><span class="section-number-2">4</span> Animal intelligence</h2>

   642 <div class="outline-text-2" id="text-4">

   643 

   644 

   645 

   646 </div>

   647 

   648 <div id="outline-container-4-1" class="outline-3">

   649 <h3 id="sec-4-1"><span class="section-number-3">4.1</span> The priority is <i>cataloguing</i> what competences have evolved, not ranking them.</h3>

   650 <div class="outline-text-3" id="text-4-1">

   651 

   652 <p>[22:03] I wasn't going to challenge the claim that humans can do more

   653 sophisticated forms of [tracking], just to mention that there are some

   654 things that other animals can do which are in some ways comparable,

   655 and some ways superior to [things] that humans can do. In particular,

   656 there are species of birds and also, I think, some rodents ---

   657 squirrels, or something &mdash; I don't know enough about the variety ---

   658 that can hide nuts and remember where they've hidden them, and go back

   659 to them. And there have been tests which show that some birds are able

   660 to hide tens &mdash; you know, [eighteen] or something nuts &mdash; and to

   661 remember which ones have been taken, which ones haven't, and so

   662 on. And I suspect most humans can't do that. I wouldn't want to say

   663 categorically that maybe we couldn't, because humans are very

   664 [varied], and also [a few] people can develop particular competences

   665 through training. But it's certainly not something I can do.

   666 </p>

   667 

   668 </div>

   669 

   670 </div>

   671 

   672 <div id="outline-container-4-2" class="outline-3">

   673 <h3 id="sec-4-2"><span class="section-number-3">4.2</span> AI can be used to test philosophical theories</h3>

   674 <div class="outline-text-3" id="text-4-2">

   675 

   676 <p>[23:01] But I also would like to say that I am not myself particularly

   677 interested in trying to align animal intelligences according to any

   678 kind of scale of superiority; I'm just trying to understand what it

   679 was that biological evolution produced, and how it works, and I'm

   680 interested in AI <i>mainly</i> because I think that when one comes up with

   681 theories about how these things work, one needs to have some way of

   682 testing the theory. And AI provides ways of implementing and testing

   683 theories that were not previously available: Immanuel Kant was trying

   684 to come up with theories about how minds work, but he didn't have any

   685 kind of a mechanism that he could build to test his theory about the

   686 nature of mathematical knowledge, for instance, or how concepts were

   687 developed from babyhood onward. Whereas now, if we do develop a

   688 theory, we have a criterion of adequacy, namely it should be precise

   689 enough and rich enough and detailed to enable a model to be

   690 built. And then we can see if it works. 

   691 </p>

   692 <p>

   693 [24:07] If it works, it doesn't mean we've proved that the theory is

   694 correct; it just shows it's a candidate. And if it doesn't work, then

   695 it's not a candidate as it stands; it would need to be modified in

   696 some way.

   697 </p>

   698 </div>

   699 </div>

   700 

   701 </div>

   702 

   703 <div id="outline-container-5" class="outline-2">

   704 <h2 id="sec-5"><span class="section-number-2">5</span> Is abstract general intelligence feasible?</h2>

   705 <div class="outline-text-2" id="text-5">

   706 

   707 

   708 

   709 </div>

   710 

   711 <div id="outline-container-5-1" class="outline-3">

   712 <h3 id="sec-5-1"><span class="section-number-3">5.1</span> It's misleading to compare the brain and its neurons to a computer made of transistors</h3>

   713 <div class="outline-text-3" id="text-5-1">

   714 

   715 <p>[24:27] I think there's a lot of optimism based on false clues:

   716 the&hellip;for example, one of the false clues is to count the number of

   717 neurons in the brain, and then talk about the number of transistors

   718 you can fit into a computer or something, and then compare them. It

   719 might turn out that the study of the way synapses work (which leads

   720 some people to say that a typical synapse [] in the human brain has

   721 computational power comparable to the Internet a few years ago,

   722 because of the number of different molecules that are doing things,

   723 the variety of types of things that are being done in those molecular

   724 interactions, and the speed at which they happen, if you somehow count

   725 up the number of operations per second or something, then you get

   726 these comparable figures).

   727 </p>

   728 </div>

   729 

   730 </div>

   731 

   732 <div id="outline-container-5-2" class="outline-3">

   733 <h3 id="sec-5-2"><span class="section-number-3">5.2</span> For example, brains may rely heavily on chemical information processing</h3>

   734 <div class="outline-text-3" id="text-5-2">

   735 

   736 <p>Now even if the details aren't right, there may just be a lot of

   737 information processing that&hellip;going on in brains at the <i>molecular</i>

   738 level, not the neural level. Then, if that's the case, the processing

   739 units will be orders of magnitude larger in number than the number of

   740 neurons. And it's certainly the case that all the original biological

   741 forms of information processing were chemical; there weren't brains

   742 around, and still aren't in most microbes. And even when humans grow

   743 their brains, the process of starting from a fertilized egg and

   744 producing this rich and complex structure is, for much of the time,

   745 under the control of chemical computations, chemical information

   746 processing&mdash;of course combined with physical sorts of materials and

   747 energy and so on as well.

   748 </p>

   749 <p>

   750 [26:25] So it would seem very strange if all that capability was

   751 something thrown away when you've got a brain and all the information

   752 processing, the [challenges that were handled in making a brain],

   753 &hellip; This is handwaving on my part; I'm just saying that we <i>might</i>

   754 learn that what brains do is not what we think they do, and that

   755 problems of replicating them are not what we think they are, solely in

   756 terms of numerical estimate of time scales, the number of components,

   757 and so on.

   758 </p>

   759 </div>

   760 

   761 </div>

   762 

   763 <div id="outline-container-5-3" class="outline-3">

   764 <h3 id="sec-5-3"><span class="section-number-3">5.3</span> Brain algorithms may simply be optimized for certain kinds of information processing other than bit manipulations</h3>

   765 <div class="outline-text-3" id="text-5-3">

   766 

   767 <p>[26:56] But apart from that, the other basis of skepticism concerns

   768 how well we understand what the problems are. I think there are many

   769 people who try to formalize the problems of designing an intelligent

   770 system in terms of streams of information thought of as bit streams or

   771 collections of bit streams, and they think of as the problems of

   772 intelligence as being the construction or detection of patterns in

   773 those, and perhaps not just detection of patterns, but detection of

   774 patterns that are useable for sending <i>out</i> streams to control motors

   775 and so on in order to []. And that way of conceptualizing the problem

   776 may lead on the one hand to oversimplification, so that the things

   777 that <i>would</i> be achieved, if those goals were achieved, maybe much

   778 simpler, in some ways inadequate. Or the replication of human

   779 intelligence, or the matching of human intelligence&mdash;or for that

   780 matter, squirrel intelligence&mdash;but in another way, it may also make

   781 the problem harder: it may be that some of the kinds of things that

   782 biological evolution has achieved can't be done that way. And one of

   783 the ways that might turn out to be the case is not because it's not

   784 impossible in principle to do some of the information processing on

   785 artificial computers-based-on-transistors and other bit-manipulating

   786 []&mdash;but it may just be that the computational complexity of solving

   787 problems, processes, or finding solutions to complex problems, are

   788 much greater and therefore you might need a much larger universe than

   789 we have available in order to do things.

   790 </p>

   791 </div>

   792 

   793 </div>

   794 

   795 <div id="outline-container-5-4" class="outline-3">

   796 <h3 id="sec-5-4"><span class="section-number-3">5.4</span> Example: find the shortest path by dangling strings</h3>

   797 <div class="outline-text-3" id="text-5-4">

   798 

   799 <p>[28:55] Then if the underlying mechanisms were different, the

   800 information processing mechanisms, they might be better tailored to

   801 particular sorts of computation. There's a [] example, which is

   802 finding the shortest route if you've got a collection of roads, and

   803 they may be curved roads, and lots of tangled routes from A to B to C,

   804 and so on. And if you start at A and you want to get to Z &mdash; a place

   805 somewhere on that map &mdash; the process of finding the shortest route

   806 will involve searching through all these different possibilities and

   807 rejecting some that are longer than others and so on. But if you make

   808 a model of that map out of string, where these strings are all laid

   809 out on the maps and so have the lengths of the routes. Then if you

   810 hold the two knots in the string &ndash; it's a network of string &mdash; which

   811 correspond to the start point and end point, then <i>pull</i>, then the

   812 bits of string that you're left with in a straight line will give you

   813 the shortest route, and that process of pulling just gets you the

   814 solution very rapidly in a parallel computation, where all the others

   815 just hang by the wayside, so to speak.

   816 </p>

   817 </div>

   818 

   819 </div>

   820 

   821 <div id="outline-container-5-5" class="outline-3">

   822 <h3 id="sec-5-5"><span class="section-number-3">5.5</span> In sum, we know surprisingly little about the kinds of problems that evolution solved, and the manner in which they were solved.</h3>

   823 <div class="outline-text-3" id="text-5-5">

   824 

   825 <p>[30:15] Now, I'm not saying brains can build networks of string and

   826 pull them or anything like that; that's just an illustration of how if

   827 you have the right representation, correctly implemented&mdash;or suitably

   828 implemented&mdash;for a problem, then you can avoid very combinatorially

   829 complex searches, which will maybe grow exponentially with the number

   830 of components in your map, whereas with this thing, the time it takes

   831 won't depend on how many strings you've [got on the map]; you just

   832 pull, and it will depend only on the shortest route that exists in

   833 there. Even if that shortest route wasn't obvious on the original map.

   834 </p>

   835 

   836 <p>

   837 [30:59] So that's a rather long-winded way of formulating the

   838 conjecture which&mdash;of supporting, a roundabout way of supporting the

   839 conjecture that there may be something about the way molecules perform

   840 computations where they have the combination of continuous change as

   841 things move through space and come together and move apart, and

   842 whatever &mdash; and also snap into states that then persist, so [as you

   843 learn from] quantum mechanics, you can have stable molecular

   844 structures which are quite hard to separate, and then in catalytic

   845 processes you can separate them, or extreme temperatures, or strong

   846 forces, but they may nevertheless be able to move very rapidly in some

   847 conditions in order to perform computations.

   848 </p>

   849 <p>

   850 [31:49] Now there may be things about that kind of structure that

   851 enable searching for solutions to <i>certain</i> classes of problems to be

   852 done much more efficiently (by brain) than anything we could do with

   853 computers. It's just an open question.

   854 </p>

   855 <p>

   856 [32:04] So it <i>might</i> turn out that we need new kinds of technology

   857 that aren't on the horizon in order to replicate the functions that

   858 animal brains perform &mdash;or, it might not. I just don't know. I'm not

   859 claiming that there's strong evidence for that; I'm just saying that

   860 it might turn out that way, partly because I think we know less than

   861 many people think we know about what biological evolution achieved.

   862 </p>

   863 <p>

   864 [32:28] There are some other possibilities: we may just find out that

   865 there are shortcuts no one ever thought of, and it will all happen

   866 much more quickly&mdash;I have an open mind; I'd be surprised, but it

   867 could turn up. There <i>is</i> something that worries me much more than the

   868 singularity that most people talk about, which is machines achieving

   869 human-level intelligence and perhaps taking over [the] planet or

   870 something. There's what I call the <i>singularity of cognitive catch-up</i> &hellip;

   871 </p>

   872 </div>

   873 </div>

   874 

   875 </div>

   876 

   877 <div id="outline-container-6" class="outline-2">

   878 <h2 id="sec-6"><span class="section-number-2">6</span> A singularity of cognitive catch-up</h2>

   879 <div class="outline-text-2" id="text-6">

   880 

   881 

   882 

   883 </div>

   884 

   885 <div id="outline-container-6-1" class="outline-3">

   886 <h3 id="sec-6-1"><span class="section-number-3">6.1</span> What if it will take a lifetime to learn enough to make something new?</h3>

   887 <div class="outline-text-3" id="text-6-1">

   888 

   889 <p>&hellip; SCC, singularity of cognitive catch-up, which I think we're close

   890 to, or maybe have already reached&mdash;I'll explain what I mean by

   891 that. One of the products of biological evolution&mdash;and this is one of

   892 the answers to your earlier questions which I didn't get on to&mdash;is

   893 that humans have not only the ability to make discoveries that none of

   894 their ancestors have ever made, but to shorten the time required for

   895 similar achievements to be reached by their offspring and their

   896 descendants. So once we, for instance, worked out ways of complex

   897 computations, or ways of building houses, or ways of finding our way

   898 around, we don't need&hellip;our children don't need to work it out for

   899 themselves by the same lengthy trial and error procedure; we can help

   900 them get there much faster.

   901 </p>

   902 <p>

   903 Okay, well, what I've been referring to as the singularity of

   904 cognitive catch-up depends on the fact that&mdash;fairly obvious, and it's

   905 often been commented on&mdash;that in case of humans, it's not necessary

   906 for each generation to learn what previous generations learned <i>in the same way</i>. And we can speed up learning once something has been

   907 learned, [it is able to] be learned by new people. And that has meant

   908 that the social processes that support that kind of education of the

   909 young can enormously accelerate what would have taken&hellip;perhaps

   910 thousands [or] millions of years for evolution to produce, can happen in

   911 a much shorter time. 

   912 </p>

   913 

   914 <p>

   915 [34:54] But here's the catch: in order for a new advance to happen ---

   916 so for something new to be discovered that wasn't there before, like

   917 Newtonian mechanics, or the theory of relativity, or Beethoven's music

   918 or [style] or whatever &mdash; the individuals have to have traversed a

   919 significant amount of what their ancestors have learned, even if they

   920 do it much faster than their ancestors, to get to the point where they

   921 can see the gaps, the possibilities for going further than their

   922 ancestors, or their parents or whatever, have done.

   923 </p>

   924 <p>

   925 [35:27] Now in the case of knowledge of science, mathematics,

   926 philosophy, engineering and so on, there's been a lot of accumulated

   927 knowledge. And humans are living a <i>bit</i> longer than they used to, but

   928 they're still living for [whatever it is], a hundred years, or for

   929 most people, less than that. So you can imagine that there might come

   930 a time when in a normal human lifespan, it's not possible for anyone

   931 to learn enough to understand the scope and limits of what's already

   932 been achieved in order to see the potential for going beyond it and to

   933 build on what's already been done to make that&hellip;those future steps.

   934 </p>

   935 <p>

   936 [36:10] So if we reach that stage, we will have reached the

   937 singularity of cognitive catch-up because the process of education

   938 that enables individuals to learn faster than their ancestors did is

   939 the catching-up process, and it may just be that we at some point

   940 reach a point where catching up can only happen within a lifetime of

   941 an individual, and after that they're dead and they can't go

   942 beyond. And I have some evidence that there's a lot of that around

   943 because I see a lot of people coming up with what <i>they</i> think of as

   944 new ideas which they've struggled to come up with, but actually they

   945 just haven't taken in some of what was&hellip;some of what was done [] by

   946 other people, in other places before them. And I think that despite

   947 the availability of search engines which make it <i>easier</i> for people

   948 to get the information&mdash;for instance, when I was a student, if I

   949 wanted to find out what other people had done in the field, it was a

   950 laborious process&mdash;going to the library, getting books, and

   951 &mdash;whereas now, I can often do things in seconds that would have taken

   952 hours. So that means that if seconds [are needed] for that kind of

   953 work, my lifespan has been extended by a factor of ten or

   954 something. So maybe that <i>delays</i> the singularity, but it may not

   955 delay it enough. But that's an open question; I don't know. And it may

   956 just be that in some areas, this is more of a problem than others. For

   957 instance, it may be that in some kinds of engineering, we're handing

   958 over more and more of the work to machines anyways and they can go on

   959 doing it.  So for instance, most of the production of computers now is

   960 done by a computer-controlled machine&mdash;although some of the design

   961 work is done by humans&mdash; a lot of <i>detail</i> of the design is done by

   962 computers, and they produce the next generation, which then produces

   963 the next generation, and so on.

   964 </p>

   965 <p>

   966 [37:57] I don't know if humans can go on having major advances, so

   967 it'll be kind of sad if we can't.

   968 </p>

   969 </div>

   970 </div>

   971 

   972 </div>

   973 

   974 <div id="outline-container-7" class="outline-2">

   975 <h2 id="sec-7"><span class="section-number-2">7</span> Spatial reasoning: a difficult problem</h2>

   976 <div class="outline-text-2" id="text-7">

   977 

   978 

   979 <p>

   980 [38:15] Okay, well, there are different problems [ ] mathematics, and

   981 they have to do with properties. So for instance a lot of mathematics

   982 that can be expressed in terms of logical structures or algebraic

   983 structures and those are pretty well suited for manipulation and&hellip;on

   984 computers, and if a problem can be specified using the

   985 logical/algebraic notation, and the solution method requires creating

   986 something in that sort of notation, then computers are pretty good,

   987 and there are lots of mathematical tools around&mdash;there are theorem

   988 provers and theorem checkers, and all kinds of things, which couldn't

   989 have existed fifty, sixty years ago, and they will continue getting

   990 better.

   991 </p>

   992 

   993 <p>

   994 But there was something that I was <a href="#sec-3-4">alluding to earlier</a> when I gave the

   995 example of how you can reason about what you will see by changing your

   996 position in relation to a door, where what you are doing is using your

   997 grasp of spatial structures and how as one spatial relationship

   998 changes namely you come closer to the door or move sideways and

   999 parallel to the wall or whatever, other spatial relationships change

  1000 in parallel, so the lines from your eyes through to other parts of

  1001 the&hellip;parts of the room on the other side of the doorway change,

  1002 spread out more as you go towards  the doorway, and as you move

  1003 sideways, they don't spread out differently, but focus on different

  1004 parts of the internal &hellip; that they access different parts of the

  1005 &hellip; of the room.

  1006 </p>

  1007 <p>

  1008 Now, those are examples of ways of thinking about relationships and

  1009 changing relationships which are not the same as thinking about what

  1010 happens if I replace this symbol with that symbol, or if I substitute

  1011 this expression in that expression in a logical formula.  And at the

  1012 moment, I do not believe that there is anything in AI amongst the

  1013 mathematical reasoning community, the theorem-proving community, that

  1014 can model the processes that go on when a young child starts learning

  1015 to do Euclidean geometry and is taught things about&mdash;for instance, I

  1016 can give you a proof that the angles of any triangle add up to a

  1017 straight line, 180 degrees. 

  1018 </p>

  1019 

  1020 </div>

  1021 

  1022 <div id="outline-container-7-1" class="outline-3">

  1023 <h3 id="sec-7-1"><span class="section-number-3">7.1</span> Example: Spatial proof that the angles of any triangle add up to a half-circle</h3>

  1024 <div class="outline-text-3" id="text-7-1">

  1025 

  1026 <p>There are standard proofs which involves starting with one triangle,

  1027 then adding a line parallel to the base one of my former students,

  1028 Mary Pardoe, came up with which I will demonstrate with this &lt;he holds

  1029 up a pen&gt; &mdash; can you see it? If I have a triangle here that's got

  1030 three sides, if I put this thing on it, on one side &mdash; let's say the

  1031 bottom&mdash;I can rotate it until it lies along the second&hellip;another

  1032 side, and then maybe move it up to the other end ~. Then I can rotate

  1033 it again, until it lies on the third side, and move it back to the

  1034 other end. And then I'll rotate it again and it'll eventually end up

  1035 on the original side, but it will have changed the direction it's

  1036 pointing in &mdash; and it won't have crossed over itself so it will have

  1037 gone through a half-circle, and that says that the three angles of a

  1038 triangle add up to the rotations of half a circle, which is a

  1039 beautiful kind of proof and almost anyone can understand it. Some

  1040 mathematicians don't like it, because they say it hides some of the

  1041 assumptions, but nevertheless, as far as I'm concerned, it's an

  1042 example of a human ability to do reasoning which, once you've

  1043 understood it, you can see will apply to any triangle &mdash; it's got to

  1044 be a planar triangle &mdash; not a triangle on a globe, because then the

  1045 angles can add up to more than &hellip; you can have three <i>right</i> angles

  1046 if you have an equator&hellip;a line on the equator, and a line going up to

  1047 to the north pole of the earth, and then you have a right angle and

  1048 then another line going down to the equator, and you have a right

  1049 angle, right angle, right angle, and they add up to more than a

  1050 straight line. But that's because the triangle isn't in the plane,

  1051 it's on a curved surface. In fact, that's one of the

  1052 differences&hellip;definitional differences you can take between planar and

  1053 curved surfaces: how much the angles of a triangle add up to. But our

  1054 ability to <i>visualize</i> and notice the generality in that process, and

  1055 see that you're going to be able to do the same thing using triangles

  1056 that stretch in all sorts of ways, or if it's a million times as

  1057 large, or if it's made&hellip;you know, written on, on&hellip;if it's drawn in

  1058 different colors or whatever &mdash; none of that's going to make any

  1059 difference to the essence of that process. And that ability to see

  1060 the commonality in a spatial structure which enables you to draw some

  1061 conclusions with complete certainty&mdash;subject to the possibility that

  1062 sometimes you make mistakes, but when you make mistakes, you can

  1063 discover them, as has happened in the history of geometrical theorem

  1064 proving. Imre Lakatos had a wonderful book called <a href="http://en.wikipedia.org/wiki/Proofs_and_Refutations"><i>Proofs and Refutations</i></a> &mdash; which I won't try to summarize &mdash; but he has

  1065 examples: mistakes were made; that was because people didn't always

  1066 realize there were subtle subcases which had slightly different

  1067 properties, and they didn't take account of that. But once they're

  1068 noticed, you rectify that. 

  1069 </p>

  1070 </div>

  1071 

  1072 </div>

  1073 

  1074 <div id="outline-container-7-2" class="outline-3">

  1075 <h3 id="sec-7-2"><span class="section-number-3">7.2</span> Geometric results are fundamentally different than experimental results in chemistry or physics.</h3>

  1076 <div class="outline-text-3" id="text-7-2">

  1077 

  1078 <p>[43:28] But it's not the same as doing experiments in chemistry and

  1079 physics, where you can't be sure it'll be the same on [] or at a high

  1080 temperature, or in a very strong magnetic field &mdash; with geometric

  1081 reasoning, in some sense you've got the full information in front of

  1082 you; even if you don't always notice an important part of it. So, that

  1083 kind of reasoning (as far as I know) is not implemented anywhere in a

  1084 computer. And most people who do research on trying to model

  1085 mathematical reasoning, don't pay any attention to that, because of

  1086 &hellip; they just don't think about it. They start from somewhere else,

  1087 maybe because of how they were educated. I was taught Euclidean

  1088 geometry at school. Were you?

  1089 </p>

  1090 <p>

  1091 (Adam ford: Yeah)

  1092 </p>

  1093 <p>

  1094 Many people are not now. Instead they're taught set theory, and

  1095 logic, and arithmetic, and [algebra], and so on. And so they don't use

  1096 that bit of their brains, without which we wouldn't have built any of

  1097 the cathedrals, and all sorts of things we now depend on.

  1098 </p>

  1099 </div>

  1100 </div>

  1101 

  1102 </div>

  1103 

  1104 <div id="outline-container-8" class="outline-2">

  1105 <h2 id="sec-8"><span class="section-number-2">8</span> Is near-term artificial general intelligence likely?</h2>

  1106 <div class="outline-text-2" id="text-8">

  1107 

  1108 

  1109 

  1110 </div>

  1111 

  1112 <div id="outline-container-8-1" class="outline-3">

  1113 <h3 id="sec-8-1"><span class="section-number-3">8.1</span> Two interpretations: a single mechanism for all problems, or many mechanisms unified in one program.</h3>

  1114 <div class="outline-text-3" id="text-8-1">

  1115 

  1116 

  1117 <p>

  1118 [44:35] Well, this relates to what's meant by general. And when I

  1119 first encountered the AGI community, I thought that what they all

  1120 meant by general intelligence was <i>uniform</i> intelligence ---

  1121 intelligence based on some common simple (maybe not so simple, but)

  1122 single powerful mechanism or principle of inference. And there are

  1123 some people in the community who are trying to produce things like

  1124 that, often in connection with algorithmic information theory and

  1125 computability of information, and so on. But there's another sense of

  1126 general which means that the system of general intelligence can do

  1127 lots of different things, like perceive things, understand language,

  1128 move around, make things, and so on &mdash; perhaps even enjoy a joke;

  1129 that's something that's not nearly on the horizon, as far as I

  1130 know. Enjoying a joke isn't the same as being able to make laughing

  1131 noises. 

  1132 </p>

  1133 <p>

  1134 Given, then, that there are these two notions of general

  1135 intelligence&mdash;there's one that looks for one uniform, possibly

  1136 simple, mechanism or collection of ideas and notations and algorithms,

  1137 that will deal with any problem that's solvable &mdash; and the other

  1138 that's general in the sense that it can do lots of different things

  1139 that are combined into an integrated architecture (which raises lots

  1140 of questions about how you combine these things and make them work

  1141 together) and we humans, certainly, are of the second kind: we do all

  1142 sorts of different things, and other animals also seem to be of the

  1143 second kind, perhaps not as general as humans. Now, it may turn out

  1144 that in some near future time, who knows&mdash;decades, a few

  1145 decades&mdash;you'll be able to get machines that are capable of solving

  1146 in a time that will depend on the nature of the problem, but any

  1147 problem that is solvable, and they will be able to do it in some sort

  1148 of tractable time &mdash; of course, there are some problems that are

  1149 solvable that would require a larger universe and a longer history

  1150 than the history of the universe, but apart from that constraint,

  1151 these machines will be able to do anything [].  But to be able to do

  1152 some of the kinds of things that humans can do, like the kinds of

  1153 geometrical reasoning where you look at the shape and you abstract

  1154 away from the precise angles and sizes and shapes and so on, and

  1155 realize there's something general here, as must have happened when our

  1156 ancestors first made the discoveries that eventually put together in

  1157 Euclidean geometry. 

  1158 </p>

  1159 <p>

  1160 It may be that that requires mechanisms of a kind that we don't know

  1161 anything about at the moment. Maybe brains are using molecules and

  1162 rearranging molecules in some way that supports that kind of

  1163 reasoning. I'm not saying they are &mdash; I don't know, I just don't see

  1164 any simple&hellip;any obvious way to map that kind of reasoning capability

  1165 onto what we currently do on computers. There is&mdash;and I just

  1166 mentioned this briefly beforehand&mdash;there is a kind of thing that's

  1167 sometimes thought of as a major step in that direction, namely you can

  1168 build a machine (or a software system) that can represent some

  1169 geometrical structure, and then be told about some change that's going

  1170 to happen to it, and it can predict in great detail what'll

  1171 happen. And this happens for instance in game engines, where you say

  1172 we have all these blocks on the table and I'll drop one other block,

  1173 and then [the thing] uses Newton's laws and properties of rigidity of

  1174 the parts and the elasticity and also stuff about geometries and space

  1175 and so on, to give you a very accurate representation of what'll

  1176 happen when this brick lands on this pile of things, [it'll bounce and

  1177 go off, and so on]. And you just, with more memory and more CPU power,

  1178 you can increase the accuracy&mdash; but that's totally different than

  1179 looking at <i>one</i> example, and working out what will happen in a whole

  1180 <i>range</i> of cases at a higher level of abstraction, whereas the game

  1181 engine does it in great detail for <i>just</i> this case, with <i>just</i> those

  1182 precise things, and it won't even know what the generalizations are

  1183 that it's using that would apply to others []. So, in that sense, [we]

  1184 may get AGI &mdash; artificial general intelligence &mdash; pretty soon, but

  1185 it'll be limited in what it can do. And the other kind of general

  1186 intelligence which combines all sorts of different things, including

  1187 human spatial geometrical reasoning, and maybe other things, like the

  1188 ability to find things funny, and to appreciate artistic features and

  1189 other things may need forms of pattern-mechanism, and I have an open

  1190 mind about that.

  1191 </p>

  1192 </div>

  1193 </div>

  1194 

  1195 </div>

  1196 

  1197 <div id="outline-container-9" class="outline-2">

  1198 <h2 id="sec-9"><span class="section-number-2">9</span> Abstract General Intelligence impacts</h2>

  1199 <div class="outline-text-2" id="text-9">

  1200 

  1201 

  1202 <p>

  1203 [49:53] Well, as far as the first type's concerned, it could be useful

  1204 for all kinds of applications &mdash; there are people who worry about

  1205 where there's a system that has that type of intelligence, might in

  1206 some sense take over control of the planet. Well, humans often do

  1207 stupid things, and they might do something stupid that would lead to

  1208 disaster, but I think it's more likely that there would be other

  1209 things [] lead to disaster&mdash; population problems, using up all the

  1210 resources, destroying ecosystems, and whatever. But certainly it would

  1211 go on being useful to have these calculating devices. Now, as for the

  1212 second kind of them, I don't know&mdash;if we succeeded at putting

  1213 together all the parts that we find in humans, we might just make an

  1214 artificial human, and then we might have some of them as your friends,

  1215 and some of them we might not like, and some of them might become

  1216 teachers or whatever, composers &mdash; but that raises a question: could

  1217 they, in some sense, be superior to us, in their learning

  1218 capabilities, their understanding of human nature, or maybe their

  1219 wickedness or whatever &mdash; these are all issues in which I expect the

  1220 best science fiction writers would give better answers than anything I

  1221 could do, but I did once fantasize when I [back] in 1978, that perhaps

  1222 if we achieved that kind of thing, that they would be wise, and gentle

  1223 and kind, and realize that humans are an inferior species that, you

  1224 know, have some good features, so they'd keep us in some kind of

  1225 secluded&hellip;restrictive kind of environment, keep us away from

  1226 dangerous weapons, and so on. And find ways of cohabitating with

  1227 us. But that's just fantasy.

  1228 </p>

  1229 <p>

  1230 Adam Ford: Awesome. Yeah, there's an interesting story <i>With Folded Hands</i> where [the computers] want to take care of us and want to

  1231 reduce suffering and end up lobotomizing everybody [but] keeping them

  1232 alive so as to reduce the suffering. 

  1233 </p>

  1234 <p>

  1235 Aaron Sloman: Not all that different from <i>Brave New World</i>, where it

  1236 was done with drugs and so on, but different humans are given

  1237 different roles in that system, yeah.

  1238 </p>

  1239 <p>

  1240 There's also <i>The Time Machine</i>, H.G. Wells, where the &hellip; in the

  1241 distant future, humans have split in two: the Eloi, I think they were

  1242 called, they lived underground, they were the [] ones, and then&mdash;no,

  1243 the Morlocks lived underground; Eloi lived on the planet; they were

  1244 pleasant and pretty but not very bright, and so on, and they were fed

  1245 on by &hellip;

  1246 </p>

  1247 <p>

  1248 Adam Ford: [] in the future.

  1249 </p>

  1250 <p>

  1251 Aaron Sloman: As I was saying, if you ask science fiction writers,

  1252 you'll probably come up with a wide variety of interesting answers. 

  1253 </p>

  1254 <p>

  1255 Adam Ford: I certainly have; I've spoken to [] of Birmingham, and

  1256 Sean Williams, &hellip; who else? 

  1257 </p>

  1258 <p>

  1259 Aaron Sloman: Did you ever read a story by E.M. Forrester called <i>The Machine Stops</i> &mdash; very short story, it's <a href="http://archive.ncsa.illinois.edu/prajlich/forster.html">on the Internet somewhere</a>

  1260 &mdash; it's about a time when people sitting &hellip; and this was written in

  1261 about [1914 ] so it's about&hellip;over a hundred years ago &hellip; people are

  1262 in their rooms, they sit in front of screens, and they type things,

  1263 and they communicate with one another that way, and they don't meet;

  1264 they have debates, and they give lectures to their audiences that way,

  1265 and then there's a woman whose son says &ldquo;I'd like to see

  1266 you&rdquo; and she says &ldquo;What's the point? You've got me at

  1267 this point &rdquo; but he wants to come and talk to her &mdash; I won't

  1268 tell you how it ends, but.

  1269 </p>

  1270 <p>

  1271 Adam Ford: Reminds me of the Internet.

  1272 </p>

  1273 <p>

  1274 Aaron Sloman: Well, yes; he invented &hellip; it was just extraordinary

  1275 that he was able to do that, before most of the components that we

  1276 need for it existed.

  1277 </p>

  1278 <p>

  1279 Adam Ford: [Another person who did that] was Vernor Vinge [] <i>True Names</i>. 

  1280 </p>

  1281 <p>

  1282 Aaron Sloman: When was that written?

  1283 </p>

  1284 <p>

  1285 Adam Ford: The seventies.

  1286 </p>

  1287 <p>

  1288 Aaron Sloman: Okay, well a lot of the technology was already around

  1289 then. The original bits of internet were working, in about 1973, I was

  1290 sitting &hellip; 1974, I was sitting at Sussex University trying to

  1291 use&hellip;learn LOGO, the programming language, to decide whether it was

  1292 going to be useful for teaching AI, and I was sitting [] paper

  1293 teletype, there was paper coming out, transmitting ten characters a

  1294 second from Sussex to UCL computer lab by telegraph cable, from there

  1295 to somewhere in Norway via another cable, from there by satellite to

  1296 California to a computer Xerox [] research center where they had

  1297 implemented a computer with a LOGO system on it, with someone I had

  1298 met previously in Edinburgh, Danny Bobrow, and he allowed me to have

  1299 access to this sytem. So there I was typing. And furthermore, it was

  1300 duplex typing, so every character I typed didn't show up on my

  1301 terminal until it had gone all the way there and echoed back, so I

  1302 would type, and the characters would come back four seconds later.

  1303 </p>

  1304 <p>

  1305 [55:26] But that was the Internet, and I think Vernor Vinge was

  1306 writing after that kind of thing had already started, but I don't

  1307 know. Anyway.

  1308 </p>

  1309 <p>

  1310 [55:41] Another&hellip;I mentioned H.G. Wells, <i>The Time Machine</i>. I

  1311 recently discovered, because <a href="http://en.wikipedia.org/wiki/David_Lodge_(author)">David Lodge</a> had written a sort of

  1312 semi-novel about him, that he had invented Wikipedia, in advance&mdash; he

  1313 had this notion of an encyclopedia that was free to everybody, and

  1314 everybody could contribute and [collaborate on it]. So, go to the

  1315 science fiction writers to find out the future &mdash; well, a range of

  1316 possible futures.

  1317 </p>

  1318 <p>

  1319 Adam Ford: Well the thing is with science fiction writers, they have

  1320 to maintain some sort of interest for their readers, after all the

  1321 science fiction which reaches us is the stuff that publishers want to

  1322 sell, and so there's a little bit of a &hellip; a bias towards making a

  1323 plot device there, and so the dramatic sort of appeals to our

  1324 amygdala, our lizard brain; we'll sort of stay there obviously to some

  1325 extent. But I think that they do come up with sort of amazing ideas; I

  1326 think it's worth trying to make these predictions; I think that we

  1327 should more time on strategic forecasting, I mean take that seriously.

  1328 </p>

  1329 <p>

  1330 Aaron Sloman: Well, I'm happy to leave that to others; I just want to

  1331 try to understand these problems that bother me about how things

  1332 work. And it may be that some would say that's irresponsible if I

  1333 don't think about what the implications will be. Well, understanding

  1334 how humans work <i>might</i> enable us to make [] humans &mdash; I suspect it

  1335 wont happen in this century; I think it's going to be too difficult.

  1336 </p></div>

  1337 </div>

  1338 </div>

  1339 

  1340 <div id="postamble">

  1341 <p class="date">Date: 2013-10-04 18:49:53 UTC</p>

  1342 <p class="author">Author: Dylan Holmes</p>

  1343 <p class="creator">Org version 7.7 with Emacs version 23</p>

  1344 <a href="http://validator.w3.org/check?uri=referer">Validate XHTML 1.0</a>

  1345 

  1346 </div>

  1347 </body>

  1348 </html>
author	Robert McIntyre <rlm@mit.edu>
date	Mon, 14 Mar 2016 07:21:07 -0700
parents	414a10d51d9f
children