Hi Thom,

Wonderful that you spoke on this at the conference — it’s good to have serious academic work in this area. Do you know of any other researchers beyond SIAI affiliates working in this specific area (ethical goal-systems of non-anthropomorphic general intelligences)?

By Joshua Fox on 05.07.09 4:09 am

Hi,

Thanks for bringing this post to my attention - I had promised to post more about the paper, but never did - expect something soon!

I’m not sure I’d go so far as to call the things I’m talking about “non-anthropomorphic”; one of the points I’m trying to get traction on is that we need humanlike robots if we are to relate to them at all.

There are a lot of folks working on that sort of thing. For a survey, I suggest Moral Machines: Teaching Robots Right from Wrong by Wendell Wallach and Colin Allen. SIAI even gets a mention.

By Thom Blake on 05.07.09 8:53 am

>I’m not sure I’d go so far as to
> call the things I’m talking
> about “non-anthropomorphic”
Exactly–you make the point that a general intelligence without a humanlike ethical system (a non-anthropomorphic intelligence) would be quite dangerous, even if the goals appear innocuous. I think that this is one point very commonly missed: Since the only general intelligence we have ever seen is human, people disregard the great variety of possible intelligences where are nothing like human.

By Joshua Fox on 05.08.09 11:50 am