Is the Matrix of Artificial Intelligence artificial?

I’ve just had a scan of this and immediately I felt depopulation.  Artificial Intelligence is an investment in robotics that do not demand higher wages, they don’t need money or food. I felt the mindset behind this disconnected from humanity, it feels rational, productivity orientated and views the world in models.  It looks at the pragmatic of overpopulation, robots serving functions that displace humans.  Rather than look at ways to develop humanity to resolve conflict, climate change, greed, overconsumption, indifference, poverty, disease – it seeks to shrink the problem and replace it with artificial intelligence. That is my feeling. The question is – where does my feeling come from? Is it mine or is it emerging from a higher intelligence? Just because science cannot measure inspiration, spirituality or how life unfolds on earth and in the universe, doesn’t mean it doesn’t exist. If you can’t see it perhaps it is because you do not know how to see then look.  This is an inner-sight.  I am focusing on the real intelligence that creates universes not labour saving devices.

I wonder about the people who play God over the lives of billions, the question is does god play dice?

I love the word used in this paper ‘desiderata’ (next blog real wisdom). 

Here is an excerpt out of paper I saw online.

https://nickbostrom.com/papers/aipolicy.pdf

Policy​ ​Desiderata​ ​in​ ​the   Development​ ​of​ ​Superintelligent​ ​AI   1

(2017)​ ​version​ ​3.7​ ​(first​ ​version:​ ​2016)    Nick​ ​Bostrom​†   Allan​ ​Dafoe​∗​ ​†  Carrick​ ​Flynn​†   †​ ​Future​ ​of​ ​Humanity​ ​Institute,​ ​Oxford​ ​University  ∗​ ​Department​ ​of​ ​Political​ ​Science,​ ​Yale​ ​University    [working​ ​paper]  www.nickbostrom.com 

ABSTRACT    Machine​ ​superintelligence​ ​could​ ​plausibly​ ​be​ ​developed​ ​in​ ​the​ ​coming​ ​decades​ ​or​ ​century.​ ​​ ​The  prospect​ ​of​ ​this​ ​transformative​ ​development​ ​presents​ ​a​ ​host​ ​of​ ​political​ ​challenges​ ​and  opportunities.​ ​​ ​This​ ​paper​ ​seeks​ ​to​ ​initiate​ ​discussion​ ​of​ ​these​ ​by​ ​identifying​ ​a​ ​set​ ​of​ ​distinctive  features​ ​of​ ​the​ ​transition​ ​to​ ​a​ ​machine​ ​intelligence​ ​era.​ ​​ ​From​ ​these​ ​distinctive​ ​features,​ ​we​ ​derive  a​ ​correlative​ ​set​ ​of​ ​policy​ ​desiderata—considerations​ ​that​ ​should​ ​be​ ​given​ ​extra​ ​weight​ ​in  long-term​ ​AI​ ​policy​ ​compared​ ​to​ ​other​ ​policy​ ​contexts.​ ​​ ​We​ ​argue​ ​that​ ​these​ ​desiderata​ ​are  relevant​ ​for​ ​a​ ​wide​ ​range​ ​of​ ​actors​ ​(including​ ​states,​ ​AI​ ​technology​ ​firms,​ ​investors,​ ​and​ ​NGOs)  with​ ​an​ ​interest​ ​in​ ​AI​ ​policy.​ ​​ ​However,​ ​developing​ ​concrete​ ​policy​ ​options​ ​that​ ​satisfy​ ​these  desiderata​ ​will​ ​require​ ​additional​ ​work.          Keywords:​ ​artificial​ ​intelligence,​ ​ethics,​ ​policy,​ ​​ ​technology,​ ​global​ ​governance,​ ​AI,  superintelligence             

1​ ​For​ ​comment​ ​and​ ​discussion,​ ​we’re​ ​grateful​ ​to​ ​Stuart​ ​Armstrong,​ ​Michael​ ​Barnett,​ ​Seth​ ​Baum,​ ​Dominic  Becker,​ ​Nick​ ​Beckstead,​ ​Devi​ ​Borg,​ ​Miles​ ​Brundage,​ ​Paul​ ​Christiano,​ ​Jack​ ​Clark,​ ​Rebecca​ ​Crootof,​ ​Richard  Danzig,​ ​Daniel​ ​Dewey,​ ​Eric​ ​Drexler,​ ​Sebastian​ ​Farquhar,​ ​Sophie​ ​Fischer,​ ​Ben​ ​Garfinkel,​ ​Katja​ ​Grace,​ ​Tom  Grant,​ ​Hilary​ ​Greaves,​ ​John​ ​Halstead,​ ​Robin​ ​Hanson,​ ​Verity​ ​Harding,​ ​Sean​ ​Legassick,​ ​Wendy​ ​Lin,​ ​Jelena  Luketina,​ ​Matthijs​ ​Maas,​ ​Luke​ ​Muehlhauser,​ ​Toby​ ​Ord,​ ​Mahendra​ ​Prasad,​ ​Anders​ ​Sandberg,​ ​Carl​ ​Shulman,  Andrew​ ​Snyder-Beattie,​ ​Nate​ ​Soares,​ ​Mojmir​ ​Stehlik,​ ​Jaan​ ​Tallinn,​ ​Alex​ ​Tymchenko,​ ​and​ ​several  anonymous​ ​referees.​ ​This​ ​work​ ​was​ ​supported​ ​by​ ​grants​ ​from​ ​the​ ​H2020​ ​European​ ​Research​ ​Council​ ​and  the​ ​Future​ ​of​ ​Life​ ​Institute.   1  …

Prospects​ ​of​ ​machine​ ​intelligence  Advanced​ ​artificial​ ​intelligence​ ​(AI)​ ​is​ ​a​ ​general-purpose​ ​technology​ ​with​ ​transformative​ ​potential.  The​ ​development​ ​of​ ​superintelligent​ ​AI—machine​ ​intelligence​ ​more​ ​cognitively​ ​capable​ ​than  humans​ ​in​ ​all​ ​practically​ ​relevant​ ​domains—would​ ​rank​ ​among​ ​the​ ​most​ ​important​ ​transitions​ ​in  history.​ ​​ ​Superintelligent​ ​machines​ ​could​ ​produce​ ​great​ ​benefits​ ​for​ ​human​ ​health​ ​and​ ​longevity,  scientific​ ​understanding,​ ​entertainment,​ ​space​ ​exploration,​ ​and​ ​in​ ​many​ ​other​ ​areas.​ ​​ ​Taken  together,​ ​these​ ​applications​ ​would​ ​enable​ ​vast​ ​improvements​ ​in​ ​human​ ​welfare.    At​ ​the​ ​same​ ​time,​ ​the​ ​development​ ​of​ ​superintelligence​ ​will​ ​be​ ​associated​ ​with​ ​significant  challenges,​ ​likely​ ​including​ ​novel​ ​security​ ​concerns,​ ​labor​ ​market​ ​dislocations,​ ​and​ ​a​ ​potential​ ​for  exacerbated​ ​inequality.​ ​​ ​It​ ​may​ ​even​ ​involve,​ ​in​ ​the​ ​form​ ​of​ ​accident​ ​or​ ​misuse,​ ​significant  existential​ ​risk​ ​(Bostrom,​ ​2014;​ ​Russell,​ ​Dewey​ ​and​ ​Tegmark,​ ​2016).     There​ ​is​ ​currently​ ​no​ ​consensus​ ​on​ ​the​ ​likelihood​ ​and​ ​timeline​ ​of​ ​the​ ​development​ ​of  superintelligent​ ​AI.​ ​​ ​In​ ​opinion​ ​surveys​ ​of​ ​AI​ ​experts,​ ​a​ ​majority​ ​place​ ​a​ ​significant​ ​chance​ ​of  high-level​ ​machine​ ​intelligence​ ​(HLMI)​ ​being​ ​developed​ ​in​ ​the​ ​coming​ ​decades.​ ​​ ​When​ ​HLMI​ ​is  defined​ ​as​ ​a​ ​machine​ ​intelligence​ ​capable​ ​of​ ​carrying​ ​out​ ​“most​ ​human​ ​professions​ ​at​ ​least​ ​as  well​ ​as​ ​a​ ​typical​ ​human,”​ ​the​ ​median​ ​view​ ​among​ ​a​ ​sample​ ​of​ ​the​ ​top​ ​100​ ​most​ ​cited​ ​AI  researchers​ ​was​ ​a​ ​10%​ ​chance​ ​of​ ​such​ ​AI​ ​being​ ​developed​ ​by​ ​2024,​ ​and​ ​a​ ​50%​ ​chance​ ​of​ ​it  being​ ​developed​ ​by​ ​2050​ ​(Müller​ ​and​ ​Bostrom,​ ​2016).​ ​​ ​When​ ​HLMI​ ​was​ ​defined​ ​as​ ​“unaided  machines​ ​[accomplishing]​ ​every​ ​task​ ​better​ ​and​ ​more​ ​cheaply​ ​than​ ​human​ ​workers,”​ ​a​ ​majority​ ​of  the​ ​sample​ ​of​ ​authors​ ​at​ ​two​ ​leading​ ​technical​ ​machine​ ​learning​ ​conferences​ ​(the​ ​2015​ ​NIPS​ ​and  ICML)​ ​believed​ ​there​ ​was​ ​at​ ​least​ ​a​ ​10%​ ​chance​ ​of​ ​it​ ​being​ ​developed​ ​by​ ​2031​ ​(Grace​ ​et​ ​al.,​ ​2017).  In​ ​view​ ​of​ ​how​ ​much​ ​could​ ​be​ ​at​ ​stake,​ ​even​ ​a​ ​modest​ ​chance​ ​of​ ​advanced​ ​general​ ​AI​ ​being  developed​ ​in​ ​the​ ​next​ ​several​ ​decades​ ​would​ ​provide​ ​sufficient​ ​reason​ ​to​ ​give​ ​this​ ​topic​ ​careful  examination.   

2 

The​ ​transformativeness​ ​of​ ​such​ ​a​ ​transition​ ​to​ ​a​ ​machine​ ​intelligence​ ​era​ ​is​ ​brought​ ​out​ ​if​ ​we  reflect​ ​on​ ​a​ ​few​ ​implications.​ ​​ ​Cheap​ ​generally​ ​intelligent​ ​machines​ ​could​ ​substitute​ ​for​ ​most  human​ ​labor. ​ ​​ ​Inexpensive​ ​(relative​ ​to​ ​human​ ​labor)​ ​superintelligent​ ​AI​ ​is​ ​the​ ​last​ ​invention​ ​that  2 humans​ ​would​ ​ever​ ​need​ ​to​ ​make,​ ​since​ ​all​ ​future​ ​inventions​ ​would​ ​be​ ​more​ ​effectively  delegated​ ​to​ ​AI​ ​(Good,​ ​1965).​ ​​ ​Early​ ​versions​ ​of​ ​machine​ ​superintelligence​ ​may​ ​quickly​ ​build​ ​more  advanced​ ​versions,​ ​plausibly​ ​leading​ ​to​ ​an​ ​“intelligence​ ​explosion”.​ ​​ ​This​ ​acceleration​ ​of​ ​machine  intelligence​ ​would​ ​likely​ ​drive​ ​all​ ​other​ ​forms​ ​of​ ​technological​ ​progress,​ ​producing​ ​a​ ​plethora​ ​of  innovations,​ ​such​ ​as​ ​in​ ​life​ ​extension​ ​medicine,​ ​space​ ​settlement,​ ​weapons​ ​systems​ ​and​ ​military  logistics,​ ​surveillance,​ ​brain​ ​emulations,​ ​and​ ​virtual​ ​reality.​ ​​ ​Economic​ ​growth​ ​rates​ ​would  increase​ ​dramatically​ ​(Nordhaus,​ ​2015),​ ​plausibly​ ​by​ ​several​ ​orders​ ​of​ ​magnitude​ ​(Hanson,​ ​2016,  ch.​ ​16).    Such​ ​developments​ ​could​ ​alter​ ​political​ ​dynamics​ ​at​ ​local​ ​and​ ​global​ ​levels,​ ​and​ ​would​ ​force  humanity​ ​to​ ​confront​ ​important​ ​collective​ ​choice​ ​problems. ​ ​​ ​These​ ​may​ ​include:​ ​securing  3 cooperation​ ​and​ ​a​ ​high​ ​level​ ​of​ ​investment​ ​in​ ​safety​ ​despite​ ​potential​ ​private​ ​incentives​ ​to​ ​shirk  on​ ​safety​ ​and​ ​engage​ ​in​ ​an​ ​AI​ ​technology​ ​race;​ ​ensuring​ ​a​ ​reasonably​ ​equitable​ ​distribution​ ​of  benefits​ ​and​ ​influence;​ ​preventing​ ​novel​ ​forms​ ​of​ ​misuse​ ​and​ ​abuse;​ ​maintaining​ ​peace​ ​and  stability​ ​under​ ​conditions​ ​of​ ​a​ ​potentially​ ​rapidly​ ​shifting​ ​balance​ ​of​ ​power;​ ​arranging​ ​for​ ​global  stabilization​ ​if​ ​offensive​ ​or​ ​destructive​ ​technologies​ ​become​ ​strongly​ ​dominant​ ​at​ ​some​ ​level​ ​of  technological​ ​development;​ ​and​ ​navigating​ ​onto​ ​a​ ​favorable​ ​long-term​ ​trajectory​ ​for​ ​humanity​ ​in  the​ ​presence​ ​of​ ​evolutionary​ ​or​ ​other​ ​competitive​ ​dynamics​ ​that​ ​might​ ​operate​ ​within​ ​or​ ​between  technologically​ ​mature​ ​civilizations.    Achieving​ ​a​ ​successful​ ​transition​ ​to​ ​the​ ​machine​ ​intelligence​ ​era​ ​will​ ​thus​ ​require​ ​solving​ ​several  different​ ​types​ ​of​ ​problem.​ ​​ ​First,​ ​technological​ ​progress​ ​is​ ​required​ ​to​ ​increase​ ​the​ ​capability​ ​of  AI​ ​systems.​ ​​ ​Great​ ​resources​ ​are​ ​devoted​ ​to​ ​making​ ​this​ ​happen,​ ​with​ ​major​ ​(and​ ​growing)  investments​ ​from​ ​both​ ​industry​ ​and​ ​academia​ ​in​ ​many​ ​countries.​ ​​ ​Second,​ ​there​ ​is​ ​the​ ​technical  control​ ​problem​ ​of​ ​developing​ ​scalable​ ​control​ ​methods​ ​that​ ​could​ ​ensure​ ​that​ ​a​ ​superintelligent  AI​ ​will​ ​be​ ​safe​ ​and​ ​will​ ​behave​ ​as​ ​its​ ​programmers​ ​intend​ ​even​ ​if​ ​its​ ​intellectual​ ​capabilities​ ​are  increased​ ​to​ ​arbitrary​ ​levels.​ ​​ ​Until​ ​recently,​ ​this​ ​problem​ ​was​ ​almost​ ​entirely​ ​neglected;​ ​but​ ​in​ ​the  last​ ​couple​ ​of​ ​years,​ ​technical​ ​research​ ​agendas​ ​have​ ​been​ ​developed,​ ​and​ ​there​ ​are​ ​now  several​ ​research​ ​groups​ ​pursuing​ ​work​ ​in​ ​this​ ​area. ​ ​​ ​Total​ ​investment​ ​in​ ​long-term​ ​AI​ ​safety,  4 however,​ ​remains​ ​orders​ ​of​ ​magnitude​ ​less​ ​than​ ​investment​ ​in​ ​increasing​ ​AI​ ​capabilities.    And​ ​third,​ ​there​ ​is​ ​the​ ​political​ ​challenge​ ​of​ ​making​ ​sure​ ​that​ ​AI​ ​is​ ​developed,​ ​deployed,​ ​and  governed​ ​in​ ​a​ ​responsible​ ​and​ ​generally​ ​beneficial​ ​way.​ ​​ ​This,​ ​also,​ ​remains​ ​relatively​ ​neglected.  Some​ ​AI-related​ ​governance​ ​issues​ ​have​ ​begun​ ​to​ ​be​ ​explored,​ ​such​ ​as​ ​the​ ​ethics​ ​of​ ​lethal  autonomous​ ​weapons​ ​(Roff,​ ​2014;​ ​Bhuta​ ​et​ ​al.,​ ​2016),​ ​AI-augmented​ ​surveillance​ ​(Calo,​ ​2010),  2​ ​An​ ​exception​ ​would​ ​arise​ ​if​ ​there​ ​is​ ​demand​ ​specifically​ ​for​ ​human​ ​labor,​ ​such​ ​as​ ​a​ ​consumer​ ​preference  for​ ​goods​ ​made​ ​“by​ ​hand”.    3​ ​Other​ ​technologies,​ ​such​ ​as​ ​nuclear​ ​weapons,​ ​also​ ​seem​ ​to​ ​have​ ​shifted​ ​the​ ​global​ ​political​ ​landscape  (Wendt​ ​2003,​ ​Deudney​ ​2007).  4​ ​Three​ ​research​ ​agendas​ ​that​ ​have​ ​raised​ ​particular​ ​interest​ ​recently​ ​are​ ​the​ ​one​ ​developed​ ​by  researchers​ ​at​ ​Google​ ​Brain​ ​and​ ​OpenAI​ ​(Amodei​ ​et​ ​al.,​ ​2016)​ ​and​ ​two​ ​produced​ ​by​ ​the​ ​Machine  Intelligence​ ​Research​ ​Institute​ ​(Soares​ ​and​ ​Fallenstein,​ ​2014;​ ​Taylor​ ​et​ ​al.,​ ​2016).​ ​​ ​Additionally,​ ​there​ ​has  been​ ​an​ ​increase​ ​in​ ​the​ ​number​ ​of​ ​published​ ​pieces​ ​on​ ​technical​ ​AI​ ​safety​ ​topics,​ ​including​ ​by​ ​researchers  from​ ​DeepMind,​ ​OpenAI,​ ​and​ ​various​ ​academic​ ​institutions​ ​(Orseau​ ​and​ ​Armstrong,​ ​2016;​ ​OpenAI,​ ​2016;  Hadfield-Menell​ ​et​ ​al.,​ ​2016;​ ​Christiano,​ ​2016;​ ​Everitt​ ​and​ ​Hutter,​ ​2016;​ ​Evans​ ​et​ ​al.,​ ​2015).  3 

over-attachment​ ​to​ ​social​ ​robots​ ​(Lin,​ ​Abney​ ​and​ ​Bekey,​ ​2011),​ ​and​ ​the​ ​design​ ​of​ ​domestic  regulatory​ ​frameworks​ ​(Scherer​ ​2016).​ ​​ ​Research​ ​also​ ​needs​ ​to​ ​examine​ ​the​ ​fundamental​ ​set​ ​of  issues​ ​that​ ​the​ ​world​ ​would​ ​confront​ ​with​ ​the​ ​arrival​ ​of​ ​superintelligent​ ​AI​ ​(Conitzer,​ ​2016).  Governance​ ​issues​ ​related​ ​to​ ​these​ ​more​ ​long-term​ ​AI​ ​prospects​ ​remain​ ​almost​ ​entirely  unexplored.   5   In​ ​this​ ​paper,​ ​we​ ​seek​ ​to​ ​begin​ ​a​ ​conversation​ ​around​ ​these​ ​longer​ ​term​ ​issues.​ ​​ ​In​ ​particular,​ ​we  are​ ​interested​ ​in​ ​the​ ​question​ ​of​ ​how​ ​one​ ​should​ ​evaluate​ ​a​ ​proposal​ ​for​ ​the​ ​governance​ ​of​ ​AI  development.​ ​​ ​What​ ​desiderata​ ​should​ ​such​ ​a​ ​proposal​ ​satisfy?    We​ ​construe​ ​this​ ​question​ ​broadly.​ ​​ ​Thus,​ ​by​ ​“governance”​ ​we​ ​refer​ ​not​ ​only​ ​to​ ​the​ ​actions​ ​of  states​ ​but​ ​also​ ​to​ ​transnational​ ​governance​ ​(Hale​ ​and​ ​Held​ ​2011)​ ​involving​ ​norms​ ​and  arrangements​ ​arising​ ​from​ ​AI​ ​technology​ ​firms,​ ​investors,​ ​NGOs,​ ​and​ ​other​ ​relevant​ ​actors,​ ​and  the​ ​many​ ​kinds​ ​of​ ​global​ ​power​ ​that​ ​shape​ ​outcomes​ ​(Barnett​ ​and​ ​Duvall​ ​2005).​ ​​ ​And​ ​while  ethical​ ​considerations​ ​are​ ​relevant,​ ​they​ ​do​ ​not​ ​exhaust​ ​the​ ​scope​ ​of​ ​the​ ​proposed​ ​inquiry—we  wish​ ​to​ ​include​ ​desiderata​ ​focused​ ​on​ ​prudential​ ​interests​ ​of​ ​important​ ​constituencies​ ​as​ ​well​ ​as  considerations​ ​of​ ​technical​ ​and​ ​political​ ​feasibility.  Distinctive​ ​circumstances​ ​and​ ​corresponding​ ​desiderata  In​ ​the​ ​most​ ​general​ ​terms,​ ​we​ ​optimistically​ ​take​ ​the​ ​overarching​ ​objective​ ​to​ ​be​ ​the​ ​realization​ ​of  a​ ​widely​ ​appealing​ ​and​ ​inclusive​ ​near-​ ​and​ ​long-term​ ​future​ ​that​ ​ultimately​ ​achieves​ ​humanity’s  potential​ ​for​ ​desirable​ ​development​ ​while​ ​being​ ​considerate​ ​to​ ​beings​ ​of​ ​all​ ​kinds​ ​whose  interests​ ​may​ ​be​ ​affected​ ​by​ ​our​ ​choices.​ ​​ ​An​ ​ideal​ ​proposal​ ​for​ ​governance​ ​arrangements​ ​would  be​ ​one​ ​conducive​ ​to​ ​that​ ​end.    To​ ​make​ ​progress​ ​in​ ​our​ ​inquiry,​ ​we​ ​will​ ​not​ ​directly​ ​attempt​ ​to​ ​precisify​ ​this​ ​very​ ​broad​ ​and  vague​ ​objective.​ ​​ ​Instead,​ ​we​ ​take​ ​an​ ​indirect​ ​approach.​ ​​ ​We​ ​identify​ ​several​ ​respects​ ​in​ ​which  the​ ​prospect​ ​of​ ​advanced​ ​AI​ ​presents​ ​​special​ circumstances

​ —challenges​ ​or​ ​opportunities​ ​that  are​ ​either​ ​unique​ ​to​ ​the​ ​context​ ​of​ ​advanced​ ​AI​ ​or​ ​are​ ​expected​ ​to​ ​present​ ​there​ ​in​ ​unusual​ ​ways  or​ ​to​ ​unusual​ ​degrees.​ ​​ ​We​ ​argue​ ​that​ ​these​ ​special​ ​circumstances​ ​have​ ​some​ ​relatively  unambiguous​ ​implications​ ​for​ ​policy​ ​analysis​ ​in​ ​the​ ​sense​ ​that​ ​there​ ​are​ ​certain​ ​policy​ ​properties  that​ ​are​ ​far​ ​more​ ​important​ ​in​ ​these​ ​special​ ​circumstances​ ​(than​ ​in​ ​most​ ​other​ ​policy​ ​contexts)​ ​for 

5​ ​Although​ ​major​ ​governments​ ​are​ ​increasingly​ ​aware​ ​of​ ​long-term​ ​AI​ ​issues,​ ​this​ ​has​ ​not​ ​yet​ ​resulted​ ​in  substantial​ ​policy​ ​formulation.​ ​​ ​In​ ​October​ ​2016,​ ​the​ ​White​ ​House​ ​released​ ​a​ ​report​ ​which​ ​considered  long-term​ ​issues​ ​in​ ​AI,​ ​including​ ​AI​ ​safety​ ​and​ ​the​ ​risks​ ​posed​ ​by​ ​recursive​ ​self-improvement​ ​(National  Science​ ​and​ ​Technology​ ​Council,​ ​2016).​ ​​ ​Included​ ​in​ ​this​ ​report​ ​was​ ​the​ ​recommendation​ ​that​ ​“AI  professionals,​ ​safety​ ​professionals,​ ​and​ ​their​ ​professional​ ​societies​ ​should​ ​work​ ​together​ ​to​ ​continue  progress​ ​toward​ ​a​ ​mature​ ​field​ ​of​ ​AI​ ​safety​ ​engineering”​ ​(ibid.,​ ​p.​ ​34).​ ​​ ​Following​ ​this​ ​report,​ ​in​ ​November  the​ ​US​ ​Senate​ ​Commerce​ ​Subcommittee​ ​on​ ​Space,​ ​Science,​ ​and​ ​Competitiveness​ ​held​ ​hearings​ ​on​ ​AI  during​ ​which​ ​some​ ​long-term​ ​safety​ ​and​ ​policy​ ​considerations​ ​were​ ​broached​ ​(U.S.​ ​Senate,​ ​2016).​ ​​ ​In​ ​the  UK,​ ​in​ ​October​ ​2016,​ ​the​ ​House​ ​of​ ​Commons​ ​Science​ ​and​ ​Technology​ ​Committee​ ​released​ ​a​ ​report​ ​that  highlighted​ ​recent​ ​work​ ​in​ ​the​ ​area​ ​of​ ​AI​ ​safety,​ ​recommending​ ​the​ ​establishment​ ​of​ ​a​ ​standing  Commission​ ​on​ ​Artificial​ ​Intelligence,​ ​and​ ​a​ ​Robotics​ ​and​ ​Autonomous​ ​Systems​ ​Leadership​ ​Council,​ ​to  produce​ ​a​ ​national​ ​strategy​ ​on​ ​AI​ ​and​ ​Robotics​ ​(House​ ​of​ ​Commons​ ​Science​ ​and​ ​Technology​ ​Committee,  2016).​ ​​ ​However,​ ​this​ ​recent​ ​surge​ ​in​ ​interest​ ​notwithstanding,​ ​there​ ​has​ ​not​ ​yet​ ​been​ ​much​ ​focus​ ​on​ ​the  unique​ ​policy​ ​issues​ ​that​ ​arise​ ​specifically​ ​from​ ​these​ ​long-term​ ​considerations.  4 

the​ ​satisfaction​ ​of​ ​many​ ​widely​ ​shared​ ​prudential​ ​and​ ​moral​ ​preferences.​ ​​ ​We​ ​express​ ​these  especially​ ​relevant​ ​and​ ​important​ ​policy​ ​properties​ ​as​ ​a​ ​set​ ​of​ ​desiderata.    The​ ​desiderata​ ​are​ ​thus​ ​meant​ ​to​ ​provide​ ​other-things-equal​ ​reasons​ ​for​ ​pursuing​ ​certain  policies.​ ​​ ​Our​ ​recommendations​ ​take​ ​the​ ​form:​ ​“Whatever​ ​weight​ ​you​ ​ordinarily​ ​give​ ​to​ ​factor​ ​X​ ​in  your​ ​decision-making​ ​(relative​ ​to​ ​other​ ​factors),​ ​in​ ​the​ ​context​ ​of​ ​advanced​ ​AI​ ​you​ ​should​ ​place  more​ ​weight​ ​on​ ​X,​ ​because​ ​of​ ​the​ ​special​ ​circumstances​ ​that​ ​are​ ​present​ ​in​ ​this​ ​domain”.  Recommendations​ ​of​ ​this​ ​form​ ​can​ ​be​ ​relevant​ ​to​ ​a​ ​wide​ ​range​ ​of​ ​actors​ ​(individuals,​ ​institutions,  or​ ​more​ ​loosely​ ​defined​ ​groups​ ​and​ ​collectives)​ ​pursuing​ ​a​ ​wide​ ​set​ ​of​ ​different​ ​goals.​ ​​ ​Because  of​ ​their​ ​different​ ​goals​ ​or​ ​beliefs,​ ​such​ ​different​ ​actors​ ​may​ ​place​ ​different​ ​weights​ ​on​ ​some​ ​factor  X;​ ​and​ ​yet​ ​it​ ​may​ ​be​ ​true​ ​for​ ​all​ ​of​ ​them​ ​that​ ​a​ ​particular​ ​circumstance​ ​should​ ​change​ ​the​ ​weight  accorded​ ​to​ ​X​ ​in​ ​the​ ​same​ ​way.​ ​​ ​The​ ​starting​ ​point​ ​and​ ​the​ ​end​ ​point​ ​are​ ​different​ ​for​ ​each​ ​actor,  but​ ​the​ ​vectors​ ​between​ ​them​ ​(representing​ ​how​ ​the​ ​actor’s​ ​policy​ ​has​ ​changed)​ ​are​ ​aligned​ ​in  similar​ ​directions.