Sunday, July 26, 2009

Will China Build AGI First?



One of the reasons I spent 4 weeks in China this summer, organizing the First AGI Summer School and collaborating on research with my friend Hugo de Garis's Artificial Brain Lab at Xiamen University, was to qualitatively investigate Hugo's stories about the great potential for AGI R&D allegedly existent in the country of China.

One thing I learned about China is: the answer to almost any nontrivial question is some complex, multidimensional form of "maybe" or "sort of."

(Eventually the maybes and sort-ofs must collapse: Chinese do make hiring and firing decisions, get married, publish papers, and take other definitive actions ... but on the whole I found that in China there is a much greater willingness to embrace uncertainty than in the US, and a much smaller desire to make things clear and definite.)

In this spirit, I can't claim I came to a definite conclusion about the potential of China to lead the world in advanced AGI R&D.

But I can say that it's a definite possibility ;-) ....

Though the situation is complex, my gut feel is that Hugo is probably right, at least in the following sense: If a moderate number of AGI researchers (from the West and China both) apply their energy to pursuing the R&D opportunities that China offers, there is a strong potential that AGI research could advance there much faster than in other parts of the world.

Here are some relevant facts, militating in favor of China's role in AGI technology:

  • Even in the current dismal world economy, China's economy is still growing
  • Chinese students and researchers are willing to work long hours (of course some Americans and Europeans are too ... but my impression is that this willingness is greater in China)
  • The Chinese education system is very good at teaching advanced mathematics and algorithm theory, which are important for AGI
  • China is interested in pursuing advanced technologies -- both with practical applications in mind, and with a nationalistic motive of displaying their technological strength relative to other nations
  • Unlike the US, the Chinese research funding establishment has no "chip on its shoulder" about AI or AGI -- it has the same status as any other advanced technology. There was never an "AI winter" in China, nor is there dramatically more skepticism about AI than about other computer technologies
  • Due to the centralized system of government, if the central administration decides they value a certain technology, there is the potential for a massive amount of resources to be directed to that technology in a relatively rapid time-frame. Things often move very slowly in China, but sometimes they can also move much faster than in less centrally organized economies.
  • The cost of highly educated labor is low in China, so that if funding from outside China is found to help support China-based projects, this funding can go a long way!

Here are some other relevant facts, which are challenges China has to overcome if it wants to lead the world in AGI:

  • The Chinese university education system is narrowly disciplinary, whereas AGI requires copious interdisciplinary knowledge
  • Because China lacks a large, robust, cutting-edge software industry, there isn't that much "cultural knowledge" about how to manage complex software projects using "agile" methodologies. Yet, AGI is a complex software project that really demands an agile methodology. (Note that China has a load of great software engineers; the issue I'm pointing out regards software project management, not software engineering.)
  • Due to the nature of Chinese culture, it is fairly common for work environments to arise in which participants don't feel free to share their innovative ideas, and to point out problems with the ideas others are pursuing (especially if these "others" have higher social status according to Chinese tradition). However, some Chinese work environments are very friendly to innovation and criticism; so IMO this is best considered as a problem that can be overcome with attention to the personalities involved and the management mechanisms
  • Due to various factors including the restrictions on travel and Internet sites that the Chinese government places, China often feels "cut off" from the rest of the world, which results in a less-than-optimal degree of interaction with the international research community

Given the above factors, my conclusion is that IF China is able to attract sufficient foreign AGI experts, it may well be able to leapfrog ahead of other nations in the race to create AGI.

What the foreign experts would bring is not just AGI expertise and ideas, but expertise in allied areas like

  • interdisciplinary education
  • agile software project management
  • the management of innovation-friendly, "flat-hierarchy" research groups

Further, the presence of foreign experts in China full-time would result in other foreign experts more frequently traveling to China to speak, and in Chinese students more often traveling outside China for conferences and research visitation -- all of which would decrease the "China isolation" factor, and increase the intellectual potency of Chinese AGI research labs.

It's also the case that in Chinese academia right now, foreigners can get away with "shaking things up" a little more than Chinese nationals can. So even if a Chinese national showed up with exactly the same expertise and personality as a foreigner, they would have different strengths and weaknesses in the context. They would be able to get some things done more easily due to their Chinese-ness, but would also not be able to "get away with" as many disruptive methodologies.

So: the reason I think foreign AGI experts are critical is not that Chinese lack good ideas about AGI. (Yes, I think my own ideas about AGI are the best ones, but that's not my point right now!) It's that I don't think any one country has a monopoly on great AGI ideas and people, so the prize is likely to go to a country that can build an international AGI research community ... and also that there are certain organizational skills that are very useful for AGI, but not that well developed in China right now.

But there are serious challenges involved in recruiting foreign AGI experts to China:

  • The salaries are low by international standards. The salaries foreign faculty get at Chinese universities allow a very nice lifestyle in China -- but even so, they don't go that far in terms of international travel, purchase of electronics, or helping family members in the West.
  • Many AGI researchers have spouses and children who don't want to live in China. There are good international schools for researchers' children; but finding appropriate jobs for spouses can be difficult due to the language barrier and the different nature of the economy
  • The Chinese system of government is alien and off-putting to some foreigners. (As an example, this blog cannot be read by most Chinese Internet users right now, because blogger.com is blocked by the Chinese government. This sort of thing really bothers some foreign researchers and/or their families)
  • Some researchers will fear "career damage" if they go to a university without international name recognition (though, this factor would disappear quickly after a critical mass of researchers went to China)

So, the sixty-four trillion dollar question is to what extent these latter factors can be overcome.

I believe they could be overcome if Chinese universities or research agencies made very clear, very clearly research-friendly offers to foreign AGI researchers -- and then followed through on these offers once the researchers arrived. AGI researchers are a dedicated bunch, and many would put up with the problems cited above in order to have a good chance to lead a team of brilliant, qualified students at implementing powerful AGI systems.

As you may have guessed, one reason I'm cataloguing these factors so systematically is that I'm debating trying to rearrange my life to either move to China or (more likely) spend one semester per year in China.

At the moment, in my own discussions with Chinese universities about AGI research funding, I am finding things mildly confusing. The discussions are going interestingly, but I feel much less clarity than I would in a comparable discussion with a US university. This is nobody's fault -- it's a natural consequence of "cultural differences" -- but it's a factor that will have to be smoothed-out, IMO, if China is going to recruit a sizeable number of foreign AGI researchers.

So we now reach the conclusion of the above chain of thought. IF Chinese universities manage to fine-tune the art of recruiting foreign AGI researchers, then I think that China has a real chance of leading the world in the development of AGI.

I predict that, if China doesn't adopt the world lead in AGI, it will be because it fails at the things I cited above. They will fail to dominate in AGI if it turns out that the Chinese way of recruiting and retaining foreigners is too alien, causing a failure to accumulate a critical mass of foreign R&D leaders. This could certainly happen. Time will tell.

What Are the Risks if China Pulls Ahead in the AGI Race?




So China might plausibly take the lead in the AGI race.

As a citizen of the USA and Brazil (not China), does this worry me?

Not really.

Like Buckminster Fuller, I consider myself a "passenger on Spaceship Earth" (and I won't hesitate much to board another vessel when one becomes available -- or better yet I'd like to send multiple copies of myself on multiple vessels! But, I digress ;-).

One thing I'm being insistent on in my collaboration with Hugo de Garis's Artificial Brain Lab at Xiamen University, is that all our work be released as open-source code. The university folks there have no problem with this. So, in the case of my and Hugo's work, it's not a situation where we're trying to develop an AGI that will be exclusively owned by the Chinese government.

And, I feel strongly that anyone else doing AGI work in China -- or anywhere else! -- should take the same approach. The main reason I decided to open-source my own AGI project (OpenCog) (while keeping some valuable AGI-related technologies proprietary within Novamente LLC), is the intuition that AGI is a sufficiently big and thoroughgoingly important thing that it should be developed by the human collective mind as a whole, not by a small group or even a single nation.

Of course, if more and more AGI research gets done in China, then more and more of the world's AGI expertise will exist in China -- which will give China a substantial leadership position in AGI, regardless of whether the AGI code is open-source. But this really doesn't worry me much either, partly because I've been so impressed with the character and spirit of the younger generation of Chinese, who (in these scenarios we're discussing) will be doing most of the AGI work.

I met wonderful Chinese people of all ages during my visit to China. But there are huge generational differences among Chinese, and the Chinese who grew up with the Internet have a drastically different view of the world than the immediately prior generations. Most of the Chinese I met aged under 30 had a reasonably modern, international understanding of the world -- and some of the Chinese I met aged under 22 had such modern attitudes that they really could have been youth anywhere. The Internet is spreading international ideas and culture around the world, just like it's spreading the AGI meme around the world -- and may increasingly start spreading AGI researchers around the world ... we'll see.

Although I used a picture of Alfred E. Neuman above (like Hugo, I'm kind of a sucker for dramatic effect), I want to be clear that I don't have a cavalier attitude about the threat that could be posed if ANY government took control of the world's first AGI for their own parochial ends.

But I think this is a problem we need to work around, regardless of which country we do our research in.

By developing open-source code (made available on SourceForge, Launchpad, Google Code and so forth), and by carrying out our research in a way that emphasizes linkages with the international research community, we'll guarantee that AGI comes about as a product of the international collective mind of AGI researchers. This provides no grand guarantee of "AGI safety" (nothing can do that), but I strongly feel it's the best approach.

12 comments:

TransAlchemy said...

Cheers on doing your best to keep AGI from governments, though Im not sure how long this may last.

Yet in your effort to keep AGI from governments, your doing something I fear more. Your exposing AGI to the internet, in essence letting the internet be it's womb.

In either case centralized or uncentralized creation of AGI has it pitfalls.

Now we play the waiting game ;)

Ben Goertzel said...

TransAlchemy: The idea that "the best path to AGI is to have the right elite group secretly develop it" is sort of like the idea that "the best government is a benevolent dictatorship" ;-)

Ben Goertzel said...

Oh, and, TransAlchemy: the game I am playing is not one of "waiting" ;-)

flamoot said...

Oh I wouldn't worry about Alfred E. Newman I found that suitably chilling

Anonymous said...

Interesting take on China and AGI. Having lived in China for the better part of the past 25 years and having been the operations director of a China based IT outsourcing company, I do not share your positive outlook.

It is exceedingly hard to find good programmers and creativity is a big weakness of Chinese programmers as well. The latter being a particularly difficult hurdle for such a highly complex and uncharted problem of AGI development.

Ultimately however I do agree that I would not mind the Chinese developing AGI... That has more to do with my thinking on the nature of AGI though than that on the Chinese government :-)

Ben Goertzel said...

Hi Stefan,

I don't know much about the difficulty of finding quality commercial software programmers in China.

But I found a bunch of very good *AI research students* at Wuhan University and Xiamen University ... with a lot of creative and interesting ideas pertinent to AGI.

Commercial software engineering is different, of course.

Part of the problem you're seeing may be due to the way their education system is structured, with Software Engineering (at least at both Xiamen and Wuhan universities) totally separate from Computer Science. So, the professional software engineers don't have much background outside software engineering (lacking the diverse input that would foster creativity), whereas the computer science and AI researchers don't have that much software engineering background.

At first I thought the Chinese grad students lacked creativity, but once I got to know them better I found I was mistaken. They were (with some exceptions) more shy about sharing their ideas than Americans would be; but once I got to know them better, they turned out to have a load of great ideas.

So ... as with anything else in China, the conclusion has to be "it's complex" ;-)

-- Ben

spiral_shell said...
This comment has been removed by the author.
spiral_shell said...

I may be wrong, but the China Brain project does not look like it will suddenly 'wake up', but has the potential to move ahead. Even if we think the project will not take off like a rocket, the issue of pulling a friendly AGI out of mind-space still seems important early on. Has or will the China Brain project include any planning for niceness in the AGI road-map, or will it be more like pulling a super-brain out of a hat?

Bullwinkle:Hey Rocky, watch me pull a *god* out of my hat!
Rocky:Aww bullwinkle, that trick never works.
Bullwinkle:This time for sure!...Presto!!!
god-agi-artillect:RRROOOOOAAARRR!!!

It would be great there a Chinese version or branch of the Singularity Institute or Foresight Institute. Perhaps this would also help to decrease the "China Isolation" factor...
I must admit, I am a little scared of closedness and secrecy of the Chinese government, most likely uneducatedly so. But is there nothing that bothers you about the possibility of the first AGI coming to fruition under any 'less open' conditions? What can be done about this?

*p.s. I really want to travel to China, it looks like it would be a great experience!

-- Cheers, Adam

Ben Goertzel said...

Spiral Shell...

You asked


Has or will the China Brain project include any planning for niceness in the AGI road-map, or will it be more like pulling a super-brain out of a hat?


The current plan for the "China Brain" project is to use OpenCog for cognition, and an evolved neural net for perception and action.

OpenCog is going to use a variant of the Psi motivational/emotional framework outlined by Joscha Bach, which provides a systematic framework for handling "Friendliness" (though provides no absolute guarantees, of course).

You asked


I must admit, I am a little scared of closedness and secrecy of the Chinese government, most likely uneducatedly so. But is there nothing that bothers you about the possibility of the first AGI coming to fruition under any 'less open' conditions? What can be done about this?


The Chinese government has some aspects I don't care for. So does the US government.

If the code is open-source and is comprehended by a diverse international community of researchers, I'm not that concerned about where the core programming team is located.

Matt said...

Yesterday I spend a couple of hours watching online videos of some of your talks on AGI. Having a background in philosophy, I was surprised to find most of the technical stuff easy to follow. But I have some comments on AGI and ethics.

You seem to subscribe to the view that if we can make thinking machines who are also ethical, we basically have nothing to worry about. My contention is that we don't know this, and to find out we need to know more about what morality is. Let me explain.

We all know that in some cases it is right to kill other humans. From this it follows that a moral machine will sometimes kill humans.

It could be argued that this is not a big problem. We want the machines to do good, and if that entails killing humans in some extreme situations then so be it. What we fear is the total annihilation of the human race by machines, and this can never be moral.

Or can it? We would really want to know this for a fact before we build a thinking machine, just as we want to rule out any risk of inadvertantly creating a black hole before we build a high-energy particle accellerator.

We have basically three serious hypthesis about what morality is: Kantian ethics, consequentialism and contractualism. If Kantian ethics is true, it is highly unlikely in my judgment that exterminating the human race can ever be the right thing to do. But if consequentialism is true on the other hand, it's relatively easy to specify under which conditions a holocaust should be the ethically superior solution.

I personally subscibe to the contractualist theory of morality, which I understand as the view that moral responsibilities has it's root in cooperation and risk management. From this it could follow that machines have moral responsibilities toward humans, if but only if there's a strong (metaphysical) reason why machines will be better off if they cooperate with humans. But if the machines could create a world in which such cooperation is unnecessary, then they would not have any moral duties to preserve the human race, and if they could be much better off in a world without humans (or with a small group of humans in a zoo) they would have a moral responsibility to bring that about.

Perhaps you should engage some moral philosophers in the AI community to help you think clearer about this, or this problem has already been solved and I just don't know about it.

Ben Goertzel said...

Matt: there are moral philosophers involved with the AGI community already ... but as one might expect, this has not resulted in the moral issues being "solved" ;-)

Li said...

Cost of living? Only if you do not take into account the cost of real estate, health and education.