Satya Nadella Says AI Must Serve Humanity and Stay Under Human Control

The quest of superintelligence, according to Microsoft CEO and chairman Satya Nadella, must be based on the idea that artificial intelligence (AI) should serve humanity and be controlled by humans. Nadella stated in an X post that artificial intelligence development is "not worth pursuing" if it does not benefit humanity or cannot be controlled by humans.

The expedited dissemination of AI's advantages to nations, communities, and businesses was another of his demands. Both closed-source and open-source models should be able to flourish in the AI ecosystem, according to Nadella. Instead of relying on just one AI model supplier, he said, businesses should keep all of their proprietary and tacit information in-house.

Microsoft’s Opinions on Developing AI

A hill climbing machine or continuous learning loop should be within the capabilities of every company, according to Nadella. Another thing he said is that businesses should have control over the models and weights that incorporate their own information. He expressed Microsoft's support for the necessary research, attention, and "deliberate pacing" to achieve proper AI alignment in design. In addition, Nadella was receptive to suggestions like "embedded evaluators" and larger initiatives to create systems that can implement AI safety standards.

Nadella emphasised that a handful of businesses should not have control over the development of powerful AI. Academics and others from various corners of the AI ecosystem should be heard, he argued. The most important thing, according to Nadella, is that no one entity should be able to dominate it. Microsoft's strategy, he added, is enterprise control over learning loops and models coupled with wide access and choice across the whole AI stack. Additionally, he said that the following day, Microsoft would bring its first-party MAI models' underlying "Code of Conduct" to public consultation.

Evan Hubinger Highlighting Ongoing Challenges in AI

Nadella's comments follow a famous anthropologist's resignation, in which the scientist accused major AI firms of jeopardising humanity's future by rushing headfirst into self-improving superintelligence. After Jacob Coxon, who oversaw pre-training research at OpenAI and Anthropic, resigned, the discussion heated up. Coxon stated that he departed from the AI lab due to his belief that there were insufficient controls in place for the development of self-improving systems.

Anthropologist Evan Hubinger made an even more disturbing confession after he resigned. He stated his belief that the likelihood of AI eradicating humans during the next decade was greater than 10%. Additionally, Hubinger admitted that Anthropic was not yet prepared to address alignment for superintelligence and was not making sufficient progress in this area. As a result of these worries, recursive self-improvement has come to the fore, or the idea that AI systems can assist in creating ever better versions of themselves.

Concerns have been voiced by researchers about the possibility that this cycle could lead to artificial intelligence capacities outpacing human comprehension, evaluation, and control.