Artificial intelligence firm OpenAI says superintelligence is the ‘most effective’ tech ever invented
Ilya Sutskever, the co-founder of OpenAI, a prominent artificial intelligence organization, along with Jan Leike, head of the alignment, has issued a warning about the necessity of controlling superintelligence to avert the potential extinction of the human race.
In a blog post published on Tuesday, they asserted that superintelligence has the potential to be the most influential technology ever created, offering solutions to some of humanity’s most critical issues.
However, they also emphasized the immense danger associated with the vast power of superintelligence, which could lead to the disempowerment or even extinction of mankind. Sutskever and Leike expressed their belief that such advancements could materialize within this decade.
They emphasized that effectively managing the associated risks would necessitate the establishment of new governance institutions and addressing the challenge of aligning superintelligence with human values. This alignment aims to ensure that AI systems that surpass human intelligence will adhere to human intent.
“In the present scenario, we lack a solution to steer or control a potentially superintelligent AI and prevent it from becoming rogue. Our current methods for aligning AI, such as reinforcement learning through human feedback, rely on human supervision of AI systems. However, as AI systems become significantly smarter than humans, our current alignment techniques will be inadequate,” they explained. “We require novel scientific and technical breakthroughs.”
In order to address these challenges, they announced the formation of a new team and pledged to allocate 20% of the computing power they have acquired thus far to this endeavor within a span of four years.
“While this objective is remarkably ambitious, and success is not guaranteed, we hold an optimistic outlook that through a dedicated and collaborative effort, we can find a solution to this problem,” they expressed.
In conjunction with their ongoing efforts to enhance existing OpenAI models such as ChatGPT and address associated risks, the newly formed team at OpenAI is concentrating on the machine learning obstacles involved in aligning superintelligent AI systems with human intent.
Their objective is to create an automated alignment researcher that reaches a level of capability comparable to humans. To achieve this, they will utilize substantial computational resources to scale the researcher and iteratively align it with superintelligence. OpenAI intends to develop a scalable training methodology, validate the resulting model, and subject its alignment pipeline to rigorous stress testing.
Notably, Microsoft is providing support for OpenAI‘s endeavors in this field.