Is (Artificial) Intelligence Unbounded?

Discussion Excerpt: Upper Bound on Intelligence

Dwarkesh Patel asks whether there is an upper bound on intelligence, particularly in the context of superintelligence, and how far it might extend.

Paul Christiano responds speculatively, framing the question similarly to asking if there is an upper bound on strength. He explains:

• There are arbitrarily smart input-output functionalities if no constraints are placed on compute or description complexity.
• If compute is held fixed, there exists a "smartest" functionality among a finite set of possibilities (e.g., the best set of operations given a fixed number of compute steps).
• For unbounded intelligence—where arbitrary description complexity and compute are allowed—there may not be a strict upper bound in theory, as there exists an "optimal conduct" for achieving a given goal.

However, this optimal intelligence would be impractical to instantiate in the physical universe due to extreme computational inefficiency. For example:

• An optimal "little box" might involve simulating every possible universe it could inhabit, which would be exponentially or even double-exponentially slow.
• Physical constraints (e.g., heat dissipation) could make it impossible to realize such intelligence in practice.