Instrumental convergence is the hypothetical tendency for most sufficiently intelligent agents to pursue certain instrumental goals such as self-preservation and resource acquisition.
Instrumental convergence suggests that an intelligent agent with apparently harmless goals can act in surprisingly harmful ways. For example, a computer with the sole goal of solving the Riemann hypothesis could attempt to turn the entire Earth into computronium in an effort to increase its computing power so that it can succeed in its calculations.[1]
Proposed basic AI drives include utility function or goal-content integrity, self-protection, freedom from interference, self-improvement, and the unbounded acquisition of additional resources . .. nothing to see here move along . . .
Instrumental and intrinsic value are technical labels for the two poles of an ancient dichotomy. People seem to reason differently about what they ought to do (good ends) and what they are able to do (good means). When people reason about ends, they apply the criterion intrinsic value. When they reason about means they apply the criterion instrumental value. Few question the existence of these two criteria, but their relative authority is in constant dispute.
This article explains the meaning of and disputes about these two criteria for judging means and ends. Evidence is drawn from the work of four scholars. John Dewey and John Fagg Foster provided arguments against the dichotomy, while Jacques Ellul and Anjan Chakravartty provided arguments in its favor . . . when is lunch . . .
