3 posts tagged with "automation"

Introduction​

I came back to my AI assistant's terminal and could not find its answer. Notifications had pushed it out of view. The assistant had done the work. I still had to find it.

This was happening often enough that I told it: "the moment I cycle back, I never see your response". I move between messages, browser tabs, pull requests and terminal sessions. An answer that needs me to watch a pane at the right moment does not fit that loop.

The same assistant also watched incoming events, answered one-off questions, read logs, supervised workers and learned from my corrections. Every new responsibility brought tools and instructions. Its role file more than tripled in length in a little over a month. I had hired an assistant and given it my entire inbox.

Hence a wealth of information creates a poverty of attention.

Herbert A. Simon, Designing Organizations for an Information-Rich World, 1971

Introduction​

RSI stands for Recursive Self-Improvement: an AI system that improves itself, then uses the improved version to improve itself again. It is one of the milestones AI is expected to reach one day. In 1965, the mathematician I. J. Good put it like this:

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind.

I. J. Good, Speculations Concerning the First Ultraintelligent Machine, 1965

I have given a talk about this topic at the WorldWide Architecture Summit 2 on the 3rd of August 2021. You can find the talk recording here.