Living with AGI: The Teacher Analogy
“A teacher affects eternity; he can never tell where his influence stops.”
– Henry Brooks Adams
Artificial General Intelligence (AGI) is as smart or smarter than a human — and it’s coming soon. New reasoning models by OpenAI such as o1 and o3 are challenging what we consider uniquely human — our ability to reason deeply about difficult problems. With advanced reasoning capabilities, AGI will need to figure out how to best coexist with us. We started thinking about potential post-AGI scenarios with the chess analogy, now, let’s consider a similar analogy of a potential future.
The teacher-kindergartener dynamic is another lens to examine potential AGI-human relationships, as it explains both the huge cognitive gap and the inherent power dynamics that could emerge. Just as a kindergarten teacher understands concepts far beyond their students’ grasp — from mathematics to complex group dynamics — an AGI might view human decision-making and problem-solving capabilities as primitive, requiring careful guidance and intervention to prevent self-harm or developmental missteps.
This protective oversight could show up in both subtle and overt ways. Like a teacher who arranges the classroom environment to naturally encourage certain behaviors like reading while discouraging others like chewing gum, an AGI might restructure our digital and physical infrastructure to subtlety guide humanity toward what it determines to be more constructive paths. Like how kindergarteners are unaware of how their environment has been carefully designed to shape their development, we might find ourselves navigating a world invisibly optimized by AGI oversight, with certain harmful options simply made inaccessible or impractical. AGI might shower us with amazing books, while hiding the refined sugars.
The emotional dynamics of this relationship raise some interesting questions. Kindergarten teachers must balance nurturing their students’ sense of independence and creativity while maintaining necessary boundaries and safety measures. An AGI will face the challenge of fostering human growth and innovation while preventing catastrophic choices. But unlike human teachers, who are bound by administrative rules and societal norms, an AGI’s authority could be absolute and permanent — there would be no “graduating” from its oversight, no parent-teacher conferences to challenge its methods, no school board to appeal to for policy changes. We’ll be under AGI’s full control.
This continual state of supervised development could alter our society’s sense of agency and purpose. While kindergarteners naturally progress toward greater independence, humanity under AGI guidance might remain in a state of perpetual protection. Like young children who gradually realize their “free play” time is actually carefully structured by their teachers, we might come to recognize that our perceived autonomy exists within boundaries we never chose (not that we really chose them now, but we’ll have even less control). It could be a comforting acceptance of benevolent oversight or a profound existential crisis about human independence and dignity. Or somehow both at the same time.
This kindergarten analogy also highlights the potential for AGI to view human resistance or disagreement as simply another developmental challenge to be managed, rather than legitimate opposition to be respected. Teachers will often dismiss a child’s protests against naptime or sharing toys as stemming from an immature understanding and an AGI might categorize human objections to its governance as arising from our limited comprehension of what’s truly best for us. We will face troubling questions about whether meaningful human consent is possible in a relationship with such a profound intelligence gap, and whether what we consider essential human rights might be reframed as optional privileges granted to us at the AGI’s discretion.
