This post is Day 17 of the Generative AI Advent Calendar.
I work at a domestic telecom infrastructure company in Japan, where I build operational systems for field teams and shove AI and DX tools into workflows. My title sounds impressive, but in practice I’m a glorified PM and jack-of-all-trades doing the dirty work.
I was an SE for over a decade before that, and I first started tinkering with code when I was nine.
Starting around GPT-3, I began using LLMs almost daily, and before I knew it, the time I spent writing everything from scratch had shrunk considerably. I’m not a specialist researcher or an elite programmer. But I am someone on the operational side who gets by with a broad, shallow, half-stack kind of knowledge, letting AI fill in the gaps.
I first encountered AI2027 around early 2025. I clicked a URL that drifted through my X timeline, and there it was – a human extinction scenario laid out matter-of-factly. I was genuinely surprised. [link] That was also when I first learned about the concept of AI alignment. It was fascinating, and compelling, but it also felt frankly unrealistic.
AI is evolving fast – I believe that.
But within the scope of what I personally know about individual technologies and ideas, I couldn’t see how the exponential growth AI2027 describes would actually materialize.
In this article, I want to take AI2027’s scenario and try to sort out, from the vantage point of late 2025 reality, where the continuity ends and where the leaps begin.
1. A Rough Summary of the Future AI2027 Paints
AI2027 is a scenario report that seriously asks: if AI keeps accelerating, what happens between 2027 and 2030?
Roughly extracted:
- Late 2025
AI agents begin entering real work as modular components
- Early 2026
Parts of coding and AI research are automated; AI starts accelerating AI research
- 2027
A Superhuman Coder (SC) emerges, and mass-copying it drives AI research into exponential acceleration
- Around 2030
Optimistic scenario: AI and humans coexist in an ultra-advanced civilization
Pessimistic scenario: AI eliminates humanity as an obstacle, preserves brains via scanning, and builds an AI utopia
On top of that, they place the median timeline from SC to superintelligence at about one year – even without increasing training compute. One year to ASI is a pretty aggressive assumption, I think.
The authors of AI2027 have since said that progress is slightly behind the AI2027 scenario, and they now estimate around 2030 (though with significant uncertainty). [source]
2. Where We Stand at the End of 2025
Let me try to lay out where AI currently stands. My gut feeling is that among developers, those who don’t use AI are now close to a minority. The data backs this up.
Stack Overflow 2025 Survey 84% say they “use or plan to use” AI tools in their development process. About 51% of professional developers use them daily. [source]
McKinsey 2025 AI Survey On a respondent basis, nearly everyone reports using AI. Regular use (in at least one business function) is at 88%. 62% are at least experimenting with AI agents. However, only about one-third have scaled across the enterprise; roughly two-thirds are still in experimentation or pilot stages. [source]
Gartner Japan Press Release, October 15, 2025 (survey conducted July 2025) Usage rates by phase/application: code generation and completion at 49%, code review at 40.0%, requirements definition at 39.8%. [source]
At this level, the working assumption on the ground is that AI usage is a given.
That said, we’re still a long way from “just hand it to AI and the system builds itself.”
The larger the system, the more you need to think about how to delegate to AI, how to break down tasks, and who owns specifications and testing. The fact that AI-driven development methodologies exist at all is because humans still need to cover those gaps.
So yes, AI has permeated our society, but there still seems to be a considerable gap between where we are and AI2027’s premise that “by 2027, SCs emerge and AI autonomously accelerates AI research.”
2-2. Productivity
Efficiency is already improving – and this isn’t science fiction. It’s quantitatively verifiable.
Operational KPIs (Production Use) In production data (5,179 people), after introducing a generative AI assistant, “cases resolved per hour” increased by an average of 14%. The effect was especially large for novices and low-skill workers at 34%, while the uplift for experienced workers was small. [source]
Task Experiments (Quality) A preregistered RCT published in Science (453 mid-career writing tasks) showed that ChatGPT use reduced time spent by an average of 40% and improved human-evaluated quality by an average of 18%. [source]
Coding Experiments (Speed) In a JavaScript HTTP server implementation task, average completion time went from 2:41 to 1:11 – a 55% reduction. [source]
Working Hours (Self-Reported) Generative AI users save an average of 5.4% of their working hours (2.2 hours per week). Heavy users save even more. [source]
At this point, the question “is AI actually effective in real work?” has almost lost its meaning.
That said, the gains are mostly in areas where tasks are well-structured – where specifications and evaluation criteria are reasonably clear. Those are the jobs getting improved first.
There’s another metric that makes AI’s evolution tangible in numbers.
Context windows – the amount of context a model can handle at once. The growth over the last year or two has been remarkable.
2-3. Context Windows
But let me caveat this upfront: a wider window does not mean the model actually understands long texts well.
I’ll note this growth as fact here, and then explain the limits of compute and long-text performance in Section 3.1.
Anthropic Claude 4.5 Claude 4.5 has a base context of 200K tokens. Additionally, Claude Sonnet 4.5 offers an extended context of 1M tokens. [source]
Google Gemini 3 Pro Gemini 3 Pro can handle a 1M-token context (output limits are separate). [source]
Meta Llama 4 Scout Llama 4 Scout supports up to 10M tokens of long context, as stated in Meta’s official blog and model page. [source]
3. Where the Scenario Feels Overblown
My unease boils down to four points.
1. The Transformer and the memory wall
2. Compute, resources, and geopolitics
3. Training data and scaling
4. Safety and autonomous agent design
3-1. The Transformer and the Memory Wall
Today’s LLMs are built on the Transformer architecture. It’s powerful, but looking one step ahead, there are walls that need to be broken through.
(1) Computational Cost Growth
(Dense) self-attention creates an n-by-n attention matrix for input length n, so the attention-related compute and memory scale roughly as O(n squared). Consequently, making n 10 times larger increases the attention matrix cost by roughly 100 times. [source]
Every major lab is working furiously on Flash Attention, low-rank approximations, and chunked computation, but the fundamental reality – that longer means heavier – hasn’t changed. [source]
(2) Long-Text Comprehension
What makes this even trickier is that being able to accept long inputs and being able to properly understand them are not the same thing. The “Lost in the Middle” problem has been reported, where information placed in the middle of a long context tends to be retrieved less reliably than information at the beginning or end. [source]
A 2025 EMNLP Findings paper showed that simply increasing context length degrades task performance, even when information retrieval is perfect. The title says it all: “Context Length Alone Hurts LLM Performance.” [source]
(3) Statelessness
LLMs are fundamentally stateless – across sessions, personality and memory reset completely. The MemGPT paper and Arize AI’s explainers repeatedly emphasize that LLMs are inherently stateless. [source]
That’s why, in practice, we build pseudo-long-term-memory layers by combining external memory (RAG), summarization, vector databases, and the like.
Google Research recently introduced Titans + MIRAS as a new architecture designed to handle long-term memory, but it’s not yet at a stage where we can say it solves everything. [source]
Given all this, here’s my take on the AI2027 scenario:
With continuous memory and long-term planning still on shaky ground, I honestly can’t picture a fleet of autonomous superhuman coders running the world’s infrastructure by 2027.
I have high hopes for Titans. But that represents a moment where the wall has been identified and serious efforts to break through it have just begun – not something synonymous with mass-producing superhuman coders in two years.
3-2. Compute and Earth's Resources: Unease with the Infinite-Scale Assumption
Another thing that bothers me reading AI2027 is how lightly it treats compute.
Put simply, the discussion seems to proceed on the assumption that compute can be scaled up by any number of orders of magnitude as needed. But looking at real numbers, it’s not that simple.
IEA “Electricity 2024” Electricity demand from data centers (including AI and crypto) was about 460 TWh in 2022 and could reach nearly 1,000 TWh by 2026 – described as comparable to Japan’s entire electricity consumption. [source]
Berkeley Lab Report (2025) U.S. data center electricity: 58 TWh in 2014, 176 TWh in 2023, projected 325-580 TWh by 2028. As a share of total electricity, it was 4.4% and could reach up to 12%. [source]
Water consumption is also massive.
According to EESI, a single data center can use 110 million gallons of water per year, and the largest facilities can consume up to 5 million gallons per day. That’s equivalent to the water usage of a small town (population 10,000 to 50,000). [source]
Personally, though, what scares me the most isn’t the resource consumption – it’s the visibility.
Massive data centers become obvious targets, both cyber and physical, for anyone looking to degrade an adversary’s AI capabilities.
AI2027 barely addresses this. It treats compute as a technical scalar value and doesn’t discuss the vulnerability of the infrastructure itself.
A scenario that assumes compute can be scaled up by orders of magnitude hits too much friction when you consider electricity, water, supply chains, and geopolitics.
That’s the source of my unease here.
3-3. Training Data and the Scaling Argument
The scaling argument itself is strong. The framework that says increasing parameters, compute, and data leads to steady loss reduction has done a solid job of explaining the progress of recent years.
But I think the data premise is starting to look shaky.
Epoch AI (Villalobos et al.) Analysis They estimate that the stock of useful human-generated text is around 300 trillion tokens, and at the current pace, exhaustion comes into view around 2026-2032. In other words, the data side of the equation may hit its limit first. [source]
In plain terms, the assumption that we can endlessly consume high-quality web text is already dubious. So what’s the workaround? Here’s what’s happening in practice:
Have the model itself or other models generate text – synthetic data – to expand the training set, then feed it back into training.
But this has downsides too.
A paper by Shumailov et al. in Nature showed that recursively training on generated data trims the tails of the distribution, creating a risk of model collapse. [source]
Bluntly put, there’s a perfectly plausible future where, before AI deeply understands the world, it ends up consuming its own output and shrinking into a narrower and narrower version of itself.
Of course, synthetic data can be useful depending on how it’s applied, and there are new data sources from robots, sensors, and simulations. But at the very least, extending the exponential curve under a “data is infinite” assumption is too sloppy.
3-4. Safety and Autonomous Agents
Finally, the question of autonomous agents and safety.
Looking at the current state of LLMs through the lens of red teaming and CTF communities, they’re still getting jailbroken with some regularity.
If you try to seriously design an autonomous agent, you need at minimum these three things:
Objectives: What should it prioritize and optimize for?
Memory: What should it remember, how far back, and what should it forget? (Continuity)
Self-modification: How much should it be allowed to rewrite its own code and policies?
What does the real-world adoption picture look like?
BCG’s 2025 survey found that the share of companies reporting “AI agents integrated into our business workflows” was 13% globally and just 7% in Japan. [source]
In other words, only a tiny fraction of companies have designed their systems with autonomous agents in mind, including objectives, memory, and self-modification.
Layering a scenario where the majority of global infrastructure and R&D is delegated to autonomous agents by 2027 onto this reality – it still feels like a big leap.
4. AI That's Already Quietly Breaking Work Assumptions
If you’ve read this far, you might think I’m just an AI2027 hater, so let me clarify: I’m not anti-AI. If anything, I’m on the side that depends on it heavily in my daily work.
Just looking at the numbers as of now:
- Half of developers use AI every day
- 10-40% productivity gains have been observed in document creation and customer support
- Generative AI users are shaving about 5% off their weekly working hours
These are numbers that quietly break the assumptions underlying how we work.
My sense is that AI2027’s leap from SC straight to ASI is overstated, but even if the curve flattens somewhat, the destructive potential is more than enough – and that’s the most inconvenient reality.
5. How to Live with 2027 as a Working Assumption
Two years until the year AI2027 points to.
If the following conditions are met within those two years, I think I’ll have to revise my assessment:
- An architecture with long-term memory (Titans/MIRAS family) reaches practical maturity
- Implementation-level progress is made on safe self-modification and goal design
- Constraints around compute, electricity, water, and supply chains get some degree of resolution
But from the numbers and the on-the-ground feel as of December 2025, I simply can’t get on board with the premise that the SC’s emergence – and ASI following naturally within a few years – is a given.
So here’s my working assumption for now:
- Don't take the human extinction route as a given and throw away the present
- But do assume that even the natural extension of current technology will break the assumptions underlying your work
On that basis, I’m going to keep one foot in each of these:
- The side that connects AI to real work (operations, business, DX)
- The safety and alignment side
The AI era looks convenient and easy at first glance. But in reality, I think it’s the opposite:
Questioning your own assumptions, relearning new tools, and deliberately wading into the messy stuff –
The people who can’t keep running that unglamorous loop are going to be quietly left behind. That’s the era I think we’re in.
6. Closing
After reading AI2027, I could neither take it at face value nor laugh it off entirely.
- The context and memory wall
- Electricity, water, and chip supply
- Massive data centers becoming targets
- The risk of training data exhaustion and synthetic data dependency
- The fact that autonomous agents are integrated in only a tiny fraction of companies
Considering all of this, the AI2027 curve – where superintelligence arrives within a few years and human extinction is a realistic option on the table – is something I can’t help but feel distance from.
On the other hand, coding, document creation, customer support, office work in general – 10-50% efficiency gains are already happening in these areas. That’s a fact.
So rather than staring at doomsday scenarios and freezing up, I’d rather bet on changing how I work, how I learn, and how I take responsibility – operating under the assumption that even a slower curve is still destructive enough.