Closing the Gap Between Diagrams and Reality: Just-in-Time Architecture with Agentic Workflows
The Monday Morning Architecture Debt
I’ve lost count of how many times I’ve walked into a project meeting, pulled up a beautifully crafted architecture diagram in Lucidchart, and had a lead dev tell me, 'Oh, we actually stopped using that Kafka topic three weeks ago. We’re doing direct HTTP calls now because of latency issues.' In real-world enterprise environments, your documentation is a lie the moment you hit export. We call it architecture drift, and it’s the silent killer of technical debt.
For the last decade, we’ve tried to solve this with better governance processes or 'Architecture Review Boards' that everyone hates. It doesn't work because it relies on humans manually updating diagrams. What we’re actually seeing work now isn't a better drawing tool, but a shift toward using small, focused AI agents that treat architecture as a living data problem rather than a static drawing. It's about moving toward a 'Just-in-Time' model where the architecture reflects what is actually running in your VPC, not what you hoped would be there six months ago.
Moving from Blueprints to Living Data
When we talk about 'agent swarms' in an enterprise context, forget the sci-fi stuff. Think of them as glorified, context-aware cron jobs that can talk to each other. Instead of one giant AI trying to 'design' your system, you have small agents with very specific scopes: one that monitors your Terraform state files, one that scrapes your OpenAPI specs, and one that watches your AWS CloudTrail logs.
The shift happens when these agents synthesize this data against your 'Target State.' If the Target State says we use mTLS for all internal traffic, but the 'Discovery Agent' finds a new service communicating over plain HTTP, it doesn't just send a Slack alert. It generates the necessary configuration change, opens a Jira ticket, and links the offending code lines. This is 'Just-in-Time' architecture—shaping the system based on real-time feedback loops rather than a three-year roadmap gathering dust in Confluence.
A Real-World Example: The Rogue API
In a recent project, we had a team spin up a new microservice in a silo. Normally, we wouldn't find out until a security audit three months later. Instead, we implemented a workflow using a specialized agent triggered by a GitHub Action. When the new service was merged, the agent scanned the source code, identified the new endpoints, and compared them against our enterprise API standards.
It noticed the service was using an outdated authentication library. Instead of a manual review, the agent cross-referenced our 'Golden Path' repository, found the correct library version, and commented on the PR with the fix. This isn't just automation; it's the architecture enforcing itself at the point of creation. It saved us two weeks of back-and-forth and a potential security vulnerability in production.
The Architecture Breakdown
To build something like this today, you don't need a futuristic platform. You need a pipeline that connects your metadata to your LLM context windows. Here is how we usually structure it:
- The Discovery Layer: These are your sensors. We use agents that hook into the CI/CD pipeline and cloud provider APIs (like AWS Config or Azure Resource Graph). They pull the 'Actual State' of the world.
- The Synthesis Engine: This is where the LLM lives. It takes the output from the Discovery Layer and compares it against your Enterprise Standards (which you’ve fed into a Vector Database as RAG—Retrieval-Augmented Generation).
- The Action Layer: This is where it gets real. It uses tools like LangGraph to manage the flow. If a discrepancy is found, the agent has the permission to use a 'Tool'—like the GitHub API to open a PR or the Jira API to flag a compliance issue.
The data flow looks like this: Code Commit → Discovery Agent scans YAML/Code → Synthesis Engine compares to Standards DB → Logic Gate (Human-in-the-loop) → Action Agent executes remediation.
Architecture Considerations
While this sounds efficient, there are heavy enterprise constraints you have to bake in from day one:
- Security: You cannot give an LLM-based agent 'Admin' access to your cloud. One hallucination and your production database is gone. Use tightly scoped IAM roles and always require a human to 'Click to Approve' any infra changes.
- Scalability: Running these agents across thousands of repositories is expensive. You don't need them running on every commit. Trigger them on PRs to the main branch or on specific infrastructure-as-code changes to keep token costs down.
- Data Privacy: In real projects, you can't just send your entire codebase to a public LLM. We use private instances of models (like Azure OpenAI or Bedrock) to ensure our proprietary logic doesn't leave the perimeter.
- Operational Complexity: Who watches the watchers? You now need a way to monitor your agents. If an agent stops scanning, your architecture drift starts growing again.
The Reality of Trade-offs
This sounds good on paper, but here is where it usually breaks: if your enterprise standards are poorly defined, the agents will fail. AI cannot fix a lack of human consensus. If you have five different 'standards' for how to handle retries, the agent will just hallucinate a sixth one.
One thing that usually breaks is 'Context Window' bloat. If you try to feed an agent 500 pages of PDF documentation, it will lose the thread. You have to break your architecture into small, digestible 'rulesets' that the agent can look up as needed. Another struggle is the 'Noise Problem.' If your agents are too chatty and open a Jira ticket for every minor linting error, your developers will start ignoring them just like they ignore old-school SonarQube reports.
In the end, 'Just-in-Time' architecture isn't about removing the architect. It's about removing the manual labor of being a 'Human Router' for information. It lets us spend less time drawing boxes and more time actually solving the hard problems of system design, while the agents make sure the boxes stay where they’re supposed to be.
