I track tickets like most people: poorly. The backlog has 240 open work items in it, the average age is 71 days, and roughly a third are duplicates of each other under slightly different wording.
A month ago I wired the Azure DevOps MCP Server into VS Code with GitHub Copilot agent mode and decided to use it as my primary triage interface. No more browser tab on the Boards UI. Just the editor and natural language. Here's what changed and what didn't.
The setup is faster than the docs make it sound
The GitHub repo for the Azure DevOps MCP Server has a 30-line setup section that took me three minutes to follow. The whole thing is:
{
"mcpServers": {
"azure-devops": {
"command": "npx",
"args": ["-y", "@azure-devops/mcp-server"],
"env": {
"AZURE_DEVOPS_ORG": "[YOUR-ORG]",
"AZURE_DEVOPS_PAT": "${input:ado_pat}"
}
}
}
}
Drop that in .vscode/mcp.json, paste a PAT into the input prompt, restart VS Code, done. The PAT needs Work Items (read & write), Code (read), and Build (read) — that's it.
The first thing that surprised me
I expected the killer feature to be "create a work item from natural language." It is not.
The killer feature is searching across work items with phrasing the search bar can't handle. Things like:
"Find every bug filed in the last 90 days where the area path includes 'payments' and the linked PR was reverted."
The Boards UI cannot answer that question. WIQL (the Azure Boards query language) technically can, but writing the WIQL takes longer than just writing the prose. The MCP server lets me ask the question in English, the agent translates it to WIQL, runs it, and shows me a markdown table.
Six of the bugs that surfaced from that exact query were duplicates of an open work item from four weeks earlier. We hadn't noticed because they were filed by different people in different area paths.
What I do daily now
Three workflows have replaced what I used to do in the browser:
Standup prep. "Show me my work items in active status, sorted by last touched, with a one-line summary of the latest comment on each." The agent fetches them, summarizes per item, and I copy-paste the summary into our standup channel. Five minutes saved daily, and the standup is more useful because I'm actually reading the latest comments instead of going off memory.
Triage of new bugs. "Look at every new bug in the last 24 hours. For each, find the most similar existing bug from the last six months and tell me whether it's likely a duplicate." The agent flags about 40% as likely duplicates with reasoning. I close roughly half of those flagged duplicates — net wins, even with the false-positive rate.
Sprint retro data pull. "For the sprint just closed, give me cycle time per work item type and call out items that took more than 2x the team median." Used to take me half an hour in Excel. Now thirty seconds.
What I expected to use but don't
I thought I'd ask the agent to create work items from chat messages. ("Hey Claude, file a bug for the thing the on-call mentioned in #incidents about the timeout.") I tried it for a week. It didn't stick. The work item the agent files is technically correct but lacks the texture I add when I'm filing manually — links to the relevant runbook, the specific commit hash from the deploy that introduced it, the slack thread permalink.
It turns out filing a work item is the part of triage where the human context I carry actually matters. Fetching, summarizing, comparing — those are the parts where I was just being a database query.
The gotcha that almost killed it for me
For the first week the agent kept inventing work item IDs. I'd ask it to "show me the latest comment on bug 4827" and it would respond with detailed text. The text was hallucinated. Bug 4827 didn't exist.
The fix was to add to my Copilot custom instructions:
Whenever you reference a work item by ID, you MUST first call the Azure DevOps MCP server to fetch it. Never paraphrase or summarize a work item from memory. If the fetch fails, say so explicitly rather than guessing at content.
After adding that, the hallucinations stopped. This is a Copilot configuration issue more than an MCP server issue — but if you're evaluating, know that you'll need that guardrail.
What the experience reminds me of
Honestly? It reminds me of when our team adopted Slack search slash-commands a few years back. Same shape: stop context-switching to a separate UI, ask the question in the place you already are. The Boards UI isn't bad — it's that any UI is slower than typing the question.
The MCP server isn't smarter than me. It's a query layer over Azure DevOps that speaks English. That's enough.
What I'd do next
Set up the same pattern for the Azure SRE Agent when it leaves preview. The premise — natural-language triage of incidents and pipeline failures — is the same shape, and if the latency holds up I'd let it own the first 60 seconds of any pipeline failure investigation.
I would not roll out MCP-driven backlog triage to a team without first writing internal docs about the hallucination guardrail. New users hit it in week one and lose trust.
Conversation
Reactions & commentsLiked this? Tap a reaction. Want to push back, share a war story, or ask a follow-up? Drop a comment below — replies are threaded and markdown works.