Last Updated on October 21, 2026
What technology professionals building with AI told us about the gap between experimentation and production, and the skills worth developing next.
Why we surveyed developers about AI
Throughout 2026, a pattern kept coming up in conversations with developers and organizations we work with: AI prototypes are multiplying, but getting them into the hands of real users remains difficult. We wanted to test that observation against what builders were experiencing.
Between August and September 2026, Udacity surveyed 1,585 developers and technology professionals. Of those, 1,146 reported actively building with AI or large language models in the past 12 months. This article details the insights derived from their responses and experiences on the frontlines of AI integration.
We asked how much AI work reaches production, what stalls it, which skills matter for deployment and career advancement, and what builders believe demonstrates those skills to hiring managers.
The gap between prototype and production
85% of builders had worked on AI or LLM projects in the past year, and 30% had worked on six or more. Experimentation is widespread. Deployment is not.
34% had not shipped a single AI project to production. Only 16% said all of their projects had reached production.
Among builders whose projects did not reach production, most did not stall early. 40% stalled at a working demo or prototype, and another 33% at an internal pilot or limited rollout.
What gets in the way? The most commonly cited barriers were budget or resources (37%), unclear business value or ROI (35%), security, compliance, or safety concerns (25%), and lack of leadership support (24%). Technical obstacles still mattered: 22% cited inconsistent outputs, 16% model limitations, and 14% a lack of technical skills.
The lesson is not that models or engineers are blameless. It is that getting to production requires more than a functioning prototype: teams must establish value, earn trust, and navigate organizational constraints.
Which skills help AI reach production?
When asked to select up to three skills most important for getting AI projects into production, builders put evaluation and testing (45%) and safety, guardrails, and compliance (44%) at the top. Deployment and operations followed at 35%.
Those labels represent practical engineering work: defining what good outputs look like, testing behavior against representative cases, controlling unsafe or unauthorized actions, and monitoring systems once deployed.
An interesting contrast emerges when builders consider career advancement. Integrating AI into existing software was selected by 42% for career advancement versus 30% for production; multi-agent orchestration by 29% versus 20%. Evaluation and testing was selected by 36% for career advancement versus 45% for production, while observability was 22% versus 27%. Safety and guardrails remained prominent in both (41% versus 44%).
This is not a move away from reliability: safety and guardrails stayed near the top for both production and careers (44% versus 41%). The real shift is in emphasis. Integration and orchestration draw more weight when builders think about careers, while evaluation and observability draw more weight when they think about getting to production. The opportunity sits with engineers who can do both: build ambitious AI systems and show that those systems are reliable and deployable.
AI skills and the changing engineering role
For career advancement over the next one to three years, 68% of respondents said progressing without strong AI skills would be somewhat or very unlikely. Among those who had shipped AI to production, 35% chose the strongest response, compared with 26% among those who had not.
Builders also anticipate changes in day-to-day work: 51% expect hand-coding and 63% expect memorizing syntax and APIs to become obsolete within one to three years. Asked which skills will keep engineers relevant as AI handles more coding, 50% selected system design and architecture, more than any other option.
The implication is less “coding disappears” than “engineering judgment becomes more visible”: deciding how components fit together, defining requirements, and validating what AI produces.
What counts as proof of AI engineering ability?
Asked what best demonstrates AI engineering skills to a hiring manager, 38% of builders chose an AI project shipped to production, 28% a portfolio of projects or demos, and 11% code samples or open-source contributions. Together, 77% selected a form of tangible work.
The format varies, but the practical takeaway is to make your work inspectable. Explain the problem, the architecture, the trade-offs, the tests, and, where applicable, the production results.
What this means for engineers building AI skills
- Go beyond the demo. Practice taking an AI feature through integration, testing, and deployment, not just getting a compelling first response.
- Treat evaluation as engineering. Define success criteria, assemble representative test cases, and check regressions as prompts, tools, or models change.
- Build for safe, observable behavior. Consider access controls, failure handling, monitoring, and the limits of autonomous actions.
- Show your decisions, not just your output. A project portfolio is stronger when it explains system design, trade-offs, reliability checks, and what you learned.
Built for the gap: AI Engineering with Claude
The survey highlights how strongly builders value evaluation, safety, and reliable deployment. Udacity’s AI Engineering with Claude Nanodegree program is designed to help developers practice related capabilities: agent design and control, evaluation and observability, safety and guardrails, and multi-agent orchestration.
Across four courses, learners work through scenarios involving insurance claims, retail inventory, supply chain risk, and enterprise code review, completing 13 hands-on projects. The value of that work is not simply the project count; it is the opportunity to demonstrate how an AI system is designed, evaluated, and controlled.
For developers aiming to move beyond prototypes, the goal is to build evidence of engineering judgment alongside working AI applications.
The findings in full
Below is the full data set from the survey. Each percentage is of the people who answered that question, and the base changes from question to question because of survey routing. Questions marked “up to 3” let people pick more than one skill, so those lists add to more than 100%.
How much teams are building
AI and LLM work over the past 12 months (n=1,351)
- Mostly experiments, prototypes, or proofs of concept: 34%
- About a 50/50 mix of experiments and products for end users: 33%
- Mostly products for end users: 18%
- Have not worked on AI or LLM projects in the past 12 months: 15%
Number of AI or LLM projects worked on (n=997)
- 1 to 3: 45%
- 4 to 5: 25%
- 6 to 10: 14%
- More than 10: 16%
What reaches production
At least one project reached production (n=997)
- Yes: 66%
- No: 34%
Share of projects that never reached production (n=997)
- 0, all deployed: 16%
- About 1 to 25%: 25%
- About 26 to 50%: 21%
- About 51 to 75%: 13%
- About 76 to 100%: 24%
Where projects stalled, among builders with at least one that did not reach production (n=691)
- Working demo or prototype, but no rollout: 40%
- Internal pilot or limited rollout, but no production deployment: 33%
- Built, but never reached a working prototype: 11%
- Planning stage, never built: 6%
- Reached production, but was later paused, rolled back, or not adopted: 6%
- All projects reached production: 4%
This question was routed to builders reporting projects that did not reach production, but about 10% still selected a production outcome (reached production but later paused, or all projects reached production). So n=691 is not a clean never-shipped base, and these are reported stall stages rather than a measured conversion funnel.
What most often kept projects from reaching production (n=691, up to 3)
- Budget or resources: 37%
- Unclear business value or ROI: 35%
- Security, compliance, or safety concerns: 25%
- Data quality or system integration: 24%
- Lack of leadership or organizational support: 24%
- Inconsistent or unreliable outputs: 22%
- Limitations of the underlying model: 16%
- Lack of the right technical skills: 14%
The skills that matter
Skills that matter most for getting projects into production (n=838, up to 3)
- Evaluation and testing: 45%
- Safety, guardrails, and compliance: 44%
- Deployment and operations (reliability, cost, CI/CD): 35%
- Prompt and context engineering: 32%
- Integrating AI into existing software: 30%
- Observability and monitoring in production: 27%
- Data retrieval and tool integration (RAG, function calling): 26%
- Multi-agent orchestration: 20%
As AI assistants handle more of the coding, skills to stay relevant (n=895, up to 3)
- System design and architecture: 50%
- Prompting, reviewing, and orchestrating AI coding assistants and agents: 39%
- Evaluating and testing AI systems for reliability: 36%
- Building guardrails, safety, and governance: 34%
- Connecting AI to data, tools, and existing systems: 33%
- Task decomposition and requirements definition: 29%
- Product and business judgment: 27%
- Debugging and troubleshooting AI systems: 26%
Skills expected to become obsolete in the next 1 to 3 years (n=897, up to 3)
- Memorizing language syntax and APIs: 63%
- Hand-coding: 51%
- Writing documentation: 45%
- Routine debugging and syntax error fixing: 29%
- Manual testing and QA: 25%
- Learning specific programming languages or frameworks: 23%
- Data wrangling and scripting: 16%
Careers and hiring
Can an engineer advance without strong AI skills over the next 1 to 3 years? (n=897)
- Somewhat unlikely: 36%
- Very unlikely, not possible to advance without AI skills: 32%
- Somewhat likely: 20%
- Very likely, AI skills are not required to advance: 12%
Builders who have shipped AI to production hold this view more strongly. Among those who shipped at least one project, 35% said advancing without AI skills is not possible. Among those who never shipped, 26% said the same.
Skills that matter most for career advancement (n=895, up to 3)
- Integrating AI into existing software: 42%
- Safety, guardrails, and compliance: 41%
- Prompt and context engineering: 40%
- Evaluation and testing: 36%
- Deployment and operations (reliability, cost, CI/CD): 31%
- Data retrieval and tool integration (RAG, function calling): 30%
- Multi-agent orchestration: 29%
- Observability and monitoring in production: 22%
Best way to demonstrate AI engineering skills to a hiring manager (n=897)
- An AI project you shipped to production: 38%
- A portfolio of AI projects or demos: 28%
- Code samples, GitHub, or open-source contributions: 11%
- Performance in a technical interview or take-home: 8%
- Years of experience on the job: 5%
- A certificate for a completed course: 4%
- A relevant academic degree: 3%
Who we surveyed
Role (n=874)
- Software development: 27%
- Entrepreneur: 14%
- Data science or data engineering: 13%
- IT: 10%
- Artificial intelligence engineer: 10%
- Operations: 9%
- Product development: 8%
- Marketing: 4%
- Finance or accounting: 2%
- Sales: 2%
- Manufacturing: 1%
- Human resources: 1%
Seniority (n=874)
- Individual contributor: 56%
- Manager: 19%
- C-suite executive or equivalent: 12%
- Director: 9%
- Vice President: 3%
Industry (n=874)
- Technology / software: 45%
- Education: 10%
- Healthcare: 8%
- Government / non-profit: 7%
- Finance / banking: 7%
- Retail / e-commerce: 6%
- Manufacturing: 6%
- Other: 11%
Employment status (n=874)
- Employed full-time: 59%
- Self-employed: 17%
- Unemployed, looking for work: 9%
- Employed part-time: 7%
- Retired: 4%
- Full-time student: 3%
- Unemployed, not looking for work: 2%
Generation (n=874)
- Millennial (born 1981-1996): 40%
- Generation X (born 1965-1980): 37%
- Generation Z (born 1997-2012): 11%
- Baby Boomer (born 1946-1964): 10%
- Other: 2%
Education (n=874)
- Graduate or professional degree: 47%
- Bachelor’s degree or equivalent: 35%
- Some higher education, no degree: 8%
- High school diploma or equivalent: 5%
- 2-year degree: 4%
- Less than high school: 1%
Methodology
Udacity surveyed 1,585 developers and technology professionals between August and September 2026. Of those, 1,146 reported actively building with AI or large language models in the past 12 months, and the analysis draws on their responses. Question bases vary because of survey routing, so each finding in the full results is shown with its own base. Respondents span roles including software development, data science, AI engineering, and entrepreneurship, and represent industries including technology, healthcare, finance, and education. The survey reflects their perspectives on the gap between AI experimentation and production deployment, the skills required to close it, and the career stakes of AI proficiency.
