The State Of AI Tools And Coding: 2023 Edition

We're sick and tired of the media and talking heads claiming that AI tools like ChatGPT are just going to replace programmers. What do they know? What data are they even basing this on?

That's the problem. There really is no data out there on how programmers are actually using ChatGPT or AI tools. Are they using them? Are they worried about them? Are their employers telling them to use them or not hiring developers anymore?

We wanted to find out the answers to these questions. So we put out a survey and gathered responses from June 3, 2023 to June 22, 2023 to find out if and how programmers are actually using AI tools.

We thought 1,000 responses would have been great, but the response was incredible. 3,240 people completed the survey, and it wasn't exactly short either!

Now 3,240 might not seem like a lot until you compare it to other AI surveys out there (or just other surveys in general).

McKinsey did one that had 1,492 respondents, one by the Verge had 2,000, and a recent developer specific survey by Github was based on only 500 developers (and every developer knows GitHub so their audience is pretty damn big).

We're not telling you this as a brag but so that you know these insights are based on a really solid sample size. And because many more people have heard of McKinsey, the Verge and GitHub.

Side note: The graphics and design on that Verge survey... so cool. Definitely check out those other surveys out as well.

But as far as we know, this is the biggest survey of developers on the topic of AI to date with responses from industry professionals across a variety of roles, companies, and levels of experience. Everyone from solopreneurs, contractors, new and long-term coders, and even team leads at 1,000+ employee companies.

Our Top 10 Takeaways

So now that you know the sample size is solid, here is a sneak peak into what we thought were some of the most interesting findings:

1. How many programmers are using these tools? 84.4% of the programmers surveyed have some level of experience with AI tools (list below). The top 5 are: ChatGPT (3.5 and 4), GitHub CoPilot, Midjourney, Visual Studio IntelliCode, and Bard
2. What AI tool is most used? ChatGPT reigns as king of AI tools (for now): 74.9% of developers are using it on a weekly basis and 41.4% on a daily basis. (No other tool came close... want to know what was #2? You'll have to keep reading 😉)
3. How are programmers using AI tools? 80.5% of programmers are using these tools as a search engine for new topics, while 58.5% are actually using them to help write code
4. How much of a developers job can AI tools impact? 46.4% of Front-End Developers are using AI tools to help with 30% percent (or more) of their job
5. Are programmers who haven't tried AI tools planning on it? 51.4% of programmers who have never used an AI tool plan to start using them in the next 6 months
6. What's stopping them? 36.6% stated that the biggest thing preventing them from using AI tools is simply the learning curve and figuring out how to use them effectively. There were a few other reasons but this was the biggest holdup. 14.6% were also concerned with the time required to learn and 13.4% were worried about the accuracy of these tools
7. New skill requirement for jobs?! 10.7% of programmers who had applied for new jobs in the last 12 months stated that experience with ChatGPT or other AI tools was listed as part of their job application requirements. But the writing is on the wall... 80.1% of programmers think AI tools will become a standard job requirement
8. Are employers expecting developers to use AI tools? 30.8% of programmers surveyed said that their current employer was expecting them to learn and use AI tools. However, only 27.9% of team leads surveyed were encouraging team members to learn them (and not making it a requirement)
9. Will AI tools improve code quality? Interestingly, 77.8% of programmers feel that these tools will have a positive impact on code quality
10. Will AI tools replace programmers? Not according to the programmers themselves. Only 13.4% think that it may replace them in their role in the next 5 years. The majority aren't worried and the overall consensus seems to be that their job will evolve alongside AI tools

Note: Keep reading for more details and additional stats as well as deep dives into how demographics and other factors affect trends in adoption, usage, and opinion.

But first we'll run you through our methodology. If you just want the stats and the takeaways, feel free to scroll by.

Methodology

Our goal with this survey was to start broad and see which AI tools (if any) programmers are using, how they’re using them, their thoughts on the future, and their effect on programming.

Rather than look into every single potential AI tool, we wanted to look at pair programming tools (or tools that might not specifically be for this, but that can achieve it).

The reason for this filter was so we could look at the types of tools that could help with a large portion of a programmer's day vs. a niche tool used for one singular task.

Why?

Because we’ve had AI tools for a while now that can do one or two things fairly well, but there’s been a huge push lately in these new tools that could actually code for us - if we learn to use them properly.

And this is where the ‘what if’s exist.

• Will this take my job?
• Will it make my job easier?
• Etc

Then we dove deeper to see if there were general trends that appeared across education level, company size, salary, role, and other metrics.

Things like:

• Are there certain countries that are adopting these tools quicker?
• What about specific roles?
• Are smaller or larger companies adopting these tools faster?
• Are companies expecting you to learn these tools, and is there a specific trend across company size?
• Do years of experience in a role affect the adoption rate of AI tools?
• How about the percentage of work that these programmers are using these tools for? And if so, by how much, and again, is that affected by experience?
• With regards to education, are self-taught programmers leaning into these tools more, or are they steering away? What about graduate students?

Alright, enough talk, let's dive into the data.

We’ve broken this down into 5 major sections

• Part 1: Demographics - so you can understand who the audience is
• Part 2: AI Tool Usage Data - so you can see broad stats as well as dives into how those demographics tie in and affect trends
• Part 3: AI Tools in the Workplace - some basic questions around current work environments and job applications with regards to AI tools
• Part 4: Management Insights - so you can get an idea of managers opinions and plan of action with these tools, and
• Part 5: Programmer Opinions - so you can see how both employees and team leads feel about these tools, job security, the future of coding, and more

Sounds good? Alright, let's get into it!

Note: Click on any graph below to open a high-res version.

Part 1: Demographics

Role breakdown

Although there were over 20+ different roles that filled out the survey, the majority of responses came from:

1. Full-Stack Devs (32.5%)
2. Front-End Devs (22.5%), and
3. Back-End Devs (8.9%)

Years of experience

There was a pretty balanced mix of responses from new developers as well as more experienced programmers. This is great as it allows us to see how adoption and opinion differ across experience levels... more on this later.

• ~57% of respondents have <3 years of experience
• ~43% of respondents have >3 years of experience

Geographic locations

The majority of responses came from the US, Europe, India, and Asia.

Our goal in finding out this information was to see if there were any particular trends across regions.

Are there specific locations that have been slower or faster to adopt these AI tools than others?

Spoiler alert: One region is adopting far faster than others and it's probably not where you might be thinking.

Age

We also wanted to see if there were any variations in adoption rates and/or opinions based on age.

• Are the young whipper snappers embracing these new tools faster?
• Or perhaps older and more experienced programmers have a better sense of how to best use these tools to make their lives easier (or maybe they just want to do anything they can to outsource some of their work 😅)?

Special shoutout to the 2.2% of 60+ programmers out there. Nothing better than being a lifelong learner!

These people are evidence that you're never too old to learn to code.

Company size

We thought it could be interesting to see if there were any particular trends or differences based on the size of company that these programmers work at.

Would larger companies veer away from these tools, or would they be the ones to lead the charge?

Highest level of formal education

Perhaps education ties into the adoption rate?

Would self-taught programmers be more agile and likely to adopt AI tools or more apprehensive to use these tools?

Current annual salary (USD)

Finally, we also asked for a salary range. This way we could see if there were patterns among higher or lower earners for adoption, job security, etc.

So, with that out of the way, let’s start looking at more specific answers, and how they connect with demographics.

Part 2: AI Tool Usage Data

84.4% of programmers have at least tried using an AI tool

While most programmers at this point are at least trying out AI tools and experimenting with them, there are still 15.6% of programmers that have never tried any.

With this in mind, we wanted to find out what why they hadn't and if they planned to in the future.

18.2% of programmers who have not tried AI tools, have no plans to use them in the future (only ~2.8% of all respondents)

However, 51.4% of those who hadn’t tried any tools yet, planned to use them over the next 6 months.

This is incredibly interesting, because although that 18.2% number seems high, when you look at that alongside planned future usage, it means that only ~3% of programmers that haven't already used an AI tool, have no plans or interest in trying them.

Or, even more impressive, by the end of 2023, ~97% of programmers will be either using AI tools, or at least tried them.

Clearly these tools are useful and people want to use them, but something is holding them back. So we figured the best way to find out, was to ask them what the issue was!

Biggest reason for not using an AI tool yet: 36.6% of programmers who have never used AI tools before are hesitant due to the learning curve

This makes sense. If you’re busy and stuck in the grind, it can be hard to slow down further just to find ways to speed up - even if it makes more sense long term.

It wasn’t just the effort to learn though. Another 13.4% were worried about the accuracy of the tools, which is a fair assessment.

It's been pretty well documented at this point that "hallucinations" can often happen, and unless you know the core topic and how to use the tools - you might be getting inaccurate information.

There were also 14.6% that stated it was the time to implement these tools as the reason for not using them yet.

We wonder what that really means...

Maybe because you could end up adding more time to your current tasks that you're already efficient at? Or perhaps this is people at bigger companies that have restrictions and hoops that need to be jumped through before employees are allowed to use these tools.

These are definitely some valid concerns, but as we've seen, the majority of programmers have been able to get past these issues and are finding ways to use these tools.

But what are they using them to do? And exactly how much are they using them? Which tools? And how effective are they?

Well, let's find out...

ChatGPT is still King (for now): 74.9% of coders are using it on a weekly basis (no other tool came close)

We provided a list of 12 AI tools that can be used for programming related tasks and asked people to let us know how often (if ever) they had used them.

This was the list of options:

• ChatGPT 3.5
• ChatGPT 4 (Paid version)
• Bard by Google
• MidJourney
• GitHub CoPilot
• Stable Diffusion
• Tabnine
• Amazon Codewhisperer
• Guidance by Microsoft
• Polycoder
• CodeT5, and
• Visual Studio Intellicode

The most popular tool by far was ChatGPT 3.5, with 58.4% of the respondents using it at least once a week (30.8% of those are using it daily).

Combined with the weekly usage of ChatGPT 4 (an additional 16.5%), that means that 74.9% of developers reported using some version of ChatGPT on a weekly basis 🤯

After ChatGPT 3.5, here were the next most used tools on a daily basis:

• Visual Studio Intellicode with 17.5% of the surveyed programmers using it daily
• GitHub CoPilot at 11.2%
• And ChatGPT 4 (paid version) at 10.6%

Why would there be such a huge difference in usage rates?

Right now, it probably comes down to breadth of usability, alongside frequency of tasks.

Or in simple terms, Visual Studio Intellicode and GitHub CoPilot are coding focused, so you would only use them for specific coding tasks. Whereas a tool like ChatGPT can be used for a much broader range of tasks - hence why it might be used more often. (More on this in a second).

A quick note on our data

Since ChatGPT 3.5 was used by far more of the survey audience than any other AI tool, we’ll be using it as our "proxy" going forward.

For example, in a coming section, if we say "Machine Learning Engineers are using AI tools more than any other role" (true story) then unless we specify otherwise, AI tools = ChatGPT 3.5 usage.

So let’s take a look at what we found out.

56.2% of Machine Learning Engineers use ChatGPT 3.5 on a daily basis

We wanted to know which roles were using these tools the most often.

To figure this out, we decided to cross-reference the 6 roles that gave the most responses, (Full-stack, Front-End, Back-End, Machine Learning Engineers, Designers, and Data Scientists), against their usage of the most popular tool (ChatGPT 3.5).

The results:

• Machine Learning Engineers came out on top, with a whopping 56.2% of them using it every single day!
• 45.1% of Data Scientists are using ChatGPT daily with work, and a further 30.5% using it at least once a week
• 37.2% of Front-End Developers are using ChatGPT daily, and a further 35.6% using it weekly
• 35.9% of Web Designers use it daily, and a further 35% using it weekly
• 35.4% of Back-End Developers use it daily, and
• 34.7% of Full-Stack Developers use it daily

We'd think Machine Learning Engineers might use AI tools in general more in their job than other roles but it’s interesting to see how many of them are specifically using ChatGPT.

Across actual primary programming roles (front-end, back-end, full-stack), there wasn't a big difference in usage.

What country or region has the highest adoption rate of AI tools?

Programmers in Africa had the highest daily adoption rate of AI tools and programmers in Asia had the highest weekly adoption rate of AI tools.

The lowest adoption rate of all regions (daily and weekly)...? The United States!

We wanted to see if there was a difference in adoption rate of these tools based on a number of demographic factors.

The first one we looked at was the geographic location of the programmer against weekly usage, and then daily usage on its own, to see if there were any trends.

The weekly results are in the chart above with Asia edging out Africa.

Daily Results:

• The highest adoption rate for daily use was in Africa, with 48.6% of programmers in this location using ChatGPT daily
• Next was Asia, with 48.2% of programmers using ChatGPT daily, although Asia was the leader in combined daily + weekly usage
• 34.3% in South America
• 32.7% in Europe
• 31.7% in the UK
• 31.0% in Canada
• 27.3% in the... United States (we wouldn't have guessed that one!)

Although we did get responses from a ton of other countries and regions, we only included the ones that had more than 100 respondents.

The younger you are = the more likely you're using AI tools for coding

Young Programmers (18-25 years old) are the biggest adopters of using AI tools for coding. 46.3% are using ChatGPT for work daily and an additional 29% on a weekly basis.

What about other age groups?

The results (using AI tools daily for work):

• 46.3% of 18-25 year olds
• 30.5% of 26-35 year olds
• 21.7% of 36-45 year olds
• 19.5% of 46-55 year olds
• 8.7% of 60+ year olds are using it daily (and 56.5% having never tried it)

Side note

Programmers who are both working and under 18 technically had the highest adoption rate (53.1% daily use), but this audience size was less than 50 people so we excluded it from these results.

(Being under 18 and working in tech is incredibly rare, so this was too small a dataset to consider accurately, but it's awesome to see even 50 people who’ve achieved this).

We're not really shocked by this result as you would expect that younger people in general would be more open to trying new things (I'm still figuring out TikTok but I'm not telling you how old I am).

However, it was a bit surprising to see a direct correlation between age and usage of AI tools for coding work.

Maybe these younger programmers are more worried about proving themselves? Or maybe they see their peers using these tools and don't want to get left behind?

Programmers with 3-5 years of work experience are using ChatGPT the most often

We wanted to see if there were any trends between years of experience in a role, and adoption of AI tools (again, using ChatGPT as our proxy).

We looked at the users for both daily + weekly combined (for a broad picture), and then those who it daily to see if there were any specific trends.

The results (daily usage for work purposes):

• Interestingly, programmers with 3-5 years experience had the highest adoption rate of both daily (43.8%) and weekly usage (an additional 31.7%)
• Programmers with 1-2 years experience: 35.1% using ChatGPT daily for work
• Programmers with 0-1 years experience: 33.8% use it daily
• Programmers with 5-10 years experience: 23.3%
• Programmers with 10+ years of experience: 19.9%

We were actually surprised to see such a significant percentage of experienced programmers using ChatGPT regularly.

19.9% of programmers with 10+ years experience, are using ChatGPT daily

While the lowest adoption is amongst programmers with 10+ years experience, almost 20% of them using it daily is extremely impressive given we would expect these people to be most resistant to change and new tools.

It'll be really interesting to see if the trend continues or if usage fades away for any of these groups when we do the 2024 edition.

Education level is not strongly correlated with using AI tools at work

We wanted to see if there was a difference in adoption rate and usage of this tool at work, based on the highest level of education someone has achieved.

Again, we looked at the combined daily + weekly users for a broad picture, and then the daily users only.

When we look at daily usage, there isn't a huge difference to report here.

Highest adopters: Programmers with a Bachelor’s Degree are the highest adopters of AI tools for work (31.2% using it daily).

Lowest adopters: Funny enough, people with a Doctorate Degree (highest education) are using AI tools for work at the same rate as people with no formal degree (least education).

The results (daily usage only)

• Bachelor’s Degree programmers had the highest daily usage at 32.1%
• 30.1% of Graduate Degree programmers used it daily
• 29.6% of programmers with a High School Degree used it daily
• 25.8% of programmers with Doctorate graduates used it daily
• 25.5% of programmers with no formal degree, used the tool daily

Higher salary = less daily usage of AI tools (no correlation between salary and weekly usage)

We compared salary ranges of programmers against their daily and weekly usage of AI tools.

The weekly results are in the chart above.

The daily usage results:

• Less than $50k reported 33.3% daily usage
• $50-$60k have 29.7% daily usage
• $60-$70k have 26.1%
• $70-$80k have 24.8%
• $90-$100k have 24.7%
• $100-$150k have 24.7%
• $150-$200k have 22.3%

Interestingly, there seems to be a pretty clear inverse correlation between salary and daily AI tool (ChatGPT) usage.

The highest daily adoption rate was among the lowest salary programmers and adoption rate dropped off as salary increased.

But when we zoomed out to weekly usage, there wasn't any correlation at all. Although the lowest salary programmers still have the highest weekly usage, the highest salary programmers have the second highest weekly usage (compared to lowest for daily usage).

Here's a deeper breakdown for the <$50K segment (daily + weekly in the chart below = total weekly):

Perhaps this is a case of when someone's salary increases, the role they are doing may contain less tasks that are more easily replaced by these AI tools (ex: repetitive tasks)?

That being said, while there is a dropoff rate at each higher salary range, it really isn't that big (ex: roughly 2 in 10 people making between $150-$200k vs. 3 in 10 people making less than $50k).

Programmers at SMBs are using ChatGPT the most

We wanted to see if there were any trends among adoption rate of AI tools and company size. Again, we looked at combined daily + weekly (see graph above) vs daily usage on its own.

Our initial theory was that solopreneurs might be the biggest adopters, simply so they could improve their output.

1 person + an AI tool = the biggest opportunity to get more done right?

However, the largest adoption was in those scrappy SMB (Small Medium Business) and start-up sized companies, with 44% of programmers that work in 11-20 employee sized companies using ChatGPT daily for some portion of their work.

In fact, the top 5 groups using ChatGPT daily were all those working at companies of less than 100 employees.

The results (daily usage)

• 44% of programmers at 11-20 employee companies use ChatGPT 3.5 daily
• 36.1% of programmers at 2-5 employee companies use it daily
• 34.2% of programmers at 6-10 employees companies use it daily
• 31.6% at 51-100 employee companies
• 30.4% at 21-50 employee companies
• Solopreneurs (1 employee companies) came in at 29.6% daily use, which is about average so no real outlier here. Our hypothesis was incorrect, possibly due to the ‘time to learn + implement’ factors we saw earlier
• 27.6% of programmers at 1,000+ employee companies use it daily
• 27.1% of programmers at 101-500 employee companies use it daily
• 25.2% of programmers at 501-1,000 employee companies use it daily

Programmers at 501-1,000 employee companies have the highest weekly usage of ChatGPT

One interesting note though is that although 501-1,000 employee companies had a lower daily usage, they had the highest weekly usage!

In fact, 30.1% of programmers at these companies are using it at least once a week.

No wonder Google is freaking out right now... What are developers actually using AI tools for? A whopping 80.5% are using ChatGPT as a quasi-search engine replacement

We asked our survey audience to list out each task that they use AI tools for at work, and the overwhelming majority are using AI tools as a quasi-search engine (learning new topics and writing code), which makes sense.

On one hand, programming is problem solving.

A lot of times other developers have had the same problem or a similar one, so programmers spend a lot of time searching on Google and sites like StackOverflow for potential solutions to their problems (many times there are code snippets that people can re-use or slightly modify for their situation).

Given the fact that you can deeply customize the results that get sent back from ChatGPT (and other AI tools), as well as it being able to literally write snippets of code for you that fit those questions, and it makes a very compelling use case.

What like?

Standard tasks like writing SQL queries or regex are great examples of things that a developer might previously have had to Google to remember the correct syntax for. But now, ChatGPT can easily write these queries for you.

On another hand, programming and tech in general also changes so fast. Developers have to be constantly learning to stay up-to-date and almost always have to learn something new for every project they work on.

So instead of searching on Google or Youtube, it seems like people are getting ChatGPT to provide them with tutorials and "how-to" type content on these new things they're trying to learn.

Editor's Note: There are well documented risks and downsides to relying blindly on ChatGPT or other AI tools though so make sure to be aware of those.

But from our experience, if you know what to look for and how to properly use them, these AI tools are super useful.

Here's a little beginner's guide to using ChatGPT for coding (and the pitfalls to watch out for).

As a final point on this, it’s also not surprising that people are using this for search, because it's the lowest entry point to using the tool.

Anyone that can enter a question into Google can just as easily enter a question into ChatGPT, so it’s expected to be the most common use case (at least in these early days).

What was interesting to note was that survey respondents who were using it to write code, were almost always using it to help with other tasks as well, such as debugging, testing, writing documentation, pair-programming, and more.

Once you prompt, you just can't stop!

So, are these tools so helpful that some programmers are using them for a majority of their work? Or is it a case of them having success with ChatGPT to solve one problem, and now they're treating it like a hammer and thinking everything is a nail?

We might need to do some follow-up here...

Work smarter, not harder: 52.9% of survey respondents are using AI tools to help with at least 30% of their work

We also asked our participants to share the volume of their work that they estimate they use AI tools to assist with.

According to our results, 70.5% of them are already using AI tools to help with at least 20% of their work.

There's a lot of metrics in this report so its easy to skim past them, but stop and think for a second, because this is more shocking than it first appears.

If you consider the fact that 20% of someone's work week would roughly equate to one total work day, that's a big difference in quality of life.

That's an entire days work (every week) that's semi-automated, outsourced, or simplified in some way!

Even more impressive?

20.9% of these people are using these tools to help with 50%+ of their work.

What does this mean? Are these people more likely to be replaced over time? Or are these the people that are going to have the first mover advantage and get themselves a leg up by working way more efficiently than their peers?

Either way, we have to confess to a mild level of jealousy to these people who are able to get assistance on 90% of their workload... we might have to reach out to these people for a follow-up article!

The results

• 29.5% use AI tools to help with 10% of their work
• 17.6% use AI tools to help with 20% of their work
• 20.0% use AI tools to help with 30% of their work
• 11.9% use AI tools to help with 40% of their work
• 8.7% use AI tools to help with 50% of their work
• 5.1% use AI tools to help with 60% of their work
• 3.9% use AI tools to help with 70% of their work
• 1.7% use AI tools to help with 80% of their work
• While 1.5% of them use AI tools to help with 90% of their work 🤯

Impressive right?

Let's break it down by role and see how that compares.

19.2% of Front-End Developers are using AI tools to help with 30% percent of their job

Because we have data on specific job roles and their estimated volume of work being assisted by AI tools, we wanted to see which roles are really pushing the envelope and using AI tools the most.

And so we decided to compare roles based on who is using these tools to help with 30% of their work.

Why 30%?

Well, using AI tools to help with 10% of your work is a good start but it doesn't feel like a game changer.

30% seems like a tipping point where a company could make a serious decision to not have to make an additional hire (ex: if their existing teams were 30%+ more efficient).

The results

• 19.2% of Front-End devs are using AI tools to help with 30% of their work
• 17.5% of both Back-End and Full-Stack devs are using AI tools to help with 30% of their work
• 16.9% of Machine Learning Engineers are using AI tools to help with 30% of their work
• 16.8% of Data Scientists are using these tools to help with 30% of their work
• And finally, 15.1% of Designers are using them to help with 30% of their work

Although front-end devs are the highest, it seems like there are many people across all these roles that are already finding a way to leverage AI tools for a big chunk of their work.

It's interesting that so many people feel so confident that their job isn't at risk but also that so many people are finding ways to essentially replace themselves.

Maybe companies don't realize how much work their employees are able to do with these tools?

Or maybe companies would happily take a productivity bump because these workers are now more efficient.

We have so many more questions!

We may have to follow up with some of these respondents to learn more about their processes, how they got past the learning curve, and what the exact tasks they are replacing that make up the 30% (or more) they are saying.

Is this expected or crazy? 67.6% of Machine Learning Engineers are using AI tools to help with AT LEAST 30% of their work (and higher!)

Here's where it gets really interesting.

Rather than just stay at the 30% mark, we decided to look deeper to see if there were any trends in specific roles that are pushing even higher.

Was there a particular role that stood out?

It turns out, there was.

The results

• 67.6% of Machine Learning Engineers are using AI tools for AT LEAST 30% of their work or higher
• 49.1% of Data Scientists are using AI tools for at least 30% of their work
• 46.4% of Front-End Developers are using it for at least 30% of their work
• 45.8% of Full-Stack Developers use it for at least 30%
• 43.7% of Web Designers
• 41.7% of Back-End Developers

This makes sense, after all, Machine Learning is all about working with AI.

However, it's really interesting when you look at how high the percentages are for each other role on this list. Most of them never need to use an AI tool, and yet over 46% of Front-End Developers are using AI tools to help with at least 30% of their work (some are even using it to help with 90% of their workload!).

Part 3: AI Tools in the Workplace

Over 30% of employers are expecting programmers to learn or use AI tools

We're honestly surprised this isn’t a higher number.

Why?

Well, we’re kind of spoiled in tech in that these tools can help assist in far more areas of our day to day job, in an actual applicable way than in other industries.

That said, I imagine this number will rise each year from here on out.

Are AI tools going to become a job requirement?

10.7% of programmers who applied for a job in the last 12 months, stated that the job listing had ‘having experience using ChatGPT (or any other AI tool)’ as a requirement.

To put that into context, of the 2,115 people in our survey who applied for a new job in the last 12 months, 227 of them reported job listings requesting some kind of AI tool experience.

Will this rise over the next few years?

Honestly, we think it’s all going to come down to people embracing the learning curve and taking the time to apply these tools. Once that becomes standard, then we have to imagine that we’ll see a huge lift.

Our bet is that learning these tools is going to be important, whether explicitly stated as a job requirement or not.

Part 4: Management Insights

OK, so what do managers and team leads think about using these AI tools?

21.6% of our survey respondents manage a team

That’s almost 700 managers providing insights here!

This allowed us to tailor a few questions to see what the thoughts and feelings are from those who have to wrangle the rest of us.

• Do managers care about their team members using AI tools?
• Do they think these AI tools are the future?
• Are they pushing their teams to learn them?

Let’s see what the data has to say.

45.6% of team leads are worried or have concerns about the use of AI tools by their team members

Obviously this is a very exciting time, however team leads are still cautious. When we look at their concerns though, it does make sense:

• How long will it take to learn?
• Will there be hallucinations in the outputs?
• And will the team member take the time to double check what the AI tool gives them, to make sure it's actually correct?

Nothing too drastic and things that can be worked through. Heck, having to check code snippets from 3rd party sources before you implement them should be standard practice anyways right?

But still, these concerns and factors seem to be why there isn't yet a huge push by managers to require employees to learn these tools yet…

27.9% of team leads are either requiring or encouraging their team members to use AI tools

Only about a quarter of managers are requiring employees to use AI tools at this point, which makes sense given the responses we received to the previous question.

There are still a few hang ups to getting up to speed (like any new tool or product) and removing errors with these tools.

The big thing to watch over the next year is whether these concerns are adequately resolved by the various tools out there.

If they are, usage will rise. If not, maybe these tools aren't going to deliver on the productivity improvements that many people are hyping them up to be capable of.

It might be more likely that existing products employees are already using (and are comfortable using) will add in additional AI capabilities (ex: Github CoPilot) rather than relying on a single AI tool for all tasks (ex: ChatGPT).

Team leads at 1,000+ employee companies are the biggest proponents of employees learning to use AI tools

Although only 27.9% of team leads are encouraging their employees to learn these tools, if we break it down by company size then it's the team leads at 1,000+ employee companies who are the biggest proponents.

We're not sure exactly what to make of this...

On one hand it makes sense right?

1,000+ employee companies are possibly more FAANG level and would expect their team members to be the most efficient coders they can be. Add in the fact that a 1% improvement in output across 1,000+ employees would be a huge impact.

Heck… imagine if they could get to the point where their team members are able to help assist 10-30% of their roles with AI tools, and how that would affect their bottom line!

On the other hand, it's a bit surprising because bigger companies tend to be slower to adopt new tools and technologies given all the red tape that exists.

Part 5: Programmer Opinions

Alright, the final section. Everyone loves to have an opinion right?

Let’s look at some general opinions around these tools and the future of them, from both an employee and team leader perspective.

• Do programmers believe AI tools are the future?
• Are they worried that their jobs are at risk?
• Do these opinions differ from employee to team lead?

Let’s find out.

Programmers rate the current helpfulness of AI tools at 7.4/10

This is pretty much the same sentiment across both employees and team leads.

The large majority of programmers are finding these tools to be at least a 7/10 or higher in terms of helpfulness, which is pretty damn good.

There is still room for improvement which shouldn't come as a surprise given we're still in the early days but given this rating, we're surprised managers aren't encouraging usage of these tools even more.

It's fascinating to see the actual usage data that people report vs. their subjective opinions.

Most people seem to find them quite helpful already so are those also the people using them weekly or monthly? Or do people just have a positive view of them but that isn't actually translating into regular usage?

We went through our data to see if there was any correlation:

As you might have guessed, the more people used the tools, the higher they scored it in terms of helpfulness.

The question becomes though... is this a chicken and egg situation?

• Are they finding it more helpful because they are using it more, have gotten past the initial learning curve and have figured out how to make it useful?
• Or are they reporting that they find it more helpful just because they are using it a lot... no one wants to basically admit they're wasting their own time (ie: using it a lot but reporting that it's actually not that good of a use of their time)

Because many of these tools are still so new, we feel it's the simple fact that people are still learning how to use them.

The more they actually use them, the more they see how other people are benefiting from them and the more the tools themselves continue to improve, the more helpful they will be for programmers.

But until they learn to use them effectively, it could seem like yet another thing to have to learn and figure out.

There's no stopping them: 80.1% of programmers think AI tools will become a standard job requirement

Even if managers and team leads aren't mandating usage of these tools yet, most programmers believe the writing is on the wall.

It makes sense that people would feel this way. Once developers learn to use these tools effectively, their productivity should greatly increase (as we've already seen from some of the respondents).

It's not just the beginners either! 78.6% of programmers with 10+ years of experience think that AI tools will become a standard requirement in the future

Rather than just looking at the broader audience, we decided to further break down this opinion by years of experience.

Overall, the consensus was quite clear across the board, with ~70-80% of programmers at all levels of experience believing that these tools will become standard requirements in the future.

The results

• 83.4% or programmers with 0-1 years of experience think this will be standard in the future
• 80.2% of those with 1-2 years of experience
• 78.6% with those with 10+ years experience
• 77.7% with those with 3-5 years experience, and
• 74.8% of those with 5-10 years experience

What was really interesting to see though was how this was also the opinion of programmers with the most experience in their roles.

Why is this important?

Well, these people have been around a long time and have seen a lot of trends come and go (with it being quite rare that something survives and sticks around).

So if they think it's here to stay, then that's worth paying attention to.

82.3% of team leads also think these AI tools will become a standard requirement in the future

So, only slightly higher than the employee consensus of 80.1%.

However, this is another interesting situation where the actions of team leads today (only ~28% requiring their teams to learn these tools) is drastically different from where they believe things are going.

They're not forcing people to use them yet, but they definitely think people will need to use them in the future.

Add in the fact that these team leads are the people who will have a strong say in deciding whether these skills will be required or not (whether from new coders or existing team members upskilling), and this gives a more accurate representation of what will happen down the line.

If you're not using these tools yet, it seems like you'll need to learn them in the coming years.

77.8% of programmers think these tools will have a positive impact on code quality

We talked about this before in our hard truths about being a software engineer post, but a lot of the time, it can feel like you're rushed to get things finished in coding.

This means less time for testing, documentation, etc, and so you might cut corners a little or miss tasks that you would like to get finished.

And let's be honest, no one really enjoys testing, QA, and documentation so they tend to be the things that are ignored.

How could AI tools change this?

Forget about the potential of AI tools helping write new code. AI tools can already help you test your code, spot bugs, suggest fixes and even write documentation for you.

Even if programmers just learned how to properly use these AI tools to help with some of these mundane and admin tasks, that could result in a huge increase in the quality of code out there and increase efficiency across the board.

We are total believers here, and even have a few new courses in the works that will show you exactly how to leverage ChatGPT and other AI tools to make your life as a programmer so much easier

Make sure to subscribe at the bottom of the post if you want to find out when we release these new courses.

In the meantime, get up to speed on the fundamentals from our ChatGPT / large language model intro course.

79% of team leads also think AI tools will have a positive impact on code quality

Only a slightly larger % than the general audience.

This probably ties into why team leads think that these tools will become standard but it's still strange then as to why such a smaller percentage of these same team leads are actually requiring or encouraging their team members to learn them.

What will be the tipping point?

We have to imagine it will be once someone creates a cohesive training program of how to use these tools effectively perhaps?

This would reduce those major barriers that we saw earlier (time and effort to learn), while also giving systems and processes for how to use it for actual ongoing tasks vs random ideas, as it might be being used for right now.

Developers with 0-1 year of experience are the most fearful of losing their job due to AI tools (34.1%)

This makes sense. If you have a new job and are at entry-level, you might be worried about a whole bunch of things, let alone a tool that claims to be able to do your job for you!

Honestly though, our view is that this is more likely imposter syndrome and being new in a role, coupled with compounding fears of the heavily publicized tech layoffs that happened around the same time as these tools picked up speed.

You can see this mirrored in the fact that as experience level grows, confidence in job security rises.

To be honest, we don’t think these tools are at a level yet where they could replace a human in any role, other than basic $5 tasks.

The real threat comes from peers who have embraced them and become that ‘10x level’ developer, while others get left behind.

Will AI tools replace programmers? NO! Only 13.4% of the developers think ChatGPT or other AI tools might replace them in their job within the next 5 years

However, the overwhelming majority (76.9%) feel that this won’t be the case.

Is it possible that the majority could be wrong? Of course. But throughout human history there have always been new tools that have changed work as we know it. The key is adapting and embracing these new tools, and how they effect change in their industry.

Sure it might shake up other industries, but given that programmers are already constantly required to be keeping their skills up-to-date, there's no reason why they won't adapt to using these tools either.

tl;dr: Roles that don't require much ongoing learning and change might have a tougher time adapting, but if you're a lifelong learner and embrace learning new things... you'll be fine.

And when we look at many of the comments from these thousands of programmers, almost all of them feel the same way... that their job will simply evolve to incorporate the usage of AI tools.

In fact, we feel so strongly about this topic that the founder of ZTM (Andrei Neagoie even just dropped a video on how to actually thrive with AI:

Conclusion

So there you have it. Hopefully, this survey has shed some light on the future of programming and AI tools and if you're a programmer yourself, perhaps alleviates some worries you might have.

Heck, maybe it's got you excited for what's to come and made you want to start learning and applying this now! Just imagine how much easier your job could be if you could get AI assistance for 10-30% of it or more.

As a final note, we just want to give a huge thank you to everyone who took the time to fill out this survey. We're already excited to do it again next year so we can start looking at the trends in 2024 as well.

Our quick unsolicited soapbox opinion on AI tools:

Since we know some people will ask what our thoughts are, we added this in. Maybe we'll do a longer post on this in the coming weeks.

Here's the 30-second version...

AI tools will be like any other tool. If used properly, they'll simply help produce a better product or make a programmer more efficient. But it won't replace them (at least no tools we've seen so far to date).

We're also likely to see a huge uptick in job listings including experience with AI tools. Programmers still need to be properly trained to understand the output these tools are providing them and understand how that fits into a larger code base.

Programming fundamentals + real-world experience + institutional knowledge (ex: specifically how things work at your company or the nuances of your industry) aren't being replaced. They are all still key.

Depending on who you ask or the stats you look at, we actually have the opposite problem of developers being replaced and there's actually a shortage of developers.

Agree? Disagree? Let us know.

Found this survey and the data helpful?

Share it with someone! We purposefully made it free and no email gating so anyone can access it anywhere, anytime.

Due to popular demand from the ZTM community and the programming community at large, we've decided to open-source the data so that anyone can do their own data analysis as well.

Access the full ZTM State of AI 2023 data set here.

Feel free to reference it and use the charts or data (we just appreciate if you cite us 🙂). And if you come up with any interesting findings that we missed, definitely let us know!