Amid Mainstream Boom, AI Skeptics Say the Tech is Overhyped

Artificial intelligence tools like ChatGPT have “unequivocally gone mainstream” in 2023, according to a new survey of over 1,500 professionals working in technology and related fields. Yet, despite rapid adoption, most respondents still view AI as overhyped. Just under a quarter called it “fairly rated.”

The survey, conducted by Retool, found broad enthusiasm tempered by skepticism. Retool,  founded in June 2017 and part of the Y Combinator accelerator program, offers a platform for building internal business software using a visual development interface.

“In 2023, AI—and specifically generative AI—has unequivocally gone mainstream,” the report states. “It seems like everyone has played around with large language models (LLMs)—and, with upwards of 100 million weekly active users, ChatGPT has practically become a household name.”

Teachers, talk show hosts, and even grandparents now use AI chatbots for more than laughs, the study found. But real-world applications only “scratch the surface” of AI’s potential.

More than half of respondents (51.6%) still rated AI as over-hyped, versus just 23.4% calling it fairly rated. Upper management and executives showed the most favorable views. Software engineers closest to implementation work leaned more skeptical.

AI transforming jobs and industries

Respondents agreed AI will substantially transform their industries and jobs within five years. On a 0 to 10 scale of impact, the average score exceeded 7. Those in operations roles expected the highest level of change, followed by product managers and engineers.

Expectations centered on four key areas: efficiency gains, less boring work, new skills to learn and updated design processes. But respondents also want assurances AI will develop safely.

“They also had a big, important ask: Investment in AI governance and ethics to ensure that the technology’s future goes in a positive direction,” the report urges.

Dependence on Stack Overflow drops

Many engineers now rely less on the popular Stack Overflow programming site for coding help. Close to 60% of survey takers use Stack Overflow less since 2022. Of those, 1 in 10 have stopped using it altogether.

The overwhelming reason cited for the fall-off? GitHub Copilot and ChatGPT. Together they accounted for almost 94% of responses.

Launched in 2021, GitHub Copilot suggests code contextually as programmers type. The AI tool works as an automated pair programmer.

Are AI skills now required?

When rating their likelihood to hire candidates adept at using ChatGPT and GitHub Copilot for writing code, respondents averaged 6.7 on a 10-point scale. Small startups showed the greatest openness, while mid-size enterprises leaned more cautious.

But nearly 30% still described themselves as neutral on AI coding skills. Only rarely did respondents view them as a negative.

“While competency at using AI to write code had some degree of positive impact for a majority (63.2%), a sizable chunk of respondents (27.1%) were still pretty neutral,” the report said.

What’s behind corporate hunger for AI?

Survey-takers see cost savings and buzz as driving most businesses to pursue AI. Top motivations cited include cost reductions (38.7%), trendiness (35.2%), customer needs (32.9%) and competitive pressure (31.1%).

By contrast only 25.5% pointed to direct revenue impact as a key driver. The report suggests companies may view AI as a way to trim expenses over increasing earnings, at least for now.

AI is useful, but inaccurate

Those companies actively using AI tools rate them as largely useful, though not perfect. Problems with accuracy, data security and outright hallucinations top their list of complaints.

“No matter what you’re using them for, today’s AI tools are unlikely to be perfect,” the report wisely observes. “But understanding any technology’s strengths and shortcomings can help position you to use it well."

Nearly 40 percent of survey respondents called "model output accuracy" the key pain point in developing AI apps, with "hallucinations" coming in third at 28.1 percent. Data security nestled between them at 33.4%.

Will OpenAI dominance last?

For enterprises implementing AI, OpenAI’s natural language offerings dominate adoption over other vendors. Various flavors of ChatGPT make up respondents’ most used models by a wide margin.

But sticking to off-the-shelf offerings still rules over customization. Companies self-hosting open-source algorithms rather than using packaged services remain the exception for now.

Behind the numbers

The insights in Retool's report come from a public survey conducted in August of 1,578 respondents. Technology workers made up 39% of participants, followed by consulting and professional services at 12% and financial services at 10%.

Over a third classified themselves as engineers, over a fifth in operations roles, and 12% as product managers. Seventeen percent held a C-suite title role like CEO, 28% were mid-senior managers, and 20% had entry-level positions.

Companies sized between 1 and 99 workers accounted for 60% of the total, mid-size firms of 100 to 999 employees represented 26%, and large enterprises over 1,000 workers made up the remaining 14%.

Editor's note: This story was drafted with Decrypt AI from sources referenced in the text, and fact-checked by Ryan Ozawa.