Peeling Back the Curtain to Unmask the Wizard of AI: Considering the Collaborative Relationship Between Non-Technical Subject Matter Experts and Artificial Intelligence.

By: Gino Galli

Introduction: The Great & Powerful in L. Frank Baum's seminal 1900 classic "The Wizard of Oz", Toto, Dorothy's scrappy little dog, peels back the curtain to reveal the truth behind the 'great and powerful Oz.' To everyone’s surprise, it was a mere mortal that was behind the illusion. Much like Toto's insightful revelation, this paper endeavors to illuminate the role of non-technical professionals – an invisible cadre encompassing subject matter experts, domain experts, and ethicists. These experts, akin to Toto's quiet unveiling, wield influence behind the scenes, actively shaping the formidable engine of generative artificial intelligence (AI). The argument presented posits that this symbiotic 1 collaboration will usher in new career opportunities across diverse professions, ranging from political scientists and early childhood education specialists to health and fitness experts, as the demand for seasoned professionals in their respective fields intensifies.

While both the Wizard and the invisible cadre of experts represent unseen forces that the general public has little knowledge of, in sharp contrast to Baum’s Wizard character, the various types of non-technical experts behind AI bring actual value from multiple perspectives and are critical to AI’s development and ongoing success. Baum’s Wizard was proven to be a charlatan, but non-technical experts are the real deal. They regularly collaborate with AI and technical experts behind the scenes on everyday tasks such as content creation, language translation, fact and reference checking, and predictive modeling on a wide range of topics and domains.

So What? The 10,000 ft. View

So, why examine the relationship between non-technical experts and AI? The rapid evolution of generative AI technologies, in particular large language models such as Chat GPT, has triggered dialogue about its role in society. In particular, discussion about AI’s impact on the job market, such as how AI will integrate with humans in the workforce on a day-to-day basis, has garnered much media attention. Both the literature and researchers, such as Wang and Ringel, express opinions that run contrary to much of the media hype. Instead of predicting how AI will take over the workforce, these sources mention the need for non-technical subject matter expertise in navigating this transformative landscape.

The Bureau of Labor Statistics (BLS) which forecasts that jobs like data scientist are expected to grow by 35% until 2032, which is much faster than the average occupation.

2 How can a non-technical expert bring value to artificial intelligence? Non-technical experts bring value by collaborating with AI and computer specialists to grow and shape current and new iterations of AI by helping organizations develop valuable AI tools relevant to their respective fields. These experts may work on tasks such as evaluating the model’s response for accuracy and quality and systematically implementing improvements to the model. Other experts, such as an AI ethicist, will also work together with AI and technical specialists on projects such as improving algorithms to address social biases and ensuring inclusivity.

Is AI a threat to the job market? While concerns about job displacement persist, a closer examination reveals that AI's true potential lies in supplementing human expertise, not replacing it. As generative AI evolves, so should the role of the non-technical SME. This tandem evolution shapes the adoption and ethical application of artificial intelligence. The research and literature review endeavors to shed light on the notion that there are non-technical experts who can move the needle forward if given the opportunity to do so. By leveraging the power of domain expertise, non-technical experts like political scientists, urban planners, poets, and musicians can usher in a new era of generative AI that is tailored, efficient, and truly transformative to their respective fields.

3 3 Ravinutala, R. "The Power of Domain-Specific LLMS in Generative AI for Enterprises." Forbes, 5 Oct. 2023, 3 So, what's the takeaway? It is important to recognize that AI is going to increase the demand for non-technical subject matter experts in the long run for many industries. This, in turn, will foster job creation and contribute to the sustained success of generative AI.

Generative AI: Not a Hallucination

Artificial intelligence is not anything new - as a technology, it has been in use since the advent of computers. In 1955, John McCarthy was credited for coining the term.4 Generative artificial intelligence, on the other hand, AI that can produce new content, became part of our social fabric with the launch of Open AI’s ChatGPT on November 30, 2022.5 By December 5th, ChatGPT had attracted over one million new users, and seven months later, by the summer of 2023, Open AI was valued at $29 billion.6 Generative artificial intelligence was an immediate success and within months competing generative AI platforms began to emerge such as Google’s Bard and Microsoft’s Copilot.

Specialized AI platforms have also emerged to cater to specific consumer and business requirements, exemplified by Open AI's DALL-E 2 for graphic applications, Tome AI and Gamma for presentations, and Midjourney and Play.ht for text-to-speech tasks. Numerous others are currently in the developmental stages. The current generation of AI Large Language Models (LLMs) exhibit remarkable abilities in producing textual content that spans a wide range of topics, however, they often lack the nuance and depth required for specific domains such as law, literature, and history. 7

Another ongoing challenge currently facing AI is hallucinations. These occur when the model fabricates incorrect information it believes is true.8 Sometimes it is fact-based, but as AI’s knowledge base continues to expand, fact-based errors will persist, despite constant efforts to ensure accuracy. Frequently, AI struggles to comprehend human emotion accurately and often can misinterpret human intention. An example from the research involves the insurance domain in which an SME would understand the nuance when clients refer to the process of modifying certain terms in their policies as "policy endorsement". However, that exact language may not be universally understood by the generic LLM and as a result, the customer is then left with a negative experience 9 The lack of human nuance, the hallucinations, and the struggle to discern human emotion represent examples of where a non-technical subject matter or domain expert can come in to help bridge the gap between AI and the intricate details of a particular domain or industry.

This article is to be continued.........

1 Wang, Weiguang, et al. "Knowledge Trap: Human Experts Distracted by Details When Teaming with AI." University of Miami Business School Research Paper No. 4395858, 21 Mar. 2023, SSRN: http://dx.doi.org/10.2139/ssrn.4395858, pg. 6., Ringel, Daniel. "Creating Synthetic Experts with Generative Artificial Intelligence." Kenan Institute of Private Enterprise Research Paper No. 4542949, 5 Dec. 2023, SSRN: https://ssrn.com/abstract=4542949 or http://dx.doi.org/10.2139/ssrn.4542949, pg.

2 The US Bureau of Labor Statistics does not have a labor classification for AI Subject Matter Experts, Domain Experts, or AI Ethicists. Data analyst/data scientist was chosen as a proxy due to its similarity in job description and because the Occupational Outlook Handbook classifies occupations based on tasks and skills, not expertise in specific fields. According to the BLS, the average occupation will grow by 3% from 2022 to 2032., but social scientists and related workers will grow by 5%. Employment of management analysts is projected to grow 10 percent, and employment of political scientists is projected to grow 7 percent from 2022 to 2032. / “Data Scientists : Occupational Outlook Handbook.” U.S. Bureau of Labor Statistics, U.S. Bureau of Labor Statistics, 6 Sept. 2023, www.bls.gov/ooh/math/data-scientists.htm

3 Ravinutala, R. "The Power of Domain-Specific LLMS in Generative AI for Enterprises." Forbes, 5 Oct. 2023, https://www.forbes.com/sites/forbestechcouncil/2023/07/20/the-power-of-domain-specific-llms-in-generative-ai-forenterprises/?sh=5159a82c1e50