{"id":281,"date":"2024-11-30T07:02:34","date_gmt":"2024-11-30T07:02:34","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2024\/11\/30\/why-internal-company-chatbots-fail-and-how-to-use-generative-ai-in-enterprise-with-impact-af06d24e011d\/"},"modified":"2024-11-30T07:02:34","modified_gmt":"2024-11-30T07:02:34","slug":"why-internal-company-chatbots-fail-and-how-to-use-generative-ai-in-enterprise-with-impact-af06d24e011d","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2024\/11\/30\/why-internal-company-chatbots-fail-and-how-to-use-generative-ai-in-enterprise-with-impact-af06d24e011d\/","title":{"rendered":"Why Internal Company Chatbots Fail and How to Use Generative AI in Enterprise with Impact"},"content":{"rendered":"

Why Internal Company Chatbots Fail and How to Use Generative AI in Enterprise with Impact
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Start with the problem and not with the\u00a0solution<\/h4>\n
\"\"
Background licensed from elements.envato.com, edit by Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

The most common disillusion that many organizations have is the following: They get excited about generative AI with ChatGPT or Microsoft Co-Pilot, read some article about how AI can \u201cmake your business better in some way,\u201d then try to find other use cases where they can slap a chatbot on and in the end are disappointed when the results are not super satisfying. And then, the justification phase comes. I often hear things like, \u201cThe model is not good enough\u201d or \u201cWe need to upskill the people to write better prompts.\u201d<\/p>\n

In 90% of the cases, these are not the correct conclusions and come from the issue that we think in Chatbots. I have developed over three dozen generative AI applications for organizations of three people to global enterprises with over three hundred thousand employees and I have seen this pattern everywhere.<\/p>\n

There are thousands of companies out there telling you that you need to have \u201csome kind of chatbot solution\u201d because everybody does that. OpenAI with ChatGPT, Microsoft Copilot, Google with Gemini and all the other companies selling you chatbots are doing a great job breaking down initial barriers to creating a chatbot. But let me tell you: 75% of the really painful problems you can solve with generative AI do not benefit from being a\u00a0chatbot<\/strong>.<\/p>\n

Too often, I see managers, program directors, or other decision-makers start with the idea: \u201c<\/em>We have here some product with AI that lets us build chatbots\u200a\u2014\u200alet\u2019s find as many places as possible to implement it.\u201d<\/em> In my experience, this is the wrong approach because you are starting from a solution and trying to fit an existing problem into it. What would be the correct way would be to look into a problem, analyze it and find an AI solution that fits. A chatbot may be a good interface for some use cases, but forcing every issue into a chatbot is problematic.<\/p>\n

\"\"
Forcing a solution onto a problem vs. starting with a problem and finding a solution. \u00a9 Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

In this article, I\u2019ll share insights and the method I\u2019ve developed through hands-on experience building countless applications. These applications, now live in production and serving thousands of users, have shaped my thinking about building impactful generative AI solutions\u200a\u2014\u200ainstead of blindly following a trend and feeling disappointed if it does not\u00a0work.<\/p>\n

Think about your Processes first\u200a\u2014\u200aChatbots (or other interfaces) second<\/strong><\/h3>\n

I tell you not to start your thinking from chatbots, so where should you start? The answer is simple: business processes<\/strong>.<\/p>\n

Everything that happens within a company is a business process. A business process is a combination of different activities (\u201cunits of work\u201d), events (for example, errors), and gateways (for example, decisions) connected into a workflow [1]. There are tools for modeling business processes [2] in well-known diagram forms and a whole research discipline centered around analyzing and improving business processes [3][4][5]. Business Process Management is a good tool because it is not theoretical but is used everywhere in companies\u200a\u2014\u200aeven though they do not know what to call\u00a0it.<\/p>\n

Let me give you an example. Imagine you are a company that does real estate valuations for a bank. Before banks give out mortgages, they ask real estate valuers to estimate how much the object is worth so that they know that in case the mortgage cannot be paid back, they have the actual\u00a0price.<\/p>\n

Creating a real estate valuation report is one large business process we can break down into subprocesses. Usually, valuers physically drive to the house, take pictures and then sit there writing a 20\u201330 page report describing their valuation. Let us, for a moment, not fall into the \u201cuh a 20\u201330 page report, let me sit in front of ChatGPT and I will probably be faster\u201d habit. Remember: processes first, then the solution.<\/p>\n

We can break this process down into smaller sub-processes like driving to the house, taking pictures and then writing the different parts of the report: location description of the house, describing the condition and sizes of the different rooms. When we look deeper into a single process, we will see the tasks, gateways, and events involved. For example, for writing the description of the location, a real estate valuer sits at their desk, does some research, looks on Google Maps what shops are around, and checks out the transport map of the city to determine how well the house is connected and how the street looks like. These are all activities (or tasks) that the case worker has to do. If the home is a single farm in the middle of nowhere, the public transport options are probably irrelevant because buyers of such houses usually are car dependent anyway. This decision on which path to go in a process is called a\u00a0gateway.<\/p>\n

\"\"
As is the process of the example modeled in BPMN 2.0. \u00a9 Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

This process-driven mindset we apply here starts with assessing the current process before throwing any AI on\u00a0it.<\/p>\n

Orchestration Instead of Chat-Based Interactions<\/strong><\/h3>\n

With this analysis of our processes and our goal we can now start looking into how a process with AI should look like. It is important to think about the individual steps that we need to take. If we only focus on the subprocess for creating the description that may look like\u00a0this:<\/p>\n

    \n
  • analyzing the locations and shops around the\u00a0house<\/li>\n
  • describing the condition of the\u00a0interior<\/li>\n
  • unless the location is very remote: finding the closest public transport stops<\/li>\n
  • writing a page of text for the\u00a0report<\/li>\n<\/ul>\n

    And yes, you can do that in an interactive way with a chatbot where you work with an \u201cAI sparring partner\u201d until you have your output. But this has in a company setting three major\u00a0issues:<\/p>\n

      \n
    1. \nReproducibility<\/strong>: Everybody prompts differently. This leads to different outputs depending on the skill and experience level of the prompting user. As a company, we want our output to be as reproducible as possible.<\/li>\n
    2. \nVarying quality<\/strong>: You probably have had interactions with ChatGPT where you needed to rephrase prompts multiple times until you had the quality that you wanted. And sometimes you get completely wrong answers. In this example, we have not found a single LLM that can describe the shops around in high quality without hallucinating.<\/li>\n
    3. \nData and existing systems integration<\/strong>: Every company has internal knowledge that they might want to use in those interactions. And yes, you can do some retrieval augemented generation (RAG) with chatbots, but it is not the easiest and most universal approach that leads to good results in each\u00a0case.<\/li>\n<\/ol>\n

      Those issues come from the core foundation that LLMs behind chatbots\u00a0have.<\/p>\n

      \"\"
      Chatbot interaction (left) vs. orchestration of a pre-define reproducible process (right). \u00a9 Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

      Instead of relying on a \u201cprompt-response\u201d interaction cycle, enterprise applications should be designed as a series of orchestrated, (partially) AI-driven process steps, each targeting a specific goal. For example, users could trigger a multi-step process that integrates various models and potentially multimodal inputs to deliver more effective results and combine those steps with small scripts that retrieve data without using AI. More powerful and automated workflows can be created by incorporating Retrieval-Augmented Generation (RAG) and minimizing human intervention.<\/p>\n

      This orchestration approach delivers significant efficiency improvements compared to manual orchestration through an interactive interface. Also, not every step in the process should be done by relying purely on an AI model. In the example above, we actually discovered that using the Google Maps API to get nearby stops and transit stations is far<\/em> superior in terms of quality than asking a good LLM like GPT-4o or even a web search RAG engine like Perplexity.<\/p>\n

      Efficiency Gains Through Orchestration<\/strong><\/h3>\n

      Let us think for a moment about a time without AI. Manual processes can take significant time. Let\u2019s assume a task takes one hour to complete manually, and the process is repeated four times, requiring four hours in total. Using a chatbot solution powered by generative AI could save 50% (or whatever percentage) of the time. However, the remaining time is spent formulating prompts, waiting for responses, and ensuring output quality through corrections and adjustments. Is that as good as it\u00a0gets?<\/p>\n

      \"\"
      Time savings when comparing manual and chatbot-based automation. \u00a9 Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

      For repetitive tasks, despite the time savings, the need to formulate prompts, wait, and adjust outputs for consistency can be problematic in organizations where multiple employees execute the same process. To address this, leveraging process templates<\/strong> becomes critical.<\/p>\n

      With templates, processes are generalized and parametrized to be reusable. The effort to create a high-quality process template occurs only once, while the execution for individual cases becomes significantly more efficient. Time spent on prompt creation, quality assurance, and output adjustments is dramatically reduced. This is the core difference when comparing chatbot-based solutions to AI-supported process orchestration with templates. And this core difference has a huge impact on quality and reproducibility.<\/p>\n

      \"\"
      The real savings of process templates. \u00a9 Marcel M\u00fcller\u00a02024<\/figcaption><\/figure>\n

      Also, we now have a narrow field where we can test and validate our solution. In a chatbot where the user can insert anything, <\/em>testing and finding confidence <\/em>in a quantifiable<\/em> way is hard. The more we define and restrict the possible parameters and files a user can insert, the better we can validate a solution quantitatively.<\/p>\n

      Using templates in AI-supported processes mirrors the principles of a Business Process Engine in traditional process management. When a new case arises, these engines utilize a repository of templates and select the corresponding template for orchestration. For orchestration, the input parameters are then\u00a0filled.<\/p>\n

      In our example case of the real estate evaluation process, our template has three inputs: The type of object (single-family home), a collection of pictures of the interior and the\u00a0address.<\/p>\n

      The process template looks like\u00a0this:<\/p>\n

        \n
      1. Use the Google Places API with the given address to find the shops\u00a0around.<\/li>\n
      2. Use the OpenAI vision API to describe the interior conditions.<\/li>\n
      3. Use the Google Places API to find the closest transport options.<\/li>\n
      4. Take the output JSON objects from 1. and 3. and the description of the transport options and create a page of text with GPT-4o with the following structure: Description of the object, shops and transport, then followed by the interior description and a conclusion giving each a\u00a0score.<\/li>\n<\/ol>\n

        In our example use case, we have implemented the application using the entAIngine platform with the built-in no-code\u00a0builder.<\/em><\/p>\n

        Note that in this process, only 1 out of 4 steps uses a large language model. And that is something good! Because the Google Maps API never hallucinates. Yes, it can have outdated data, but it will never \u201cjust make something up that sounds like it could be a reality.\u201d Second, we have verifiability for a human in the loop because now we have real sources of information that we can analyze and sign off\u00a0on.<\/p>\n

        In traditional process management, templates reduce process variability, ensure repeatability, and enhance efficiency and quality (as seen in methodologies like Six Sigma). This is the same mindset we have to adopt\u00a0here.<\/p>\n

        Interfaces for Generative AI Applications<\/strong><\/h3>\n

        Now, we have started with a process that uses an LLM but also solves a lot of headaches. But how does a user interact with\u00a0it?<\/p>\n

        The implementation of such a process can work by coding everything manually or by using a No-Code AI process engine like entAIngine [6].<\/p>\n

        When using templates to model business processes, interactions can occur in various ways. According to my experience in the last 2 years, for 90% of generative AI use cases, the following interfaces are relevant:<\/p>\n

        \u2022 Knowledge Retrieval Interface<\/strong>: Functions like a search engine that can cite and reference sources.<\/p>\n

        \u2022 Document Editor Interface<\/strong>: Combines text processing with access to templates, models, and orchestrations.<\/p>\n

        \u2022 Chat Interface<\/strong>: For iterative, interactive engagement.<\/p>\n

        \u2022 Embedded Orchestration without a Dedicated Interface (RPA)<\/strong>: Integrates into existing interfaces via\u00a0APIs.<\/p>\n

        The question in the end is, what is the most efficient way of interacting? And yes, for some creative use cases or for non-repetitive tasks, a chat interface can be the tool of choice. But often, it is not. Often, the core goal of a user is to create some sort of document. Then, having those templates available in an editor interface is a very efficient way of interacting. But sometimes, you do not need to create another isolated interface if you have an existing application that you want to augment with AI. The challenge here is merely to execute the right process, get the input data for it in the existing application, and show the output somewhere in the application interface.<\/p>\n

        These mentioned interfaces here form the foundation for the majority of generative AI use cases that I have encountered so far and, at the same time, enable scalable integration into enterprise environments.<\/p>\n

        The Bottom\u00a0Line<\/h3>\n

        By getting their minds away from \u201cHow can I use an AI chatbot everywhere?\u201d to \u201cWhat processes do which steps and how can generative AI be utilized in those steps?\u201d businesses create the foundation for real AI impact. Combine AI with existing systems and then only look into the type of user interface that you need. In that way, you can unlock efficiency that businesses that cannot think beyond chatbots never even dream\u00a0of.<\/p>\n

        References<\/h3>\n

        [1] Dumas et al., \u201cFundamentals of Business Process Management\u201d, 2018<\/p>\n

        [2] Object Management Group. \u201cBusiness Process Model and Notation (BPMN) Version 2.0.2.\u201d OMG Specification, Jan.\u00a02014<\/p>\n

        [3] van der Aalst, \u201cProcess Mining: Data Science in Action\u201d,\u00a02016<\/p>\n

        [4] Luthra, Sunil, et al. \u201cTotal Quality Management (TQM): Principles, Methods, and Applications.\u201d 1st ed., CRC Press,\u00a02020.<\/p>\n

        [5] Panagacos, \u201cThe Ultimate Guide to Business Process Management\u201d, 2012<\/p>\n

        [6] www.entaingine.com<\/p>\n

        \"\"<\/p>\n

        \n

        Why Internal Company Chatbots Fail and How to Use Generative AI in Enterprise with Impact<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n

        \t

        \n
        <\/BR>
        \n Dr. Marcel M\u00fcller
        \n \t

        \n
        <\/BR>
        \n        Go to original source<\/a>
        \n \t

        \n
        <\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"

        Why Internal Company Chatbots Fail and How to Use Generative AI in Enterprise with Impact Start with the problem and not with the\u00a0solution Background licensed from elements.envato.com, edit by Marcel M\u00fcller\u00a02024 The most common disillusion that many organizations have is the following: They get excited about generative AI with ChatGPT or Microsoft Co-Pilot, read some […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[151,62,247,248,250,249],"tags":[98,251,252],"class_list":["post-281","post","type-post","status-publish","format-standard","hentry","category-ai","category-aimldsaimlds","category-automation","category-chatbots","category-generative-ai-tools","category-notes-from-industry","tag-ai","tag-chatbots","tag-generative"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/281"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=281"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/281\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=281"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=281"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=281"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}