According to a recent survey conducted by the IBM Institute for Business Value (IBV) and Oxford Economics, 64% of business leaders and executives face significant pressure from investors, creditors, and lenders to accelerate the adoption of generative AI.
Yet, despite this pressure and the whole hype fueled by ChatGPT and GPT-4 releases, the executives seem to approach generative AI adoption rather cautiously. As we can read in the mentioned survey, leaders today are better acquainted with generative AI than they were with traditional AI in 2016, amid the first AI hype cycle, and more aware of how to plan its adoption and succeed.
While – for most enterprises – the end of 2022 was the time of consideration and evaluation of the ideas for the generative AI adoption, 2023 is the time of experimenting. According to IBV, 42% of organizations will focus on piloting, implementing, operating, and optimizing the usage of generative AI, and the trend will continue throughout the next year.
And what does it look like in your organization? Do you consider implementing generative AI? Or maybe you’ve already started the pilot?
In this article, you’ll find the second patch of 39 questions* to help you plan a successful generative AI adoption. This time, we will focus on the project side of implementation and the challenges that you may encounter.
If you missed the first part of the list, focusing on the business side of the project (identifying the right use cases, aligning the Gen AI implementation with your strategic goals, and evaluating its business impact), you can find it here: 39 Questions to Ask When Implementing Generative AI. Part 1: Strategy & Use Cases
Otherwise, let’s start!
* Based on the original list published on LinkedIn.
Generative AI vs. Compliance
When you identify and evaluate the potential use cases for generative AI adoption, it is important to think about compliance from the very beginning and design the generative AI implementation process in a way that supports it. Think about the following questions:
#21 How do we make sure to follow internal policies?
Internal policies serve as guidelines for ethical conduct, data handling, and other operational aspects within an organization. Compliance with these policies helps maintain consistency, integrity, and alignment with the organization’s values – that’s why it is important to establish mechanisms and procedures to integrate the Gen AI system with internal policies. This may involve:
- Creating rules or constraints within the system’s algorithms and workflows,
- conducting regular audits to verify compliance,
- providing training and awareness programs for the staff involved in using the generative AI system.
#22 How do we make sure to follow regulatory policies?
It is equally important to follow regulatory policies. By considering this question, you can identify the specific regulatory requirements applicable to your organization and the use of generative AI. This includes understanding data privacy and security regulations, industry-specific compliance standards, and other legal obligations. Taking steps to follow regulatory policies may involve conducting thorough compliance assessments, implementing necessary safeguards and controls, documenting compliance efforts, and staying updated with any changes in relevant regulations.
#23 How do we make sure to protect intellectual property?
Steps to protect intellectual property (including proprietary algorithms, datasets, creative outputs, etc.) may involve implementing security measures like access controls and encryption, establishing clear policies and agreements on IP ownership and usage, conducting regular IP audits, and considering legal protections such as patents or copyrights.
While it may not seem to be crucial at the beginning, when you’re just validating the feasibility of using generative AI for a certain use case, it is very important to include these steps in the implementation plan.
Generative AI & Data Management
In general, LLMs are already trained with large data sets, so you don’t need any data to use them. However, you may specialize the foundation model to serve the exact needs of your organization, depending on the use case, e.g., product documentation for the customer service chatbot. Another issue worth considering is data safety. Considering that models may require access to potentially sensitive datasets, it is important to implement strong data management procedures. Here are the questions that will help you prepare for the data-related challenges you may encounter:
#24 Do you have the data that you can use?
Do you possess the necessary data sets that align with the objectives and requirements of the project? It is important to consider the quantity, relevance, and diversity of the data you have, as well as its potential gaps and limitation. By addressing this question, you can determine if additional data acquisition or data collection efforts are necessary or if data augmentation techniques can be employed to enhance the existing data. It also allows you to ensure that the data used is of sufficient quality, accurately represents the target domain, and complies with relevant data governance policies and legal requirements.
#25 What kind of data public data can you use?
If you don’t have much data, check if there are any publicly available data that could be used for your business case. Remember to check the permissions and licenses, as it allows you to determine the boundaries within which you can collect, analyze, and utilize publicly available information. It will help you identify reliable sources, understand any usage restrictions or licensing requirements, and ensure that the use of public data aligns with your organization’s data governance policies.
#26 What kind of proprietary data do you need to use?
Identifying the specific proprietary data sets enables you to implement appropriate security measures, access controls, and data protection protocols. It typically includes trade secrets, customer information, financial data, or research and development findings.
Understanding the value and sensitivity of proprietary data allows you to determine the level of security required to safeguard it. This may involve encryption, data segmentation, or anonymization techniques to minimize the risk of unauthorized access or data breaches. Additionally, it will help you ensure compliance with data protection regulations such as the General Data Protection Regulation (GDPR) or industry-specific regulations.
#27 How will you protect data security?
Once you identify potential vulnerabilities, you can establish incident response procedures and train employees on data security best practices to mitigate risks and prevent unauthorized access, data breaches, or malicious attacks. This may involve implementing a comprehensive data security framework, which may include measures such as encryption, access controls, intrusion detection systems, and regular security audits.
#28 How will you ensure data privacy?
Planning data privacy protection in advance will help you ensure that personally identifiable information (PII) or sensitive data is handled with utmost care and in accordance with applicable privacy laws. Implementing privacy safeguards may include anonymization techniques, data minimization, consent management, and privacy impact assessments.
#29 How will you protect the Intellectual Property of the model’s generated content?
While with most of the models, you have the rights over the AI-generated content – it’s always worth checking it in terms of service, though – it’s valid to think about how you can protect it from unauthorized use, copying, or infringement. Steps to protect IP may include copyrighting the content, utilizing digital rights management tools, and implementing legal agreements or licenses.
#30 Do you use on-premise infrastructure or the cloud?
Choosing between on-premise and cloud-based infrastructure involves considering factors such as data storage requirements, computational resources, budget constraints, and regulatory compliance. On-premise infrastructure provides a higher level of control but may require a significant upfront investment, while the cloud offers scalability and convenience but may raise concerns regarding data sovereignty and security. By evaluating this question, you can make an informed decision based on the specific needs and priorities of your organization.
#31 How will you secure the infrastructure?
Consider network security, system hardening, regular patching and updates, and access controls. Implementing robust security measures will help you protect against unauthorized access, data breaches, or infrastructure vulnerabilities, ensuring the reliability and availability of the generative AI system.
Overcoming Limitations of Generative AI
According to the quoted IBV research on Enterprise Generative AI,
Four in five executives see at least one trust-related issue as a roadblock to generative AI adoption. Cybersecurity, data privacy, and accuracy top the list, and they also have broader concerns about explainability, ethics, and bias in both data sources and the responses that models deliver.
While Gen AI indeed creates a lot of opportunities, it doesn’t come without limitations that you need to be aware of.
#32 How will you handle the model’s bias?
Generative AI models learn from existing data, and if the training data contains biases, the model may replicate and amplify those biases in its generated outputs. Yet, there are ways to mitigate bias and promote fairness in the generated content, such as carefully curating and diversifying the training data, applying bias-correction techniques during model training, or implementing post-processing methods to identify and mitigate bias in the generated outputs.
Read more: Preventing Bias in Generative AI: How to Ensure Models’ Fairness and Accuracy?
#33 How will you handle the model’s hallucinations?
Generative AI models hallucinate. They often create false or misleading information. We all know it. But there are mechanisms to detect and filter out hallucinated or misleading outputs, and it’s worth considering them when planning the implementation. This may involve implementing validation processes, incorporating human oversight and review, or integrating feedback loops to improve the accuracy and reliability of the Gen AI system.
Generative AI Implementation
Even though generative AI implementation may seem like a huge project and may be a bit overwhelming, it doesn’t have to be like that. First, validate your assumptions, test using Gen AI in a chosen use case, and see how it works. Generative AI projects are much like experiments – and should be approached as such. Ok, so how do we get started?
#33 What is the unknown that you want to test?
If you already know where you want to apply generative AI and how it will support your business goals, it’s time to design an experiment. First, you need to specify what you want to test. Write down your leap of faith assumption:
I believe that…
I believe that generative AI can help me improve customer support.
And try to turn it into a hypothesis to be verified:
If… then…
If I use GPT-4 to answer user questions based on the product documentation, then the average time to resolve an issue will be three times shorter than it is now.
As you can see, a hypothesis is much more specific than a bare assumption. Remember to specify metrics and the period of time in which you expect to verify your hypothesis.
#34 What will the test look like?
A Proof of Concept should tell you whether your idea is doable. In the discussed case of customer support, a PoC may just check if it’s technically possible to connect the customer support chat with your internal database and provide answers that meet certain quality standards. If it is, then you’ll plan the pilot to test it in action.
#35 How will you evaluate the Proof of Concept?
Specify your success and failure criteria. Consider side metrics that may be useful in evaluating the success of your PoC. It will allow you to objectively measure the impact, identify any limitations or areas for improvement, and make informed decisions about the scalability and wider deployment of generative AI within your organization after the success of the PoC. At this point, it is important to consider both the measures you have to achieve to execute a pilot and the measures that will tell you to drop the idea.
#36 How will you measure the success or failure of the pilot?
Similarly to how you evaluated the PoC, plan the evaluation of the pilot. Now, you’re working on a larger scale, and you have other metrics to look at.
First, determine the metrics that will allow you to measure the impact of generative AI on the chosen use case. In the example of improving customer support, you could consider metrics such as average response time, customer satisfaction ratings, or ticket resolution rates. Next, design a controlled experiment where you compare the performance of the generative AI system against the current approach. This could involve randomly assigning a portion of user queries to be handled by the Gen AI system while the rest are handled by human agents. Ensure that you have a clear baseline for comparison and collect data throughout the experiment period.
Learn from the hands-on experience of Caju AI!
#37 How can generative AI be integrated into our existing systems and processes?
At this point, it is crucial to evaluate technical requirements, integration challenges, and the potential impact of the generative AI solutions on workflows and existing systems. The aim of this question is to help you develop a plan to seamlessly integrate Gen AI, minimize disruptions, and maximize the benefits of leveraging this technology within your organization.
#38 What measures do you have to achieve to scale to full implementation?
By addressing this question, you evaluate the resources, infrastructure, and capabilities required to expand the implementation beyond the initial pilot and outline a roadmap for scaling up the generative AI implementation. Consider data availability and quality, compliance, scalability of the Gen AI model, IP protection, and operational readiness. Once you run a pilot and have some observations, you may want to review the questions that showed up at the earlier stages.
#39 What measures do you have to achieve to keep, reiterate, or drop the project?
You need to remember that generative AI implementation is an ongoing process that should be monitored and iterated over time. To guide this process, it’s crucial to determine the measures needed to keep, reiterate, or drop the project. By doing so, you can establish clear criteria and performance indicators to assess how the project is progressing and whether it aligns with your organization’s goals. This empowers you to make informed decisions on whether to keep investing in and optimizing the project, refine its implementation, or even decide to discontinue it if it no longer serves its purpose.
Successful Path To Generative AI Adoption – Key Takeaways
While Gen AI implementation may be a long and complex project, you increase your chances of success by approaching it iteratively, as it allows you to learn, adapt, and make improvements throughout the project.
When preparing for generative AI implementation, remember to consider:
- Your strategic goals
- Existing processes in your organization
- The potential use cases
- The potential business impact of the designed solution
- Legal & compliance issues
- Data management & security
- Limitations of the specific models
- Plan for the pilot with clearly defined success criteria.
