AI Project Management: Strategies for Success and Overcoming High Failure Rates
Hate to start with bad news, but there’s an 80% chance your AI project will fail. Why? Because the approaches that worked for traditional software development wouldn’t work for an AI project. However, many teams keep ignoring core differences between traditional vs AI projects and make the same mistakes. As an outcome, we have the failure rate mentioned.
Having a proven track record of successful cases, we at Flyaps have managed AI projects in different domains, from human resources to urban planning. Based on our experience, we want to talk about the most common misconceptions our clients come to us when planning AI projects. We will explain how AI project management is different from managing traditional projects and what strategies we advise you to incorporate to avoid being in that 80% of defeated players.
The differences between traditional PM and AI PM processes
Research from the Project Management Institute states that 88% of organizations have reported gaps in their project management practices when it comes to AI projects. This percentage would have been lower if organisations had adopted AI-specific practices. So let's discuss where the traditional approach wouldn't work and what differences need to be considered.
Data. Lots of data
The main purpose of any AI system is to mimic human brain activity especially when it comes to learning from data. Just like humans, AI learns from the information presented to it, and the more and higher the quality of this information, the better it can reproduce certain patterns. That’s why a solid data foundation is the core aspect of any AI-driven product. The quality and quantity of data the AI system is trained with determines whether it will be able to perform certain tasks.
Suppose, your company developed an internal AI-driven chatbot designed for employees (especially newcomers) to ask questions about the organization, management, and useful contacts for different departments. However, instead of organization’s data, the system was trained on data about the company from the internet. As a result, the chatbot gives incorrect and outdated information when answering.
Different development lifecycle
Agile is still relevant for AI projects. However, it has changed in several ways:
- Training AI models can take many iterations. In the iterative process of training, the AI model is exposed to a dataset and tasked with learning patterns or relationships within that data. After each iteration, the model's performance is evaluated, and adjustments are made based on the errors or successes encountered during the previous attempt. Therefore, in every sprint, you might not end up with a fully functioning version of the AI model.
- Traditionally, when all functionality is developed, testing and documentation the project is considered to be completed. When it comes to AI projects, however, there’s no such thing as “completed”. Even after your AI project is publicly released, you will have to track the indicators like accuracy, fluency, or bias checks (depending on the project) to make sure your AI-powered solution works as intended.
- For AI projects, regular software version control is not enough. You'll want to keep track of changes to both the code and the data used in the model. Suppose, your AI model is taking data from a dynamic source like social media, Slack or Gmail where new information constantly appears and needs to be analyzed by the model. In this case, your development team has to adjust the special data pipeline for AI to handle this data. The data pipeline has to be maintained over time to address any issues, incorporate improvements, and adapt to changes in the data sources or project requirements.
- For AI projects, it is better to give developers and data engineers additional time for experimentation and prototyping. As this is a relatively new technology with many different models and tools appearing every day, technical specialists may need some time to select the best possible AI tech stack and then test their assumptions about some technologies or sets of technologies.
AI is constantly evolving
As we’ve mentioned prior, AI is getting better every day with new models and other advancements. Therefore, to keep up with other similar products, PMs must make sure engineers in their team choose the latest proven technologies.
Difficult to predict the exact ROI for AI product
The impact of adopting AI tends to have a more long-term effect rather than an instant one because AI systems often require time to learn and improve. They become more effective as they gather more data and undergo iterations of training and refinement. Moreover, people and organizations need time to adapt to new AI-driven processes and technologies. There may be resistance or challenges in integrating AI into workflows, which takes time to overcome.
Now that it’s more clear what sets AI projects apart from traditional ones, here are some ways you can avoid the hidden pitfalls and make your project a success.
Project management strategies for AI development
You may already have some ideas about what to do with your AI project based on what we've discussed. However, let's take a closer look at some of the strategies we recommend for PMs with little experience in AI projects.
Assess your resources for AI requirements
AI is not a 100% fit for every business. Model training requires a lot of quality data, but also computing resources that not every company can afford. So before jumping into this undoubtedly revolutionary and powerful technology, PMs should consider whether there are cheaper and more resource-efficient alternatives that can solve the problem at hand just as effectively. Even within AI, there are more complex and simpler models for adoption. The simpler the solution, the better.
For example, some small businesses don't need to implement AI-powered chatbots when a much cheaper automated email response system could be just as effective at freeing up human customer service agents.
Have realistic expectations
The media hype around AI can lead to unrealistic expectations among stakeholders. It's the PM's responsibility to explain the capabilities of the AI tool to stakeholders and what they should expect from it before they test it.
Let's say the management of a small local bank decides to implement an AI-powered chatbot in the bank's app. The project manager's job, in this case, would be to stress that while the chatbot can efficiently handle routine banking queries and transactions, it won’t be able to handle complex financial issues or provide personalized financial advice like a human banker.
Solve the data-related challenges
The situation with data is even more delicate than the previous paragraph might suggest. The data source on which the models are trained and the rights to use it are even more important than the quantity and quality. AI is a relatively new technology, so the laws and regulations to cover all aspects of data use are yet to be released. As new regulations (such as the European Parliament's Artificial Intelligence Act) may appear at any time, it's better to have a legal advisor to help you understand what data you can use and how. Another option is to use some “free” datasets from this list.
For more information, read our article on AI compliance and strategies for managing it.
Natalia Palamarchuk
Understand the life cycle for AI development
We’ve already mentioned some differences between traditional and AI lifecycles. Still, let’s talk more about the importance of the minimum viable product (MVP) in AI development.
The key to any AI project is to develop a small but fully functional solution and only then extend it with additional functionality. The best case scenario is for the engineers preparing and improving the datasets to work independently from other developers so that the team won't lose time and everyone works within the same timeframe.
Imagine your team developing a virtual assistant powered by AI. You start with a basic chatbot prototype, allowing users to ask simple questions and set reminders. After gathering user feedback, the team could enhance the prototype with voice recognition and task scheduling capabilities in the following sprints. By iteratively adding new functionalities based on user needs, you can develop a product that adapts to market demands while maintaining agility in the development process.
Carefully monitor how AI solution is scaled
When planning to scale your AI solution, developers need to be really careful about making changes to the machine learning parts of the product. Even small changes, such as using new algorithms or datasets, can sometimes make the product work differently than before. So the PM needs to check that everything, especially the ML parts, has been well tested before the product goes live.
Cooperate within and across project teams
There will be some roles within the AI team that are different from traditional projects, but with whom PMs will have to interact all the time.
- Data scientists are similar to advanced data analysts but build complex machine-learning models to make predictions or find patterns in the data. They need to be really good at programming, especially in languages like Python or R, and they're comfortable working with large amounts of data.
- Data engineers put together all the pieces of data, handle issues like different ways data is stored or multiple copies of data.
- Infrastructure engineers build and maintain the underlying structure that supports elements like networking, cloud computing, security mesures, data storage, making sure everything runs smoothly across different locations and hardware. They work to create a scalable and efficient environment where ML applications can serve millions of users.
It’s hard to build an AI-focused team without much experience in this field. Therefore, the best possible option for PMs is to find a reliable tech partner with hands-on experience in AI cases that did not join the 80% failure club mentioned in the beginning. In other words, us.
At Flyaps, we are well known in the HR industry after developing the popular CV Compiler resume parser or GlossaryTech plugin used by recruiters at Amazon, Disney, Tesla and Cisco. We also have years of AI expertise in telecom, fintech, logistics and urban planning. If any of these are your field, we're definitely here for you.
Final thoughts
AI is everywhere now, so, to succeed with AI projects, PMs are better to dive into details of AI development as soon as possible. The most important thing to remember about AI projects is that you have to pay close attention to the data and the experts you’re about to work with it. The classic Agile isn't going anywhere, but it has been updated for AI era, which could cause hiccups in the development stage. That's why PMs are better off teaming up with a well-established team experienced in AI/ML development.
Still have questions about the AI development process? Drop us a line - we have so much to share!
Flyaps
