Why can a pilot project go wrong, and how to fix it?

In corporate SaaS, any implementation, in most cases, is dramatically changing prior procedures. And surprisingly, new things come up that no one ever thought could be important.

For example, we are working with AI that is being trained on images. We need shelf images and a catalog of the products or SKUs.

Seems that it’s not something complicated to find. Stores are open, and sales representatives can do many images. However, in COVID, we lost a pilot when the sales representatives were told to limit their commute. No one remembers the full context now, but the result was negative, and while we all were sitting at home, we could do nothing about it. 

Sometimes managers try to double-check everything and ask us to set up a whole image recognition system with perfect recognition accuracy without making the pilot live, i.e., going to stores and providing a few hundred images. Knowing that it can happen, we could collect the dataset independently, but sometimes it’s hard to guess.

Another point is the KPI system. In some offices, the KPIs system is more like a tradition: no one knows how it affects the business and why the target number is 10%, not 20%. So when we start digging into it, we see that most KPIs are managed ad-hoc and don’t translate into business value. Moreover, when we start automating their calculations, the overall results cannot be “fixed,” ad-hoc management isn’t possible, and people are frustrated.