In the first post about fraud in digital merchandising systems, we talked about what fraud is, and how and why it occurs. This second post is devoted to the three main fraudulent methods which are most commonly used by merchandising personnel.
All the following scenarios are taken from real life. Mostly they were conceived by the personnel of our customers, and we as the vendor had to struggle with fraud attacks. As we have lots of customers around the world, we noticed that merchandisers come up with more or less the same fraud methods. Here we will discuss different types of fraud scenarios we have faced.
Use a fake camera app and reference images
Mobile apps, which are in abundance on Google Play, can be easily installed on Android-based devices. These mobile apps are designed to prank people. The apps have a realistic user interface of a camera app; however, they are not designed for taking pictures: they allow using fake images instead.
A long time ago in the first product releases we had a system camera which was integrated into the Eyrene platform, and when a merchandiser was going to take a photo of a shelf, the system camera app was opened. It turned out to be a weak point of the product.
What’s more, it turned out that merchandisers used chat groups to share images with each other. In fact, they had a comprehensive gallery of images which could be used in almost any situation. Consequently, all the merchandisers had amazing results at the end of the day.
Technologically, this bug with fake images could be easily eliminated. What we did was we applied changes to our platform and replaced the system camera with our own customized camera software. The app allows us to successfully cope with this kind of fraud now.
We were surprised to find out that other similar software products developed for handling this or that part of the merchandising workflow continue to use the system camera app. It can be easily tricked, as we now know.
When we found out about this kind of fraud, we also developed a specific algorithm to check images' uniqueness. Therefore, we prohibited using the same image in one system, and it worked both for one person and for several.
It is worth mentioning that camera apps and using the camera inside a software product is a tricky thing, as camera apps depend on the hardware, can behave unpredictably, and as a result can require large investments into development.
Take images from the screen of another device or from a printed image
In this case, merchandisers can have a large library of high-quality images. For instance, the main KPI for a merchandiser can be a perfect planogram in a mono-branded refrigerator (beer, non-alcoholic drinks, dairy products, and so on). Therefore, merchandisers can exchange high-quality images in which all the required products are presented in the refrigerator. A merchandiser can have two mobile devices; he opens the image with a perfect presentation of the brand’s products on one device and takes pictures with the camera of the second mobile device. The first one is usually a personal device and the second is corporate.
In the worst cases, merchandisers can take pictures of printed images. For example, there are so-called merchandising books, in which the best possible ways of presenting a brand's products are available. Images from merchandising books can be used by merchandisers, and even in this case the products are perfectly detected and recognized by the neural networks.
We are familiar with the situations when merchandisers can take photos of the displays as they are, and if they understand that they are not accomplishing their goals, they may use merchandising books or pictures shown on other mobile devices to improve their KPIs. There may be new products, which usually have more weight in the objectives system, so merchandisers can cheat it by taking fake photos using merchandising books and images taken in other locations.
As a result, we had to develop a neural network to detect which method of taking photos was used, whether from the screen of a mobile device or from a paper book. We collected a large dataset of fake images from our customers all over the world, and the database is updated all the time. These neural networks work in the case of an image recognized on a server or in the cloud and even on a mobile device. It helps to detect with a high certainty if the image was taken onsite, or was taken from the screen of another device.
Our customers’ products can be placed in stores of various sizes, in urban and suburban areas, images can be taken in completely different conditions (too much light, not enough light), and so on. We came across a situation where the pictures were taken by a camera on a basic mobile phone in poor lighting conditions. The overall impression was that it was a fake photo. However, the investigation showed that the images were real.
So, it becomes obvious that though it seems an easy task to recognize fake and real images, it is a challenge. It’s almost next to impossible to differentiate fake from real images and vice versa. Eyrene’s neural networks weigh the results and as an output show a score that allows one to understand with which level of certainty it is a fake or real image. We continue to update the database not only with examples of perfectly taken images from device screens or from paper, but we also add real images taken in conditions that are far from perfect. We estimate the image recognition accuracy of real and fraudulent images up to 98%.
We are also aware of a similar situation in facial identification. People can try to cheat the system by wearing masks or showing photos instead of a real face. The problem is far from being resolved, and vendors of facial identification systems tend to do the same things we do, just updating the datasets with newer images and training neural networks to identify real images from fraudulent ones.
One more challenge we have been encountering in recent years is that the quality of mobile device screens is becoming better very fast. The quality of images has already become quite close to real life. So, our neural networks are in constant competition against fraud. If previously it was enough to detect pixels to identify fraud, today it can be reflections on the screen, or in some silly cases, a person can forget to exclude the phone’s screen frame.
Use a fake GPS app to take images in other stores
Even before image recognition technology entered the scene, personnel tried to improve their scores using this method. In this scenario, a merchandiser marks that he took images in one store, and in real life, they were from another store. The reasons why they act like this can be various, starting from the fact that in the other store they can get higher scores, or the nearest store is in the neighborhood. To sum up, this is a fraud scenario related to the real and fake geolocation of field personnel.
This method is one of the oldest, so vendors invented a way to monitor geolocation. Some solutions on the market can record the starting point. A merchandiser (or some other specialist) must push a button in the app to mark that he’s beginning work from this geolocation point and at a certain time. The app can have rules for personnel, for example, that they can initiate the work process in the app only if they are at the required geolocation point (of course, we all know that GPS can have issues with accuracy; this fact is usually taken into consideration by developers).
We also know cases when a person used weak points in the system and began by visiting the target store and ended up in the other one. This situation is quite obvious if there are two stores located next to each other.
There are also solutions that make records at both the start and the end of the store visit, and they make up a major part of the solutions on the market. In the Eyrene platform, we had similar limitations for a long time. However, we came across a situation when a field specialist could start in the target store, then leave it for some other place, and end the visit in the target store again. And that’s what we call a next-level fraud scenario.
In the end, we decided to switch to using permanent geolocation monitoring. In the statistics, merchandisers’ supervisors can see the route of a merchandiser through the store. In Eyrene, we use so-called geofencing technology. The technology works like a virtual fence. It means that when a merchandiser leaves the borders of a store, first the platform notifies him that he left the area. Then the system notifies him about the time left to get back to the store, or the visit will be finished and marked as potentially fraudulent.
One more case within this scenario is fake GPS. These apps are also quite popular and can be easily downloaded from Google Play as well.
Technologically, this type of fraud can be quickly detected and eliminated. In Eyrene, we differentiate if the platform gets the data from the real GPS or a fake one, and quickly block the user. In this case, the user is notified with a message which blocks the system and says that they must switch to a real GPS.
Summary
In the second part of the fraud series, we discussed the first three fraud scenarios within merchandising systems, including using a fake camera, taking pictures from the screen of another mobile device or from a printed image, as well as faking geolocation. More methods are coming in the next post!
Have a look at the first post: What is fraud in digital merchandising systems? Why does it occur?
