Predicting Success of Racehorses by 3D Depth Images

Client

The Client manages several American-based horse breeding farms that serve the horse racing industry. To enhance services and boost business growth, the Client’s company vigorously introduces AI technologies into racehorse breeding and other workflows. This time, they considered the application of technologies to predict whether a young horse has prospects in races or not.

Challenge: predict race performance of young horses by depth photos

The Client turned to InData Labs with the concept of using 3D image data to predict racehorse performance. The challenge was to create an application able to work with data on existing horses and sales datasets, provide filtering and navigation options. Another predefined requirement was the hardware the application was expected to run on, namely Microsoft Surface Pro tablets.

The Client’s hardware included an internal Microsoft Surface camera as well as a mounted external Intel RealSense D-series camera that allowed taking depth horse photos. For several reasons, the cameras weren’t the best fit for this task. The InData Labs’ team had to develop and adjust the application so that it would smoothly function on Client’s devices.

Solution: Windows 10-based desktop application for tablets to work with internal and external cameras

Our team closely communicated with the Client’s side throughout the project. The following project steps were mapped out:

1. Analyze data provided by the Client.
2. Create an application to gather horse data by using the Intel RealSense camera to create 3D depth photos or point clouds. As opposed to 2D images, the use of the 3D image data would allow doing the following:

• To take measurements of horses on photos
• To further use the data to analyze the dynamics of growth and emphasize other key indicators that influence horse performance

Considering the unusual nature of the business, our team had to dive into the horse racing domain. We figured out that race data provided by the Client had no predictive power. Other data on horses were insufficient.

Our team offered a different approach to gathering data. It took about 70% of the time to fine-tune the proposed method and ensure that the collected data would be of value. The following groups of data were employed for building point clouds:

• Measurements of different parts of a horse body as features
• Data on the dynamics of chosen measurements
• Correlations with known features, like heart size

However, the depth camera distortion and usability issues caused obstacles to taking accurate measurements. During depth photo processing, we had to segment horses on photos and stabilize segmented parts to prepare point clouds for further uses. The result can be seen in the images below:

Having the issues solved, we tailored the application able to use external API to automatically upload existing data on horses and sales or allow manual upload via CSV.

Result: robust app to capture and process depth photos, store data, and work with the target database

The InData Labs’ team delivered the custom app that enabled the Client to implement AI into horse breeding and predicting race performance. We also provided recommendations on the kinds of data and ways of data gathering that will be the best fit to meet the business needs.

The AI-driven app allows users to select a horse, take a photo of it, label and save the photo, and send data to a secure data storage. The user-friendly interface makes it easier to catch, store, and systematize horse photos, and thus, accumulate a large amount of data. When big enough, this data can be used for data science, which can help enhance racehorse breeding.

Start Your AI Software Development Project with InData Labs