Analytics on Table Tennis Equipment Part 2: Recomendation Engine

After creating a text classifier on Part 1 of this series of analysis on Table Tennis equipment, there is another aspect of choosing equipment that is tricky: making sense of all the specs of rubbers. Luckily, Revspin.net has user-generated data we can use for this. Characteristics like Speed, Spin, Control, Gears, Tackyness are rated from 1 to 10 for a big number of rubbers. The issue with this is that is very hard to make sense of all those numbers, Revspin does not offer a comparison tool currently, and even if it did, typical 1-to-1 comparison toolsfall short when you want to find similar products.

With the goeal of making product discovery and recomendation easier, I analysed data from the Top 100 rated rubbers on Revspin to build a recomendation tool using unsupervised learning techniques. Let’s jump to the code!

data = pd.read_csv('top100 rubbers by overall score.csv')
data.head()

data.dtypes

Rubber              object
Speed              float64
Spin               float64
Control            float64
Tacky              float64
Weight             float64
Sponge Hardness    float64
Gears              float64
Throw Angle        float64
Consistency        float64
Durable            float64
Overall            float64
Ratings              int64
Estimated Price     object
dtype: object

Initial Exploration and clustering Link to heading

We can use pair plots to explore posible clusters in the data, next I run pair plots only for Speed, Spin and Control to keep it manageable. They are also the most important aspects on a rubber so it’s all good.

import seaborn as sns
sns.pairplot(data[['Speed','Spin','Control']])

<seaborn.axisgrid.PairGrid at 0x7fbe945e9f10>

Below is an initial clustering using Speed vs Spin

kmeans = cluster.KMeans(8)
kmeans.fit(data[['Speed', 'Spin']])
labels = kmeans.labels_

fig = px.scatter(data, x="Speed", y="Spin",
                 color=labels, hover_data=['Rubber','Estimated Price'], title='Speed vs Spin')
fig.show()

Exploring clusters Link to heading

data['cluster'] = kmeans.labels_
data.sort_values(by='cluster',ascending=False).head(10)

Dimensionality Reduction with PCA Link to heading

Clustering by Speed/Spin is good, but we’re missing a lot of information like Tackiness, Weight, Gears and others. We’ll now use Principal Component Analysis (PCA) to perform dimensionality reduction with the intention of exploring the data in 2D.

%time
data2 = pd.read_csv('top100 rubbers by overall score.csv')
data2["Rubber"]=data2['Rubber'].str.strip()
data2.index = data2['Rubber']
data_pca = data2.drop(columns=['Rubber','Ratings','Estimated Price']).copy()

CPU times: user 6 µs, sys: 0 ns, total: 6 µs
Wall time: 11.4 µs

%time
x = data_pca.values
x = StandardScaler().fit_transform(x)

CPU times: user 4 µs, sys: 0 ns, total: 4 µs
Wall time: 6.91 µs

%time
pca = PCA(n_components=2, random_state=2020)
principalComponents = pca.fit_transform(x)
principalDf = pd.DataFrame(data = principalComponents
             , columns = ['Component_1', 'Component_2'], index=data_pca.index)

CPU times: user 6 µs, sys: 1 µs, total: 7 µs
Wall time: 11.2 µs

principalDf.sample(5)

principalDf = principalDf.join(data2['Estimated Price'])
principalDf.sample(5)

Now we have columns Factor 1 and Factor 2 that we can use for plotting purposes. I added price for an extra point of comparison.

Clustering with Principal Components Link to heading

kmeans = cluster.KMeans(7)
kmeans.fit(principalDf[['Component_1', 'Component_1']])
labels = kmeans.labels_
principalDf['Rubber'] = principalDf.index

fig = px.scatter(principalDf, x="Component_1", y="Component_2",
                 color=labels, hover_data=['Rubber','Estimated Price'])
fig.show()

#Exploring clusters
principalDf['Group'] = kmeans.labels_
principalDf.sort_values(by=['Group','Estimated Price'],ascending=False).head(10)

# Instantiate the KElbowVisualizer with the number of clusters and the metric 
visualizer = KElbowVisualizer(kmeans, k=(2,15) ,timings=False)

# Fit the data and visualize
visualizer.fit(x)    
visualizer.poof() 

<matplotlib.axes._subplots.AxesSubplot at 0x7fbe8df7a650>

principalDf = principalDf.reset_index(drop=True)
principalDf['Rubber'] = principalDf['Rubber'].str.strip() #Fixing whitespaces

K-Nearest Neighbors Link to heading

Now we build our unsupervised KNN model to build our recommender

%time
knn_model = NearestNeighbors()
knn_model.fit(principalDf[['Component_1', 'Component_2']].values)

CPU times: user 6 µs, sys: 0 ns, total: 6 µs
Wall time: 12.2 µs





NearestNeighbors(algorithm='auto', leaf_size=30, metric='minkowski',
                 metric_params=None, n_jobs=None, n_neighbors=5, p=2,
                 radius=1.0)

Exploring Similar Rubbers with KNN Link to heading

At this point we have a model that maps our 2D space we created with PCA. KNN allow us to access the closest points given an initial one and since we condensed all the rubber characteristics into 2 features, our model effectively gives us the most similar rubbers given an initial one. Now we can use this to recommend similar rubbers! Let’s try with the famouse Butterfly Tenergy 05

rubber_name = 'Butterfly Tenergy 05' #Rubber to explore
rubber_point = principalDf[principalDf['Rubber']==rubber_name][['Component_1','Component_2']].values

%time
#Getting Neighbors
kneighbors = knn_model.kneighbors(rubber_point,n_neighbors=11)
kneighbors #Note how it gives us the distance and an index list sorted by it

CPU times: user 9 µs, sys: 1e+03 ns, total: 10 µs
Wall time: 17.6 µs





(array([[0.        , 0.4379683 , 0.52560896, 0.60282679, 0.67766423,
         0.68223504, 0.68839685, 0.72638111, 0.75281269, 0.79369034,
         0.79785331]]),
 array([[97, 44, 88, 80, 42, 24, 38, 62, 46, 75, 48]]))

#Creating a mapping to sort the data based on the distance to the coordinates of interest.
sorter_temp = kneighbors[1][0]
sorterIndex = dict(zip(sorter_temp,range(len(sorter_temp))))

knn_df_sorted = principalDf[principalDf.index.isin(sorter_temp)].copy() #Matching the kneighbors to get the rest of the data
knn_df_sorted['Similarity Rank'] = knn_df_sorted.index.map(sorterIndex)

similar_rubbers = knn_df_sorted.join(data, rsuffix='_r').sort_values(by=['Similarity Rank'])
similar_rubbers.drop(columns=['Component_1','Component_2','Rubber_r', 'Estimated Price_r'], inplace=True)

similar_rubbers

Here we can see some results after testing the model with Tenergy 05. Overall the recomendations make sense, after a quick google search many of the rubbers shows up as good Tenergy alternatives like Tibhar MX-P or Xiom Omega IV Pro.

Conclusion Link to heading

It’s always very nice to see the effectivenses of traditional ML tools like PCA and KNN. Working on this is also a good reminder of the importance of knowing your data, it was easy for me think about what next step made sense given the data and that intuition was useful to evaluate the final results as well.

The next step on this project is to build an actual tool people can use to explore the data. I’m looking forward to it. Stay tunned.