Suppose we wanted to build a movie recommendation system based on the following dataset:

Note the following:

• itemID refers to the movie ID.

• rating is a numeric value that ranges from 0 to 5.

• occupation is the profession of the user. A user can only have a single occupation.

• genre refers to the type of the movie. An item can have multiple genres (e.g. "Comedy|Musical").

Our goal is to build a movie recommendation system based on the following features:

• user-item interaction data - rating.

• user attribute data - occupation.

• item attribute data - genre.

Data preparation

Preparing LightFM Dataset

The first step is to prepare the dataset (Dataset) that we will feed into our LightFM model. There are two things we must supply:

• a list of unique user occupations.

• a list of unique movie genres.

The list of unique occupations can be obtained like so:

The list of unique movie genres can be obtained like so:

Here, we are using the Series' split(-) method to obtain a list of genres (e.g. "Comedy|Musical" becomes ["Comedy","Musical"].

We can now build the LightFM dataset like so:

Preparing users features

Next, we must build the users' features. The input format expected by LightFM is as follows:

We can obtain this input format like so:

We then pass this into the LightFM's build_user_features(-) method:

Here, the shape of sm_user_features is 943 by 964. This is because there are 943 unique users and each userID is treated as an user attribute. We also have 21 unique occupations, which means there are a total of 964 user attributes.

Preparing items features

Similarly, we build the items' features:

We then pass this into LightFM's build_item_features(-) method:

Preparing interaction data

Now that we've prepared the user and item data, we can move on to preparing the interaction data by using the build_interactions(-) method:

Internal mapping of IDs

Instead of using the original IDs of the users and items in our dataset, LightFM internally assigns a new consecutive non-negative integer ID to each user and item. We can see the mapping like so:

The user_id_map is:

The user_feature_map is:

Remember, LightFM also treats the ID of every user as a feature - this is why we see the ID of our users included in the user_feature_map. We will later use these mappings to perform predictions.

Evaluating performance

Since we want to evaluate the performance of our LightFM model, we will use the library's random_train_test_split(-) method:

It's finally time to fit the LightFM model:

It took me roughly 5 minutes to train the model using my M1-chip MacOS. Let's now evaluate the performance (precision@k) of our model using our testing data:

Here, we have the precision@k value for every user. We can compute the mean average precision@k (MAP@K) like so:

User-to-Item recommendation

Suppose we wanted to recommend 5 movies to a particular user with ID 63. We can use the predict method of our model like so:

Note the following:

• we convert the user_id to the user ID used by LightFM internally using user_id_map dictionary that we obtained earlier by dataset.mapping().

• we supply a list of movie IDs that we want to obtain a recommendation score for. Since we are interested in finding the top k movie recommendations for this user, we need to compute the recommendation score for every movie.

• the item_id_map is a dictionary that maps the original movie IDs to the non-negative consecutive integers used by LightFM internally:

  item_id_map

  {242: 0,

  257: 1,

  111: 2, ...

We convert the list of recommendation scores into a Series such that we can assign the original movie IDs as the index:

Finally, we sort the scores in descending order to obtain the top movie recommendations for the user:

Here, we see that the movie with ID 222 is the top recommended movie for this particular user. Note that the magnitude of the scores does not carry any meaning - they are simply used for ranking purpose only.

Item-to-item recommendation

To obtain the vector embedding of each movie, we can use the get_item_representations(-) method:

Note the following:

• for every movie, we get a vector representation that encodes the characteristics of the movie.

• the shape is 352 by 20 because there are 352 movies in total and we set the latent vector size (NO_COMPONENTS) to 20 during model fitting.

We compute the cosine similarity for each pair of movies:

It's good practise to convert this NumPy array into a DataFrame as we can assign the movie IDs to the rows and columns like so:

Remember, the item_id_map is a dictionary obtained by dataset.mapping() from earlier that maps the original movie IDs to the non-negative consecutive integers used by LightFM internally:

To get the top 5 recommendations for a particular movie, say movie with ID 1049:

As expected, the first value is the movie itself, which receives a perfect score. We see that the movie with ID 832 is the most recommended (similar) movie!

Users' vector embeddings

Not only has LightFM learned the vector embedding of each movie, it has also learned it for each user. To obtain the users' vector embeddings, use the get_user_representations(-) method:

To find out how similar two users are, we can do exactly what we did for item-to-item recommendations, that is, compute the pairwise cosine similarity and sort the values in descending order:

Here, we see that user with ID 485 is most similar to user 196.