On Matrix Factorization in Recommender Systems

Overview

In this article, I will explain the method of matrix factorization in recommender systems.

I will illustrate the definition, properties, and applications of matrix factorization using concrete formulas and Python code.

I will also discuss the advantages and disadvantages of matrix factorization, and introduce an implementation example using the “movielens-100k” dataset.

Source Code

github

  • The Jupyter notebook file is available here

google colaboratory

  • To run on Google Colaboratory, click here

Execution Environment

The OS is macOS. Note that the options differ from Linux or Unix commands.

!sw_vers

ProductName:		macOS
ProductVersion:		13.5.1
BuildVersion:		22G90

!python -V

Python 3.9.17

I will import the basic libraries and use watermark to check their versions. Additionally, I will set the seed for random numbers.

import random

import pandas as pd
import numpy as np

seed = 123
random_state = 123

random.seed(seed)
np.random.seed(seed)

from watermark import watermark

print(watermark(python=True, watermark=True, iversions=True, globals_=globals()))

Python implementation: CPython
Python version       : 3.9.17
IPython version      : 8.17.2

numpy : 1.25.2
pandas: 2.0.3

Watermark: 2.4.3

Definition and Properties of Matrix Factorization

Matrix factorization is a method that decomposes a given matrix into two low-rank matrices. In recommender systems, it extracts latent factors by decomposing the user-item matrix to make recommendations.

Mathematical Representation

The rating matrix $\mathbf{R}$ of user $u$ and item $i$ is decomposed as follows:

$$ \mathbf{R} \approx \mathbf{P} \mathbf{Q}^T $$

Here, $\mathbf{P}$ is the user matrix, and $\mathbf{Q}$ is the item matrix. The dimensions of each matrix are as follows:

$$ \mathbf{P} \in \mathbb{R}^{m \times k}, \quad \mathbf{Q} \in \mathbb{R}^{n \times k} $$

where $m$ is the number of users, $n$ is the number of items, and $k$ is the dimension of the latent factors. The goal is to minimize the sum of squared differences between the rating matrix $\mathbf{R}$ and the predicted matrix $\mathbf{P} \mathbf{Q}^T$.

Optimization Problem

The optimization problem is formulated as follows:

$$ \min_{\mathbf{P}, \mathbf{Q}} \sum_{(u,i) \in \mathcal{K}} \left( r_{ui} - \mathbf{p}_u \cdot \mathbf{q}_i^T \right)^2 + \lambda \left( \sum_u |\mathbf{p}_u|^2 + \sum_i |\mathbf{q}_i|^2 \right) $$

Here, $\mathcal{K}$ is the set of user-item pairs with ratings, and $\lambda$ is the regularization parameter. This regularization term prevents overfitting.

Applications and Implementation

Matrix factorization is applied in various recommender systems. Below is an implementation example using the “Movielens-100k” dataset.

Preparing the Dataset

First, load the “Movielens-100k” dataset and prepare the rating matrix. Assume the dataset is stored in the ml-100k directory.

from scipy.sparse.linalg import svds

# Load the dataset
ratings = pd.read_csv("./ml-100k/u.data", sep="\t", header=None, names=["user_id", "movie_id", "rating", "timestamp"])
ratings = ratings.pivot(index="user_id", columns="movie_id", values="rating")

# Center observed ratings by user mean and fill missing residuals with zero
user_ratings_mean = ratings.mean(axis=1)
R_demeaned = ratings.sub(user_ratings_mean, axis=0).fillna(0).values

Implementation of Matrix Factorization

As a simple example, apply truncated SVD to the user-mean-filled rating matrix. This differs from directly solving the matrix-factorization objective above.

# Singular Value Decomposition
U, sigma, Vt = svds(R_demeaned, k=50)
sigma = np.diag(sigma)

# Create the predicted rating matrix
all_user_predicted_ratings = np.dot(np.dot(U, sigma), Vt) + user_ratings_mean.reshape(-1, 1)
predicted_ratings = pd.DataFrame(all_user_predicted_ratings, columns=ratings.columns)

# Recommend movies for user 1
user_id = 1
user_row_number = user_id - 1  # Row number starts from 0
sorted_user_predictions = predicted_ratings.iloc[user_row_number].sort_values(ascending=False)

# Display the highest predictions for unrated movies
user_data = ratings.loc[user_id]
recommendations = sorted_user_predictions[user_data.isna()].head(5)

display(recommendations.to_frame("Predicted Rating"))

Conclusion

This article covered matrix factorization in recommender systems and truncated singular value decomposition as a simple example.

It presented the formulation and a concrete Python example.

This method is applied in many recommender systems, achieving high-accuracy recommendations.

References