---
abstract: 'Collaborative Topic Regression (CTR) combines ideas of probabilistic matrix factorization (PMF) and topic modeling (e.g., LDA) for recommender systems, which has gained increasing successes in many applications. Despite enjoying many advantages, the existing CTR algorithms have some critical limitations. First of all, they are often designed to work in a batch learning manner, making them unsuitable to deal with streaming data or big data in real-world recommender systems. Second, the document-specific topic proportions of LDA are fed to the downstream PMF, but not reverse, which is sub-optimal as the rating information is not exploited in discovering the low-dimensional representation of documents and thus can result in a sub-optimal representation for prediction. In this paper, we propose a novel scheme of Online Bayesian Collaborative Topic Regression (OBCTR) which is efficient and scalable for learning from data streams. Particularly, we [*jointly*]{} optimize the combined objective function of both PMF and LDA in an online learning fashion, in which both PMF and LDA tasks can be reinforced each other during the online learning process. Our encouraging experimental results on real-world data validate the effectiveness of the proposed method.'
author:
- |
    Chenghao Liu$^{1,2}$, Tao Jin$^1$, Steven C.H. Hoi$^2$, Peilin Zhao$^3$, Jianling Sun$^1$\
    $^1$School of Computer Science and Technology, Zhejiang University, China\
    $^2$School of Information Systems, Singapore Management University, Singapore\
    $^3$Institute for Infocomm Research, A\*STAR, Singapore\
    twinsken@zju.edu.cn, taoj@zju.edu.cn, chhoi@smu.edu.sg, zhaop@i2r.a-star.edu.sg, sunjl@zju.edu.cn
bibliography:
- 'ijcai16.bib'
title: Online Bayesian Collaborative Topic Regression
---

Introduction
============

Collaborative Topic Regression (CTR) has been actively explored in recent years [@wang2011collaborative]. Instead of purely relying on Collaboretive Filgering(CF) approaches, CTR aims to leverages content-based techniques to overcome inaccurate and unreliable predictions with traditional CF methods due to data sparsity and other challenges. More specifically, CTR combines the idea of probabilistic matrix factorization (PMF) [@mnih2007probabilistic] for predicting ratings, and the idea of probabilistic topic modeling, e.g., Latent Dirchelet Allocation (LDA), for analyzing the content of items towards recommendation tasks. CTR has been shown as a promising method that produces more accurate and interpretable results and has been successfully applied in many recommender systems, such as tag recommendation [@wang2013collaborative; @lu2015content], and social recommender systems [@purushotham2012collaborative; @kang2013ctr].

Despite being studied actively [@wang2011collaborative; @wang2013collaborative], the existing CTR techniques suffer from several critical limitations. First of all, they are often designed to work in a batch mode learning fashion, by assuming that all text contents of items as well as the rating training data are given prior to the learning tasks. During the training process, both LDA and PMF models are usually trained separately in a batch training fashion. Such an approach would suffer from a huge scalability drawback when new data (users or items) may arrive sequentially and get updated frequently in a real-world online recommender system. Second, the existing CTR approach only leverages the content information to improve the CF tasks, but not reverse. The document-specific topic proportions of LDA are fed to the downstream PMF. This two-step procedure is rather suboptimal as the the rating information is not used in discovering the low-dimensional representation of documents, which is clearly not an optimal representation for prediction as the two methods are not tightly coupled to fully exploit their potential. Our work is motivated to explore more efficient, scalable, and effective techniques to maximize the potential exploiting extremes in dealing with data streams from real-world online recommender systems.

To overcome the limitations of traditional CTR, we propose a novel scheme of Online Bayesian Collaborative Topic Regression (OBCTR), which jointly optimizes a unified objective function by combining both PMF and LDA in an online learning fashion. In contrast to the original CTR model, OBCTR is able to achieve a much tighter coupling of both PMF and LDA, where both LDA and PMF tasks influence each other naturally and gradually via the joint optimization in the online learning process. This interplay yields item representations that are more suitable for making accurate and reliable rating prediction tasks. To the best of our knowledge, the proposed OBCTR algorithm is the first online learning algorithm for solving CTR tasks with fully joint optimization of both LDA and PMF. Our encouraging results from extensive experiments on a large real-world data set show that the proposed online learning algorithms are scalable and effective, and the OBCTR technique not only outperforms the state-of-the-art methods for rating prediction tasks but also yields more suitable latent topic proportions in topic modeling tasks.

In the following, we first review some important related work, then present a formal formulation of CTR tasks and the novel Online Bayesian Collaborative Topic Regression algorithms. After that, we conduct extensive empirical studies and compare the proposed algorithms with the existing techniques, and finally set out our conclusions of this work.

Related work
============

In this section, we review two groups of studies related to our work, including (1) variants of CTR models and (2) online Bayesian inference. **Variants of CTR model:** Researchers have extended CTR models to different applications of recommender systems. Some researchers extended CTR models by integrating with other side information. In CTR-smf [@purushotham2012collaborative], authors integrated CTR with social matrix factorization models to take social correlation between users into account. In LA-CTR [@kang2013ctr], they assumed that users divide their limited attention non-uniformly over other people. In HFT [@mcauley2013hidden], they aligned hidden factors in product ratings with hidden topics in product reviews for product recommendations. Some researchers extended CTR to other recommendation tasks. In CSTR [@ding2013celebrity], authors explored how to recommend celebrities to general users in the context of social network. In CTR-SR [@wang2013collaborative], authors adapted CTR model by combining both item-tag matrix and item content information for tag recommendation tasks. There were also several works that attempted to extract latent topic proportions of text information in CTR via deep learning techniques [@wang2014collaborative; @wang2015relational; @van2013deep]. However, all of these work follow the same parameter estimation scheme as [@wang2011collaborative] in a batch learning mode.

**Online Bayesian Inference:** Although the classical regime of online learning is based on decision theory, much progress has been made for developing online variational Bayes [@hoffman2010online; @hoffman2013stochastic; @kingma2013auto; @foulds2013stochastic]. Most of them have adopted stochastic approximation of posterior distribution by sub-sampling a given finite data set, which is unsuitable for many applications where data size is unknown in advance.

To relax this assumption, researchers in [@broderick2013streaming; @ghahramani2000online] made streaming updates to the estimated posterior. The intuition behind this idea is that we could treat the posterior after observing $T-1$ samples as the new prior for the incoming data points. Specifically, suppose the training data $\{\bo_t\}_{t\ge 0}$ are generated i.i.d. according to a distribution $p(\bo|\bx)$ and the prior $p(\bx)$ is given. Bayes’ theorem implies the posterior distribution of $\bx$ given the first $T$ samples $(T \ge 1)$ satisfies $
p(\bx|\{\bo\}^T_{t=0}) \propto p(\bx|\{\bo\}^{T-1}_{t=0})p(\bo_T|\bx).
$ For complex models, we can use approximate inference methods to compute the posterior. For example, [@broderick2013streaming] explored a mean-field variational Bayes algorithm for LDA inference. In addition, [@mcinerney2015population] introduced the population Variational Bayes (PVB) method which combines traditional Bayesian inference with the frequentist idea of the population distribution for streaming inference. [@shi2014online] proposed the Online Bayesian Passive-Aggressive (BayesPA) method for max-margin Bayesian inference of online streaming data. The high scalability of the above methods motivates us to propose Online Bayesian inference for CTR models.

[cc]{}

Collaborative Topic Regression: Revisited
=========================================

Suppose there are $I$ users and $J$ items. Each data sample is a 3-tuple $(i,j,r_{ij})$ where $i \in \{1,2,\cdots,I\}$ is the user index, $j \in \{1,2,\cdots,J\}$ is the item index and $r_{ij} \in \mathbb{R}$ is the rating value assigned to item $j$ by user $i$. We assume the rating data arrives sequentially in an online recommender system. Let $\bR$ denote the whole rating samples and the collection of $J$ items is regarded as a document set $\bW = \{\bw_j\}^{J}_{j=1}$. Let $\bZ=\{\bz_j\}^J_{j=1}$ and $\bTheta =\{\btheta_j\}^J_{j=1} $ denote all the topic assignments and topic proportions of each item. We represent users and items in a shared latent low-dimensional space of dimension $K$, which is equal to the number of topics, user i is represented by a latent vector $\bu_i \in \mathbb{R}^K$ and item j by a latent vector $\bv_j \in \mathbb{R}^K$.

Figure \[fig:CTR\] shows the graphical model of CTR. Basically, the CTR model assumes that each item is generated by a topic model and additionally includes a latent variable $\bm\epsilon_j$ which offsets the topic proportions $\btheta_j$ when modeling the user’s latent vector. This offset variable $\bm\epsilon_j$ can capture the item preference of a particular user based on their ratings. Assume there are $K$ topics $\bPhi=\{\bphi_k\}^{K}_{k=1}$. The generative process of the CTR model is as follows:

1.  For each user $i$, draw user latent vector\
    $\bu_i \sim \mathcal{N}(0,\frac{1}{\sigma_u^2}\bI_K)$

2.  For each item $j$,

    1.  Draw topic proportions $\btheta_j \sim Dirichlet(\alpha)$.

    2.  Draw item latent offset $\bm\epsilon_j \sim \mathcal{N}(0,\frac{1}{\sigma_\epsilon^2}\bI_K))$ and set the item latent vector as $\bv_j = \bm\epsilon_j+\btheta_j$.

    3.  For each word $w_{jn}$($1\le n \le N_j$),

        1.  Draw topic assignment $z_{jn} \sim Mult(\btheta_j)$.

        2.  Draw word $w_{jn} \sim Mult(\bphi_{z_{jn}})$.

3.  For each user-item pair $(i,j)$, draw the rating $
    r_{ij} \sim \mathcal{N}(\bu_i^T\bv_j,\frac{1}{\sigma_r^2}).
    $

In step 2 (c) ii. $\bphi_{z_{jn}}$ denotes the topic selected by the non-zero entry of $z_{jn}$. The topics are random samples drawn from a prior, e.g., $\bphi_k \sim Dirichlet(\beta)$. Note that $\bv_j = \bm\epsilon_j+\btheta_j$, where $\bm\epsilon_j \sim \mathcal{N}(0,\frac{1}{\sigma_\epsilon^2}\bI_K)$, is equivalent to $\bv_j \sim \mathcal{N}(\btheta_j,\frac{1}{\sigma_\epsilon^2}\bI_K)$. Given the document set $\bW$ and rating data $\bR$, we let $\bU=\{\bu_i\}^I_{i=1},\bV=\{\bv_j\}^J_{j=1}$, the goal of CTR is to infer the posterior distribution $$\begin{aligned}
\label{objective1}
&p(\bU,\bV,\bZ,\bPhi,\bTheta |\bW, \bR) \propto p_0(\bU,\bV,\bZ,\bPhi,\bTheta) \nonumber\\
&p(\bW|\bZ,\bPhi)p(\bZ|\bTheta)p(\bV|\bTheta) p(\bR|\bU,\bV).\end{aligned}$$ Because computing the full posterior of $\bU,\bV,\bZ,\bPhi,\bTheta$ directly is intractable, CTR proposed a heuristic two-stage batch learning method for approximate inference . First, CTR approximately infers posterior $p(\bZ,\bPhi,\bTheta|\bW)$ of LDA model via variational inference method [@blei2003latent]. Then, it applies ALS algorithm [^1] for learning the posterior $p(\bU,\bV|\bR, \bTheta)$ of PMF model by feeding the results of $\bTheta$ in the first step. This batch learning approach only leverages the content information to improve the CF tasks, but not reverse and tends to get trapped into local optimum.

Online Bayesian Collaborative Topic Regression {#OCTRbad}
==============================================

Before introducing our novel online parameter estimation method of Online Bayesian Collaborative Topic Regression (OBCTR), we first modify the graphical model of CTR as shown in Figure \[fig:OBCTR1\] to jointly optimize both LDA and PMF and adapt to our online learning method. It is worth noting that this minor modification \[fig:OBCTR\] does not break the main structure of CTR, and our online parameter estimation method could be applied to the various variants of CTR introduced in Section 2.

CTR depicts the generative process of $\bv_j$ with $\bv_j \sim \mathcal{N}(\btheta_j,\frac{1}{\sigma_\epsilon^2}\bI_K)$. In their parameter estimation method, topic proportions $\btheta_j$ (result of LDA) provide features for $\bv_j$ in PMF, but information flow is one-way, which ignores that $\bv_j$ could provide feedback to guide the extraction of topic proportions $\btheta_j$ (they estimates $\btheta_j$ via traditional LDA algorithm which only based on $\bW$ not $\bv_j$). To address this limitation, we first assume that the item latent vector $\bv_j$ is directly close to $\bar{\bz}_j$, where $\bar{\bz}_j$ is a vector with element $\bar{\bz}_j=\frac{1}{N}\sum^N_{n=1}\mathbb{I}(z^k_n=1)$ and $\mathbb{I}$ is the indicator function that equals to 1 if predicate holds otherwise 0. In this way, $\bv_j$ can directly influence topic assignments $\bz_j$ during the procedure of inferring LDA model (variable $\bz_j$ plays a key role in LDA since other hidden variable $\bPhi$ and $\bTheta$ depend on $\bz_j$ and we can easily derive the update rule of them based on $\bz_j$). Second, we replace the generative process of $p(\bv_j|\btheta_j)$ with $p(\epsilon_j|\bar{\bz}_j,\bv_j)$, $\bm\epsilon_j \sim \mathcal{N}(\bv_j-\bar{\bz}_j,\frac{1}{\sigma_\epsilon^2}\bI_K)$, as shown in Figure \[fig:OBCTR1\]. In our setting, $\bv_j$ and $\bar{\bz}_j$ are conditionally dependent, which means their probability of occurrence depends on either event’s occurrence and allows two-way interation. In addition, instead of learning two point estimates of coefficients $\bu_i,\bv_j$, we take a more general Bayesian-style approach and learn the posterior distribution $q(\bu_i,\bv_j)$ in an online method. For rating prediction, we take a weighted average over all the possible latent vectors $\bu_i$ and $\bv_j$, or more precisely, an expectation of the prediction over $q(\bu_i,\bv_j)$ which is defined as $\hat{r}_{ij}\triangleq \mathbb{E}[\bu_i^\top\bv_j]$.

Finally, Algorithm \[OBCTR\] summarizes the detailed framework of the proposed OBCTR algorithm. At each round t, we receive data sample and update both the parameters of LDA and PMF. The following discusses the optimization and each step of the algorithm in detail.

Now, we propose our novel online parameter estimation method of Online Bayesian Collaborative Topic Regression (OBCTR) which is efficient and scalable for learning from data streams. Let us first review the objective function of CTR defined in (\[objective1\]), from a variational point of view, this posterior is identical to the solution of the following optimization problem: $$\begin{aligned}
\label{objective}
&\min\limits_{q(\bU,\bV,\bZ,\bPhi,\bTheta)} KL[q(\bU,\bV,\bZ,\bPhi,\bTheta)\|p_0(\bU,\bV,\bZ,\bPhi,\bTheta))] \nonumber\\
&\quad\quad\quad\quad\quad-\mathbb{E}_q[\log p(\bW|\bZ,\bPhi)p(\bV|\bTheta) p(\bR|\bU,\bV)] \nonumber\\
&\quad\qquad s.t.\quad q(\bU,\bV,\bZ,\bPhi,\bTheta) \in \mathcal{P},\end{aligned}$$ where $KL(q\|p)$ is the Kullback-Leibler divergence, and $\mathcal{P}$ is the space of probability distributions. If we add the constant $\log p(\bW)p(\bR)$ to the objective, it is the minimization of $KL(q(\bU,\bV,\bZ,\bPhi,\bTheta)\|p(\bU,\bV,\bZ,\bPhi,\bTheta|\bW,\bR))$, which is similar with the variational formulation of original LDA [@blei2003latent]. Formally, we formulate our OBCTR model as the optimization problem below: $$\begin{aligned}
\label{objective}
&\min\limits_{q(\bU,\bV,\bZ,\bPhi,\bTheta)} KL[q(\bU,\bV,\bZ,\bPhi,\bTheta)\|p_0(\bU,\bV,\bZ,\bPhi,\bTheta))] \nonumber\\
&\quad\quad\quad\quad\quad-\mathbb{E}_q[\log p(\bW|\bZ,\bPhi)p(\bm\epsilon|\bZ,\bV) p(\bR|\bU,\bV)] \nonumber \\
&\qquad\qquad s.t.\quad q(\bU,\bV,\bZ,\bPhi,\bTheta) \in \mathcal{P},\end{aligned}$$ where $p(\bm\epsilon|\bZ,\bTheta)=\prod^J_{j=1}p(\bm\epsilon_j|\bar{\bz}_j,\bv_j)$. Inspired by streaming Bayesian inference [@broderick2013streaming; @ghahramani2000online], on the arrival of new data $(i,j,r_{ij},\bw_j)$, if we treat the posterior after observing $t-1$ samples as the new prior, the post-data posterior distribution $q_{t+1}(\bu_i,\bv_j,\bz_j,\bPhi,\bTheta)$ is equivalent to the solution of the following optimization problem: $$\begin{aligned}
\label{onlineobjective}
&\min\limits_{q} KL[q(\bu_i,\bv_j,\bz_j,\bPhi,\bTheta)\|q_t(\bu_i,\bv_j,\bz_j,\bPhi,\bTheta))] \nonumber \\
&\quad\quad-\mathbb{E}_q[\log p(\bw_j|\bz_j,\bPhi) p(\bm\epsilon_j|\bar{\bz}_j,\bv_j)p(r_{ij}|\bu_i^\top\bv_j)] \nonumber \\
&s.t. \quad q(\bu_i,\bv_j,\bz_j,\bPhi,\bTheta) \in \mathcal{P}.\end{aligned}$$ This problem is intractable to compute. With the mean field assumption that $q(\bu_i,\bv_j,\bz_j)=q(\bu_i)q(\bv_j)q(\bz_j)$, we can solve this problem via an iterative procedure that alternatively updates each factor distribution as follows in detail.

**For $\bu_i$:** By fixing the distribution $q(\bv_j)$, we can ignore irrelevant terms and solve $$\begin{aligned}
\min\limits_{q(\bu_i)} KL[q(\bu_i)q(\bv_j)\|q_t(\bu_i)p(r_{ij}|\bu_i^\top\bv_j)]. \nonumber\end{aligned}$$ The optimal solution has the following closed form solution: $$\begin{aligned}
q_{t+1}(\bu_i)\propto q_t(\bu_i)\exp(\mathbb{E}_{q(\bv_j)}[\log p(r_{ij} |\bu_i^\top\bv_j)]). \nonumber\end{aligned}$$ If initial prior is normal $q_0(\bu_i)=\mathcal{N}(\bu_i;\bbm_{ui}^0,\Sigma_{ui}^0)
$, by induction we can show that the inferred distribution at each round is also a normal distribution. Let us assume $q_t(\bu_i)= \mathcal{N}(\bu_i; \bbm_{ui}^t,\Sigma_{ui}^t)$. Then, we have

$$\begin{aligned}
q_{t+1}(\bu_i) \hspace{-0.25in}&& \propto \exp(-\frac{1}{2}(\bu_i-\bbm_{ui}^t)^\top(\Sigma_{ui}^t)^{-1}(\bu_i-\bbm_{ui}^t) \nonumber\\
&&\hspace{0.15in}+\mathbb{E}_{q(\bv_j)}[-\frac{(r_{i,j}-\bu_i^\top\bv_j)^2}{2\sigma_r^2}]) \nonumber \\
&& = \mathcal{N}(\bu_i;\bbm_{ui}^\ast,\Sigma_{ui}^\ast),\nonumber\end{aligned}$$

where the posterior parameters are computed as

$$\begin{aligned}
\label{updateu}
&&\Sigma^\ast_{ui} = ((\Sigma^t_{ui} )^{-1}+\frac{\bbm_{vj}\bbm_{vj}^
\top}{\sigma_r^2\bI_K})^{-1}, \\
&&\bbm^\ast_{ui}  =  \bbm^t_{ui} + \frac{r_{i,j}-\bbm_{vj}^\top\bbm^t_{ui}}{\sigma_r^2+\bbm_{vj}^\top\Sigma^t_{ui}\bbm_{vj}}\Sigma^t_{ui}\bbm_{vj}.\nonumber\end{aligned}$$

To make it more efficient, we only update the diagonals of covariance matrix $\Sigma_{ui}^\ast$.

**For $\bv_j$:** The update rule of $\bv_j$ is similar to $\bu_i$ except adding a Gaussian distribution $p(\bm\epsilon_j|\bar{\bz}_j,\bv_j)$, a constraint about the distance between $\bv_j$ and $\bar{\bz}_j$, that explains the difference between topic assignments in content and item preference based on ratings. By fixing the distribution of $q(\bu_i)$ and $q(\bz_j)$, we have the update rule

$$\begin{aligned}
\hspace{-0.2in}&&\hspace{-0.2in}q_{t+1}(\bv_j)\propto q_t(\bv_j)\exp(\mathbb{E}_{q(\bu_i,\bz_j)} [\log p(r_{ij}|\bu_i^\top\bv_j) p(\bm\epsilon_j|\bar{\bz}_j,\bv_j)]) \nonumber \\
%&&\hspace{0.15in}p(\bm\epsilon_j|\bar{\bz}_j,\btheta_j)]) \nonumber \\
\hspace{-0.2in}&&\hspace{-0.2in} \propto \exp(-\frac{1}{2}(\bv_j-\bbm_{vj}^t)^\top(\Sigma_{vj}^t)^{-1}(\bv_j-\bbm_{vj}^t) \nonumber\\
\hspace{-0.2in}&&\hspace{-0.2in}+\mathbb{E}_{q(\bu_i)q(\bz_j)}[-\frac{(r_{i,j}-\bu_i^\top\bv_j)^2}{2\sigma_r^2}\hspace{-0.03in}-\hspace{-0.03in}\frac{(\bar{\bz}_j-\bv_j)^\top(\bar{\bz}_j-\bv_j)}{\sigma^2_\epsilon\bI_K}])\nonumber \\
%&&\hspace{0.15in}-\frac{(\bar{\bz}_j-\btheta_j)^\top(\bar{\bz}_j-\btheta_j)}{\sigma^2_\epsilon}]) \nonumber \\
\hspace{-0.2in}&&\hspace{-0.2in}= \mathcal{N}(\bv_j;\bbm_{vj}^\ast,\Sigma_{vj}^\ast),\nonumber\end{aligned}$$

where the posterior parameters are computed as

$$\begin{aligned}
\label{updatev}
&&\hspace{-0.25in}\Sigma_{mix}=(\Sigma_{vj}^{-1}+\frac{1}{\sigma_\epsilon^2})^{-1}, \\
&&\hspace{-0.25in}\Sigma^\ast_{vj} = ((\Sigma^t_{vj} )^{-1}+\frac{1}{\sigma^2_\epsilon\bI_K} + \frac{\bbm_{ui}\bbm_{ui}^
\top}{\sigma_r^2\bI_K})^{-1}, \nonumber\\
&&\hspace{-0.25in}\bbm_{vj}^\ast = \Sigma_{mix}\Sigma_{vj}^{-1}\bbm_{vj}^t + \Sigma_{mix}\frac{1}{\sigma_\epsilon^2}\bar{\bz}_j-\Sigma_{mix}\frac{1}{\sigma_r^2}\bbm_{ui}\nonumber\\ &&\hspace{0.15in}(\frac{\bbm_{ui}^\top\Sigma_{mix}\Sigma_{vj}^{-1}\bbm_{vj}^t + \bbm_{ui}^\top\Sigma_{mix}\frac{1}{\sigma_\epsilon^2}\bar{\bz}_j-r_{ij}
}{1+\bbm_{ui}^\top\Sigma_{mix}\frac{1}{\sigma_r^2}\bbm_{ui}}). \nonumber\end{aligned}$$

Besides, we adopt the same strategy that only updating the diagonals of covariance matrix $\Sigma_{vj}^\ast$.

**For $\bPhi$ and $\bTheta$:** By fixing the distribution $q(\bZ)$, the update rule for Dirichlet distribution $\bPhi$ and $\bTheta$ is similar to the original LDA, that is,

$$\begin{aligned}
\label{updatePhi}
\theta_{jk} = \frac{C_j^k+\alpha}{\sum^K_{k=1}C_j^k+K\alpha},  \quad \Phi_{kw} = \frac{C^w_k+\beta}{\sum^{D}_{w=1}C^w_k+D\beta},\end{aligned}$$

where $D$ is the vocabulary size, $C_j^k$ is the number of times that terms being associated with topic k within the j-th item, $C^w_k$ is the number of times the term $w(1\le w \le D)$ being assigned to topic $k$ over the whole corpus.

**For $\bz_j$:** Given the distribution of other variables, the conditional distribution of $\bz_j$ is:

$$\begin{aligned}
%\label{updatez}
&&\hspace{-0.35in}q_{t+1}(\bz_{j}|\bv_j,\bPhi,\bw_j)\nonumber\\
&&\hspace{-0.35in}\propto q_t(\bz_{j})\exp( \mathbb{E}_{q(\bPhi)q(\bv_j)}[\log p(\bw_j|\bz_{j},\bPhi)p(\bm\epsilon_j|\bar{\bz}_j,\bv_j)]) \nonumber \\
&& \hspace{-0.35in}\propto q_t(\bz_t)\exp(\sum_{n \in [N_j]}\Lambda_{z_{jn},w_{jn}}-\mathbb{E}_{q(\bv_j)}[\frac{(\bv_j-\bar{\bz}_j)^\top(\bv_j-\bar{\bz}_j)}{\sigma^2_\epsilon\bI_K}])\quad\end{aligned}$$

where $\Lambda_{z_{jn},w_{jn}}= \mathbb{E}_{q(\Phi)}[\log (\bPhi_{z_{jn},w_{jn}})]$. We can do Gibbs sampling to infer $q(\bz_j)$ by canceling out common factors. This hybird strategy has shown promising performance for LDA [@mimno2012sparse; @shi2014online]. Specifically, the conditional distribution of one varibale $z_{jn}$ (the topic assignment of the n-th word in item $j$ ) given others $\bz_{j\neg n}$ is

$$\begin{aligned}
\label{updatez}
&&\hspace{-0.3in}q(z_{jn} = k | \bz_{j\neg n},\bv_j, \bPhi, w_{jn}=w)\\
&&\hspace{-0.3in}\propto (\alpha+C^{k}_{j\neg n})\exp(\Lambda_{k,w_{jn}}+\frac{1}{2\sigma_\epsilon^2N_j} (2m_{vjk}- \frac{1+2C^{k}_{j \neg n}}{N_j})),\nonumber\end{aligned}$$

where $\bz_{j\neg n}$ is the topic assignments in item $j$ (except the n-th word) and $C^{k}_{j \neg n}$ is the number of words in item $j$ (except the n-th word) that are assigned to topic $k$.

Experimental Results
====================

Our experiments were conducted on an extended MovieLens dataset, named as “MovieLens-10M-Plot"[^2], which was originated from the MovieLens 10M[^3]. Specifically, the original MovieLens 10M dataset provides a total of 10,000,053 rating records for 10,681 movies (items) by 69,878 users. However, the original dataset has very limited *text* content information. We enrich the dataset by collecting additional text contents for each of the movie items. Specifically, for each movie item, we first used its identifier number to find the movie listed in the IMDb[^4] website, and then collected its related text of “plot summary". We then combine the “plot summary" text together with each movie’s title and category text given in the MovieLens-10M dataset as a text document to represent each movie. For detailed text preprocessing, we follow the same procedure as the one described in [@wang2011collaborative] to process text information. Finally, we form a vocabulary with 7,689 distinct words. Note that we did not consider the CiteUlike dataset [^5] as used in the previous study [@wang2011collaborative], because their dataset only provides “like" and “dislike" preference, which is kind of implicit feedback and thus unsuitable for our regression task. By contrast, the MovieLens-10M dataset has explicit feedback with ratings ranging from 1 to 5.

Experimental Setup and Metric
-----------------------------

The dataset has more than 10-million rating records. For each experiment, we randomly shuffle the rating records, and then divide them into two parts: the first 90% of the shuffled rating records are used as the training data, and the rest 10% rating data are used as test set. We also randomly draw 5% out of the training data as the validation set for parameter selection. To make fair comparisons, all the algorithms are conducted over 5 experimental runs of different random permutations. For performance metric, we evaluate the performance of our proposed method for prediction task by measuring Root Mean Square Error (RMSE). In the online learning experiments, we evaluate the RMSE performance on the test set after every 50,000 online iterations. In addition, we also evaluate the performance of topic modeling via the log-likelihood of each word in text collection [@hoffman2010online].

Baselines for Comparison and Experimental Settings
--------------------------------------------------

In our experiments, we evaluate the proposed OBCTR algorithms for rating predictions by comparing with some important baselines as follows:

-   **PA-I**: An online learning algorithm for solving online collaborative filtering tasks by applying the popular online Passive-Aggressive (PA) algorithm [@blondel2014online];

-   **CTR**: the existing Collaborative Topic Regression [@wang2011collaborative] . In our context, we replace the ALS algorithm [@hu2008collaborative] with SGD algorithm [@koren2009matrix] since ratings data are explicit, and keep the rest same as the original CTR (note that the LDA step is still performed in a batch manner);

-   **OCTR**: To evaluate the efficacy of joint optimization. We propose a simplified variant of OBCTR, named OCTR, which runs online LDA [@hoffman2010online] for LDA part and SGD for PMF part (but without joint optimization as OBCTR) sepearately. OCTR closely resembles the original CTR — the most important difference is that we extract topic proportions from LDA part, and then feed it to the downstream update of PMF part every time data sample arrives;

-   **OBCTR**: The proposed Online Bayesian CTR algorithm in Algorithm \[OBCTR\].

Besides, to evaluate the topic modeling performance, we also compare our method with the typical Online LDA method:

-   **Online-LDA**: an online Bayesian variational inference algorithom for LDA model[@hoffman2010online]. We take it as a baseline to evaluate how well the model fits the data with the predictive distribution.

For parameter settings, we fix $\alpha=K^{-1},\beta=K^{-1}$ and find the optimal parameters for different algorithms (PA-I, CTR and OBCTR). Specifically, the parameters including $c$ in PA-I, $\sigma_u$, $\sigma_v$ and $\rho$ in CTR and OCTR, and $\sigma_\epsilon$ and $\sigma_r$ in OBCTR. All of these parameters are found by performing a grid search as follows: $ \sigma_\epsilon,\sigma_r \in \{0.5, 1, 2, 4, 8, 16, 32\}$, $c \in \{0.01,0.1,0.2,0.5,1\}$, $\rho \in \{0.01,0.05,0.1,0.2,0.5\}$, $\sigma_u, \sigma_v \in \{0.01, 0.02, 0.04, 0.08, 0.16, 0.32\}$ and $K \in \{5,10,20\}$.

  K       5      10     20
  ------- ------ ------ ------
  PA-I                  
  CTR                   
  OCTR                  
  OBCTR   ****   ****   ****

  : RMSE results after a single pass over training set

\[finalresult\]

Evaluation of Online Rating Prediction Tasks
--------------------------------------------

Figure \[fig:obctr\_rmse\_5\],\[fig:obctr\_rmse\_10\],\[fig:obctr\_rmse\_20\] compares the online performance of the above methods in $K=5$, $K=10$ and $K=20$. We note that the CTR method took at least 6 hours [^6] to precompute the parameters $\bTheta$ and $\bPhi$ by a batch variational inference algorithm. Figure \[fig:rmse\] shows only its performance in the downstream collaborative filtering phase.

As we can see from Figure \[fig:obctr\_rmse\_5\],\[fig:obctr\_rmse\_10\],\[fig:obctr\_rmse\_20\], the CTR-based approaches outperform the online CF algorithm (PA-I) for most cases, which is in line the experiments in [@wang2011collaborative] and validates the efficacy of leveraging additional text information to improve the performance of PMF for online rating prediction tasks. Second, among different CTR-based approaches, the proposed OBCTR consistently outperforms the other algorithms for most cases. This validates the importance of jointly optimizing both online PMF and online LDA to achieve tight coupling of the two techniques. Moreover, it is interesting to find that the gap between the proposed OCTR variant and OBCTR tends to become more significant when $K$ is smaller. We conjecture that this is because when $K$ is small, the PMF performance is relatively inaccurate and thus including the joint optimization becomes more critical for enhancing the unreliable PMF prediction performance.

Finally, Table \[finalresult\] summarizes the final test-set RMSE results after finishing the whole online learning tasks (by a single pass over the training set). Similar observations can be found , in which OBCTR achieves the lowest RMSE result on the test set for rating prediction among all the algorithms. In addition, CTR has better performance than OCTR. This is because CTR directly takes the batch LDA results (pre-computed $\bTheta$ and $\bPhi$) as input for leveraging online PMF task, while online CTR may converge relatively slowly (without the tight coupling). This again shows that it is crucial for the joint optimization in OBCTR.

Performance on Online Topic modeling Tasks
------------------------------------------

Figure \[fig:likelihood\] shows the results about online average predictive log likelihood for OBCTR and Online LDA. Online learning allows us to conduct a large-scale comparison. We can see that OBCTR exhibits consistently better performance than Online LDA, which ignores ratings information, regardless of how many topics we use. That is due to the utilization of rating information to discover the low-dimensional topic proportions, where OBCTR yields additional benefit on this task.

Evaluation of Parameter Sensitivity
-----------------------------------

Figure \[fig:parameter\] shows how RMSE is affected by the choice of two key parameters $\sigma_\epsilon$ and $\sigma_r$ in OBCTR.

As observed from Figure \[fig:parameter\], at the beginning, increasing $\sigma_\epsilon$ leads to decrease the RMSE quickly. After arriving some optimal value, increasing $\sigma_\epsilon$ further may increase the RMSE gradually. Second, we found the optimal value of $\sigma_\epsilon$ also largely depends on the setting of the parameter $\sigma_r$. When $\sigma_r$ is smaller, the optimal value of $\sigma_\epsilon$ is relatively smaller. However, after reaching the optimal value, the further performance changing becomes limited. This indicates that overall, it is relatively easy to choose a good value of $\sigma_\epsilon$ given a fixed $\sigma_r$ setting due to its less sensitivity in the range of optimal values. Our results were consistent to the similar phenomena observed in [@wang2011collaborative].

Figure \[fig:modelsize\] demonstrates the effect of increasing model complexity $K$. This investigation is done by selecting the best achievable RMSE and log-likelihood during the grid parameter search process. As shown in the diagram, increasing the complexity of models (higher $K$ values) leads to improvement of both RMSE and log-likelihood results. However, the gain of predictive performance is paid by a significant computational overhead for more complex models (as shown in Table \[time\]). In a practical online recommender system, one may want to choose a proper value of $K$ to balance the tradeoff between accuracy and computational efficiency.

      $K$        5      10     20     50     100
  ------------ ------ ------ ------ ------ -------
   Time Ratio   1.00   1.29   2.62   5.64   11.43

  : Running time consumed for each model size ($K$). Time Ratio indicates the amount of time required compared with that of the simplest model ($K=5$). []{data-label="time"}

Conclusion
==========

This paper investigated online learning algorithms for making Collaborative Topic Regression (CTR) techniques practical for real-world online recommender systems. Specifically, unlike CTR that loosely combines LDA and PMF, we propose a novel Online Bayesian CTR (OBCTR) algorithm which performs a joint optimization of both LDA and PMF to achieve a tight coupling. Our encouraging results showed that OBCTR converges much faster than the other competing algorithms in the online learning, and thus achieved the best prediction performance among all the compared algorithms. Our future work will analyze model interpretability and theoretical performance of the proposed algorithms.

[^1]: CTR adopts the ALS algorithm [@hu2008collaborative] to solve an implicit feedback problem. In our context, we use the SGD algorithm [@koren2009matrix] since ratings data are explicit.

[^2]: We will release the dataset after the paper is accepted.

[^3]: http://grouplens.org/datasets/movielens/

[^4]: http://www.imdb.com

[^5]: http://www.citeulike.org/faq/data.adp

[^6]: For the vanilla LDA inference method, a larger K value often needs more time for computation.
