---
abstract: 'We propose StartNet to address Online Detection of Action Start (ODAS) where action starts and their associated categories are detected in untrimmed, streaming videos. Previous methods aim to localize action starts by learning feature representations that can directly separate the start point from its preceding background. It is challenging due to the subtle appearance difference near the action starts and the lack of training data. Instead, StartNet decomposes ODAS into two stages: action classification (using ClsNet) and start point localization (using LocNet). ClsNet focuses on per-frame labeling and predicts action score distributions online. Based on the predicted action scores of the past and current frames, LocNet conducts class-agnostic start detection by optimizing long-term localization rewards using policy gradient methods. The proposed framework is validated on two large-scale datasets, THUMOS’14 and ActivityNet. The experimental results show that StartNet significantly outperforms the state-of-the-art by $15\%$-$30\%$ p-mAP under the offset tolerance of $1$-$10$ seconds on THUMOS’14, and achieves comparable performance on ActivityNet with $\times 10$ smaller time offset.'
author:
- |
    Mingfei Gao$^1$[^1] Mingze Xu$^2$ Larry S. Davis$^1$ Richard Socher$^3$ Caiming Xiong$^3$[^2]\
    $^1$University of Maryland $^2$Indiana University $^3$Salesforce Research\
    [{mgao,lsd}@umiacs.umd.edu, mx6@indiana.edu, {rsocher,cxiong}@salesforce.com]{}
bibliography:
- 'egbib.bib'
title: 'StartNet: Online Detection of Action Start in Untrimmed Videos'
---

[^1]: Work done when the author was an intern at Salesforce Research.

[^2]: Corresponding author.
