Revisiting Early Detection of Sexual Predators via Turn-level Optimization
By: Jinmyeong An , Sangwon Ryu , Heejin Do and more
Potential Business Impact:
Finds predators trying to trick kids online.
Online grooming is a severe social threat where sexual predators gradually entrap child victims with subtle and gradual manipulation. Therefore, timely intervention for online grooming is critical for proactive protection. However, previous methods fail to determine the optimal intervention points (i.e., jump to conclusions) as they rely on chat-level risk labels by causing weak supervision of risky utterances. For timely detection, we propose speed control reinforcement learning (SCoRL) (The code and supplementary materials are available at https://github.com/jinmyeongAN/SCoRL), incorporating a practical strategy derived from luring communication theory (LCT). To capture the predator's turn-level entrapment, we use a turn-level risk label based on the LCT. Then, we design a novel speed control reward function that balances the trade-off between speed and accuracy based on turn-level risk label; thus, SCoRL can identify the optimal intervention moment. In addition, we introduce a turn-level metric for precise evaluation, identifying limitations in previously used chat-level metrics. Experimental results show that SCoRL effectively preempted online grooming, offering a more proactive and timely solution. Further analysis reveals that our method enhances performance while intuitively identifying optimal early intervention points.
Similar Papers
Reducing Sexual Predation and Victimization Through Warnings and Awareness among High-Risk Users
Social and Information Networks
Warns risky users to stop online predators.
Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learning
Artificial Intelligence
Finds ways to trick AI, even when it learns.
Empowering Multi-Turn Tool-Integrated Reasoning with Group Turn Policy Optimization
Machine Learning (CS)
Teaches AI to solve math problems step-by-step.