# Safe Multi-Objective Reinforcement Learning: Zero Constraint Violation and Primal-Averaging Convergence

**Authors:** Chenxi Li, Yao Shu, Siyang Gao, Shuang Qiu, Guangxin Jiang  
**Lab:** Harbin Institute of Technology · **Published:** 29 Oct 2026  
**Links:** [arXiv](https://arxiv.org/abs/2610.39409) · [PDF](https://arxiv.org/pdf/2610.39409) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.39409.html)

## Abstract

Many applications of multi-objective reinforcement learning (MORL) require policies that balance conflicting objectives while satisfying safety constraints. We study safe MORL with zero constraint violation throughout learning, covering both cumulative and stepwise constraints. A key challenge is to explore sufficiently under unknown dynamics and constraint costs without violating the constraints. For a given preference over objectives, we develop an algorithm to learn a safe Pareto optimal policy by combining optimistic reward estimates with pessimistic cost bounds. With high probability, every policy executed over $T$ episodes incurs zero constraint violation. The algorithm returns a safe Markov policy induced by the averaged occupancy measure and achieves an \widetildeO(1/\sqrtT) suboptimality gap under smooth Tchebycheff scalarization. We further develop a preference-free algorithm that first explores safely without a prespecified preference and then computes a safe policy for any given preference without further interaction. Over $T$ exploration episodes and $K$ planning iterations, it retains the same safety guarantee and achieves an \widetildeO(1/\sqrtT+1/\sqrtK) suboptimality gap. Both algorithms require knowledge of a policy that strictly satisfies the constraints and its costs. We then relax these requirements, incurring at most constant cumulative violation. To our knowledge, we provide the first sample-efficient MORL algorithms that guarantee safety for every executed policy.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.