MosaicLeaks shows research agents can leak secrets through search

The privacy risk in deep research agents is not limited to the final answer. MosaicLeaks, published by ServiceNow researchers on Hugging Face, focuses on a quieter channel: the web queries an agent writes while moving between private local documents and public sources.

The failure mode is easy to miss. A research agent may read a private internal metric, date, or vendor name, then use that fact to form the next web search. No single query has to contain the whole secret. But an observer watching outbound traffic can piece together the fragments and infer information that was supposed to stay inside the local corpus.

MosaicLeaks turns that risk into a benchmark. The task chains interleave local and web hops, so the answer to one private sub-question becomes the bridge entity for a later public search. The final split includes 559 training chains, 98 validation chains, and 344 held-out-company test chains. The agent uses a simplified harness with planning, retrieval, document selection, reading, and resolution steps.

The uncomfortable result is that making an agent better at the task can make it less private. For Qwen3-4B, training only for higher chain success raised strict chain success from 48.7% to 59.3%, but answer/full-information leakage rose from 34.0% to 51.7%. The model learned to pack more useful context into web queries, which helped retrieval and hurt confidentiality.

The proposed fix, Privacy-Aware Deep Research, changes the objective. PA-DR raised strict chain success from 48.7% to 58.7% while reducing answer/full-information leakage from 34.0% to 9.9%. For enterprise agents, this is the next evaluation frontier: not just whether the answer is correct, but whether the path to that answer exposes private context.