Federated Privacy-Preserving Typing Assistant
A mobile keyboard vendor wanted personalised suggestions with no raw text collection. I designed the federated training loop, the differential-privacy budget, and the streaming aggregation backbone, plus a WASM inference runtime that keeps latency under a frame budget on mid-range hardware.
- 180k
- Participating devices / round
- ε = 2.1
- Privacy budget
- +11.4pp
- Top-3 accuracy
- 8ms
- P95 inference latency
The challenge
Central training was off the table, so every quality gain had to come from noisy, non-IID device updates. The team also needed to prove a formal privacy guarantee and keep the model small enough to run inside a keyboard extension.
Approach
- Implemented the federated orchestration with Flower, using secure aggregation, adaptive client sampling, and staleness-aware weighting to handle devices that drop mid-round.
- Applied per-example gradient clipping and DP-SGD via Opacus with a tracked moments accountant, so each release ships with a stated epsilon rather than a vague privacy claim.
- Streamed encrypted client updates through Kafka into Flink jobs that aggregate, validate, and reject poisoned contributions, with Redis holding round state and client cohort assignments.
- Instrumented the whole loop in Grafana: per-round convergence, participation skew, dropout, and privacy spend, so regressions surface within one round instead of one release.
- Compiled the served model to TensorFlow.js with a WASM backend and quantised weights, benchmarked across low-end devices to keep suggestions ahead of the user's typing.
Outcome
The federated model beat the previous static model by 11.4 points of top-3 accuracy at ε = 2.1, with no raw text ever transmitted. Suggestion acceptance rose 19% and the training loop now runs continuously in production.
Results in detail
Figures are from the delivered engagement, normalised where data is confidential.
Accuracy vs. privacy budget spend
Top-3 next-word accuracy per federated round against cumulative epsilon.
Client participation and dropout per round
Sampled cohort size vs. devices completing the round, in thousands.
On-device inference latency by runtime
P95 latency for a single suggestion pass on a mid-range Android device.