Advancing Decentralized AI: Scalable, Adaptive, and Client-Centric Learning Systems

Decentralised AI systems, particularly those employing federated learning (FL), offer a promising approach to training machine learning models across distributed data sources while preserving privacy. However, they face significant challenges, including system heterogeneity, dynamic client availability, and resource constraints. Addressing these issues is crucial for the effective deployment of FL in real-world scenarios. In this talk, I will discuss our recent efforts to enhance the robustness and adaptability of FL systems. I will introduce REFL, a resource-efficient FL framework that decouples the collection of participant updates from model aggregation, intelligently selecting participants based on their likelihood of future availability to maximise resource utilisation. Then I will cover the development of FLOAT, an automated tuning framework that dynamically optimises resource utilisation to meet training deadlines, mitigating stragglers and dropouts through various optimisation techniques. Additionally, QKT is presented as a framework that enables tailored knowledge acquisition to fulfil specific client needs without direct data exchange, employing a data-free masking strategy to facilitate communication-efficient query-focused knowledge transfer while refining task-specific parameters to mitigate knowledge interference and forgetting. Finally, our UKRI-EPSRC-funded project, KUber, addresses these challenges by developing a distributed knowledge delivery system to enhance FL scalability and efficiency. KUber’s architecture facilitates seamless knowledge exchange among learning entities, optimising resource utilisation and model convergence.