Efficient Resource Orchestration For Distributed Large Language Model Inference At The Edge

The increasing deployment of Large Language Models (LLMs) in real-time andresource-constrained environments has exposed critical limitations of centralizedcloud inference, including high latency, cost, and scalability concerns. This thesisaddresses these challenges by proposing two integrated solutions for efficient and fairdistributed LLM inference at the edge: the Fair Cost-Efficient Incentive Mechanism(FCIM) and the Adaptive Dynamic Scheduling Algorithm (ADSA). FCIM introducesan auction-based layer allocation framework that ensures truthful participation andfairness among heterogeneous devices, dynamically balancing task latency, rewardcost, memory feasibility, and system-wide alignment. ADSA complements FCIM byscheduling layer execution in a deadline-aware and preemption-conscious manner,reducing queuing delay while adapting to fluctuating device availability and networkconditions.Together, FCIM and ADSA offer a scalable, incentive-compatible, and resourceefficientapproach for edge-based inference. The mechanisms are extensively evaluatedthrough simulations across diverse model architectures, including GPT-Neo,GPT-3, and BLOOM, under varying GPU configurations and bidding scenarios. Resultsdemonstrate that FCIM reduces communication overhead by up to 54.7% andtask processing time by 36.9%, while ADSA decreases queuing delays by 39% comparedto conventional schedulers. Fairness is quantitatively validated using Jain’sindex over both reward and layer distributions, with FCIM consistently outperformingbaseline methods. This thesis establishes a principled foundation for deploying LLMs in federated, latency-sensitive environments and offers insights into futureextensions involving reinforcement learning, multi-tenant inference, and real-worldedge deployments.