Loughborough University
Browse

Bayesian reinforcement learning and Bayesian deep learning for blockchains with mobile edge computing

Download (1.05 MB)
journal contribution
posted on 2024-10-09, 14:06 authored by Alia AsheralievaAlia Asheralieva, Dusit Niyato

We present a novel game-theoretic, Bayesian reinforcement learning (RL) and deep learning (DL) framework to represent interactions of miners in public and consortium blockchains with mobile edge computing (MEC). Within the framework, we formulate a stochastic game played by miners under incomplete information. Each miner can offload its block operations to one of the base stations (BSs) equipped with the MEC server. The miners select their offloading BSs and block processing rates simultaneously and independently, without informing other miners about their actions. As such, no miner knows the past and current actions of others and, hence, constructs its belief about these actions. Accordingly, we devise a Bayesian RL algorithm based on the partially-observable Markov decision process for miner's decision making that allows each miner to dynamically adjust its strategy and update its beliefs through repeated interactions with each other and with the mobile environment. We also propose a novel unsupervised Bayesian deep learning algorithm where the uncertainties about unobservable states are approximated with Bayesian neural networks. We show that the proposed Bayesian RL and DL algorithms converge to the stable states where the miners' actions and beliefs form the perfect Bayesian equilibrium (PBE) and myopic PBE, respectively.

Funding

National Natural Science Foundation of China (NSFC): project 61950410603

Singapore Energy Market Authority (EMA) Energy Resilience: grant NRF2017EWT-EP003-041

Singapore: grant NRF2015-NRF-ISF001-2277

Singapore NRF National Satellite of Excellence, Design Science and Technology for Secure Critical Infrastructure NSoE DeST? SCI2019-0007

A*STAR-NTU-SUTD Joint Research Grant Call on Artificial Intelligence for the Future of Manufacturing RGANS1906

WASP/NTU: grant M4082187 (4080)

Singapore MOE Tier 1: grant 2017-T1-002-007 RG122/17

Singapore MOE Tier 2: grant MOE2014-T2-2-015 ARC4/15

History

School

  • Science

Department

  • Computer Science

Published in

IEEE Transactions on Cognitive Communications and Networking

Volume

7

Issue

1

Pages

319 - 335

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Version

  • AM (Accepted Manuscript)

Rights holder

© IEEE

Publisher statement

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Acceptance date

2020-05-04

Publication date

2020-05-14

Copyright date

2020

eISSN

2332-7731

Language

  • en

Depositor

Dr Alia Asheralieva. Deposit date: 29 May 2024