The defense willtake place on Tuesday, May 3 at 10:00 am, in the amphitheatre of the Telecommunications Department (Claude Chappe building), INSA Lyon, Villeurbanne.

The presentation will be available on Youtube at the following link: https://youtu.be/k7dh-thqbSk

Title

Large-scale Automatic Learning of Autonomous Agent Behavior with Structured Deep Reinforcement Learning

Abstract

Autonomous robotic agents have begun to impact many aspects of our society,with application in automated logistics, autonomous hospital porters, manufacturing and household assistants. The objective of this thesis is to explore Deep Reinforcement Learning approaches to planning and navigation in large and unknown 3D environments. In particular, we focus on tasks that require exploration and memory in simulated environments. An additional requirement is that learned policies should generalize to unseen map instances. Our long-term objective is the transfer of a learned policy to a real-world robotic system. Reinforcement learning algorithms learn by interaction. By acting with the objective of accumulating a task-based reward, an Embodied AI agent must learn to discover relevant semantic cues such as object recognition and obstacle avoidance, if these skills are pertinent to the task at hand. This thesis introduces the field of Structured Deep Reinforcement Learning and then describes 5 contributions that were published during the PhD.

We start by creating a set of challenging memory-based tasks whose performance is benchmarked with an unstructured memory-based agent. We then demonstrate how the incorporation of structure in the form of a learned metric map, differentiable inverse projective geometry and self-attention mechanisms; augments the unstructured agent, improving its performance and allowing us to interpret the agent’s reasoning process.

We then move from complex tasks in visually simple environments, to more challenging environments with photo-realistic observations, extracted from scans of real-world buildings. In this work we demonstrate that augmenting such an agent with a topological map can improve its navigation performance. We achieve this by learning a neural approximation of a classical path planning algorithm, which can be utilized on graphs with uncertain connectivity.

From work undertaken over the course of a 4-month internship at the research and development department of Ubisoft, we demonstrate that structured methods can also be used for navigation and planning in challenging video game environments. Where we couple a lower level neural policy with a classical planning algorithm to improve long-distance planning and navigation performance in vast environments of 1km×1km. We release an open-source version of the environment as a benchmark for navigation in large-scale environments.

Finally, we develop an open-source Deep Reinforcement Learning interface for the Godot Game Engine. Allowing for the construction of complex virtual worlds and the learning of agent behaviors with a suite of state-of-the-art algorithms. We release the tool with a permissive open-source (MIT) license, to aid researchers in their pursuit of complex embodied AI agents.

Jury

• • Mme. Elisa Fromont – Université de Rennes 1 – Rapporteur
  • M. David Filliat – ENSTA Paris – Rapporteur
  • M. Cédric Démonceaux – Université de Bourgogne – Examinateur
  • M. Karteek Alahari – INRIA Grenoble – Examinateur
  • Mme. Christine Solnon – INSA-Lyon – Examinateur
  • M. Olivier Simonin – INSA-Lyon – Directeur de thèse
  • M. Jilles Dibangoye – INSA-Lyon – Co-encadrant de thèse
  • M. Christian Wolf – Naver Labs Europe – Co-directeur de thèse

This thesis has been done within Nokia Bell Labs France and the Maracas team of the CITI laboratory.

The defense willtake place on Friday, January 28 at 2:00 pm, in the showcase room of the Telecommunications Department (first floor), INSA Lyon, Villeurbanne.

The presentation will be available on Zoom at the following link: https://insa-lyon-fr.zoom.us/j/96199554997

Title

Applications of Deep Learning to the Design of Enhanced Wireless Communication Systems

Abstract

Innovation in the physical layer of communication systems has traditionally been achieved

by breaking down the transceivers into sets of processing blocks, each optimized independently based on mathematical models. This approach is now challenged by the ever-growing demand for wireless connectivity and the increasingly diverse set of devices and use-cases. Conversely, deep learning (DL)-based systems are able to handle increasingly complex tasks for which no tractable models are available. By learning from the data, these systems could be trained to embrace the undesired effects of practical hardware and channels instead of trying to cancel them. This thesis aims at comparing different approaches to unlock the full potential of DL in the physical layer.

First, we describe a neural network (NN)-based block strategy, where an NN is optimized to replace one or multiple block(s) in a communication system. We apply this strategy to introduce a multi-user multiple-input multiple-output (MU-MIMO) detector that builds on top of an existing DL-based architecture. The key motivation is to replace the need for retraining on each new channel realization by a hypernetwork that generates optimized sets of parameters for the underlying DL detector. Second, we detail an end-to-end strategy, in which the transmitter and receiver are modeled as NNs that are jointly trained to maximize an achievable information rate. This approach allows for deeper optimizations, as illustrated with the design of waveforms that achieve high throughputs while satisfying peak-to-average power ratio (PAPR) and adjacent channel leakage ratio (ACLR) constraints. Lastly, we propose a hybrid strategy, where multiple DL components are inserted into a traditional architecture but trained to optimize the end-to-end performance. To demonstrate its benefits, we propose a DL-enhanced MU-MIMO receiver that both enable lower bit error rates (BERs) compared to a conventional receiver and remains scalable to any number of users.

Each approach has its own strengths and shortcomings. While the first one is the easiest to implement, its individual block optimization does not ensure the overall system optimality. On the other hand, systems designed with the second approach are computationally complex and do not comply with current standards, but allow the emergence of new opportunities such as high-dimensional constellations and pilotless transmissions. Finally, even if the block-based architecture of the third approach prevents deeper optimizations, the combined flexibility and end-to-end performance gains motivate its use for short-term practical implementations.

Jury

• • Reviewer: Didier LE RUYET, Professor, CNAM, Paris, France
  • Reviewer: Charlotte LANGLAIS, Permanent Research Staff, IMT Atlantique, Brest, France
  • Examiner: Inbar FIJALKOW, Professor, ENSEA, Cergy, France
  • Examiner: Stephan TEN BRINK, Professor, University of Stuttgart, Stuttgart, Allemagne
  • Thesis supervisor: Jean-Marie GORCE, Professor, INSA Lyon, Villeurbanne, France
  • Thesis co-supervisor: Jakob HOYDIS, Principal Research Scientist, Nvidia, France*
  • Thesis co-supervisor Fayçal AIT AOUDIA, Senior Research Scientist, Nvidia, France*

* Previously at Nokia Bell Labs France

The defense will be held in the Amphitheater, Claude Chappe, and will be streamed live here.

Title

On the Performance of Spatial Modulation and Full Duplex Radio Architectures

Abstract

Index modulation techniques have exhibited great potential in the scenarios foreseen in next-generation wireless networks. Applying in the spatial domain, spatial modulation (SM) as a single radio-frequency (RF) multiple-input–multiple-output (MIMO) solution has attracted wide attention. The SM system has only one transmitting antenna activated for each time slot which results in low system complexity and cost. It exploits the index of the transmitting antennas to convey additional information bits.

To analyze the SM performance, a simulated framework over the time-varying Rician fading channel is built with ADS and Matlab software and a channel state information (CSI) detector is highlighted. The simulation results are verified by the experimental implementation based on the National Instruments (NI) PXI chassis hardware and LabVIEW programming environment. In the practical analysis, two models of the propagation environments are considered, where a channel sounding method is employed in order to extract the channel coefficients.

Despite issues on system complexity and cost, a shortage of spectrum resources can also restrict the development of mobile communications technology. Full duplex (FD) communications have been developed to double the radio link data rate and spectral efficiency through simultaneous and bidirectional communication. The main challenge of FD systems is self-interference (SI), which is caused by the coupling of the transmitting antenna with the receiving one. The combination of FD and SM will not only maintain spectral efficiency but also decrease the complexity of the self-interference cancellation (SIC) because of the single RF chain.

Based on these, a full duplex spatial modulation (FDSM) system is proposed as well as the SIC method. Moreover, the impact of SIC accuracy on the system performance is studied. We focus on the FDSM system imperfections including IQ imbalance, phase noise, power amplifier (PA) nonlinearities and RF switch nonidealities. The bit error rate (BER) performance under different scenarios with these imperfections is analyzed, along with the estimation and cancellation method.

Jury

• Marco Di Renzo, Research Director at CNRS, Reviewer
• Matthieu Crussière, Professor at INSA-Rennes, Reviewer
• Christelle Aupetit-Berthelemot, Professor at Université de Limoges, Examiner
• Taneli Riihonen, Associate Professor at Tampere University, Examiner
• Jean-Marie Gorce, Professor at INSA-Lyon, Examiner
• Dinh-Thuy Phan-Huy, Engineer at Orange Lab, Examiner
• Guillaume Villemaud, Associate Professeur at INSA-Lyon, Thesis director
• Florin-Doru Hutu, Associate Professor at INSA-Lyon, Co-director