↓Skip to main content

Article Reviews

Theses posts simply summarize what I’ve learned and found in some key research articles that I’ve come across in my research. Full credits are given to the anthors which are listed in each review. I do not provide new information on each topic: I might just give some simplified summaries in my own words.

Simplifying Deep Temporal Difference Learning

·6 mins
The authors propose PQN, a simplified version of DQN that leverages LayerNorm and L2 regularization to get rid of the target networks while maintaining a high level of training stability. The authors also prove a convergence property on PQN, which wasn’t available for the current de-facto algorithm PPO.

Retrieval Augmented Generation with Knowledge Graphs

·8 mins
Overview of the recent and popular GraphRAG method that utilizes a knowledge graph to retrieve information more relevant to the user’s query. Utilizing a graph and community summaries, GraphRAG proposes a pipeline to answer questions that require a broader understanding of a dataset.

Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning

·4 mins
Review of a MARL algorithm that enables learning a set of different policies while sharing gradients from a policy to all others. Thus enabling the agents to learn from it’s own on-policy data and also from off-policy data coming from other agents in that environment.

Particle Filtering Tractography

·1 min
Particle filtering tractography is used to optimize tractography parameters in terms of connectivity and to understand the benefits of a relaxed stopping criterion using partial volume estimation maps.