Q-Learning Grid World Agent
A reinforcement learning coursework project using Q-learning in a 5×5 grid with obstacles and sparse rewards.
Project Description:
The Q-Learning Grid World Agent is a reinforcement learning project where an autonomous agent learns how to navigate a 5×5 grid world containing obstacles, sparse rewards, penalties, and a special jump transition. The agent uses Q-learning and ε-greedy exploration to learn an optimal policy through trial and error. Your uploaded paper defines this as a 5×5 grid problem with obstacles, a terminal reward, a jump reward, step penalties, and experiments using different epsilon values.
Problem
The challenge is to train an agent to reach a goal state efficiently while avoiding obstacles and minimising unnecessary steps. The agent must balance exploration and exploitation: exploring unknown paths while gradually exploiting the best learned route.
Solution
The solution uses a Q-table where each state-action pair is updated using the Q-learning update rule. Different ε values are compared to evaluate how exploration affects convergence and total reward. The best-performing configuration is used to derive the final greedy policy.
Q-Learning Grid World Agent
FieldDetail
Project Type
Reinforcement Learning
Best Category
AI / Machine Learning / Academic Project
Status
Strong academic project
Main Skill Shown
Q-learning, ε-greedy exploration, reward optimisation
Suggested Tech Stack
Python, NumPy, Matplotlib, Jupyter/Google Colab
Main Features
FeatureDescription
Grid Environment
5×5 state space
Obstacles
Impassable states that force route planning
Reward System
Step penalty, terminal reward and jump reward
Q-table
Stores learned action values
ε-greedy Policy
Controls exploration vs exploitation
Training Graphs
Shows reward progression and episode steps
Policy Visualisation
Shows final optimal direction from each state
Documentation Angle
This is one of your strongest academic projects. It should be documented as a reinforcement learning experimental paper with sections for problem formulation, Bellman equation, Q-table construction, training process, hyperparameter tuning, results, convergence analysis and limitations.
