Deep Reinforcement Learning Experiments

This repository is personal learning: after studying the theory of deep reinforcement learning, I implemented the algorithms and ran experiments on classical tasks. The goal was to understand environments, algorithms, and how training proceeds by doing the work, rather than to produce a research system.

I used Gymnasium for the environments and PyTorch for the models, and implemented DQN and PPO myself. I avoided higher-level reinforcement-learning libraries so I could follow what the algorithms were doing.

I treated the repository as complete once the following three Gymnasium tasks were done. Each task has a note.md, and the experiment write-up and videos live in report/report.md.

CartPole (DQN)

The task is to keep a pole upright on a moving cart. At 200000 steps of more_large_model, the video shows the pole staying upright while the cart moves.

Pendulum (PPO)

The task is to swing a pendulum upright. Around environment step 65536 of baseline, I laid out 16 episodes. In that clip, the pendulum was upright in every visualized environment.

LunarLander (PPO)

The task is to land a spacecraft on the pad. For the best policy from seed 42, the video compares continuous actions, a 6-point table, and binning, under two initial conditions.

Raw training outputs — TensorBoard events, checkpoints, and videos recorded during training — stay in runs/ and are not committed. The YAML files in each task’s configs/ are the source of truth for reruns. Figures and videos used in a report are copied into report/, and the report links only to those paths.

I used an AI coding agent, with rules in AGENTS.md that it must not change anything outside the requested scope and must change at most one file at a time, so I could still follow the code. The environment is Python 3.12, managed with uv.

The implementation, configs, and reports are on GitHub.

Stack

• Language: Python 3.12
• Environments: Gymnasium, uv
• Deep learning: PyTorch
• Algorithms: DQN, PPO