Autonomous MarioKart Ice Hockey Team

Deep Learning Assignment - UT Austin
November 2019 - December 2019

The Deep Learning final project tasked us with creating agents that could autonomously play 2v2 ice hockey against other teams in the class. The game took place within SuperPyTuxKart (an open source version of MarioKart). We used convolutional neural nets implemented in PyTorch to track the location of the puck and a custom controller to both score and defend. The custom controller operated as a state machine that leveraged the known player of the position and the visually-predicted location of the puck. Our grades were entirely based on how our team performed against other teams – we took 2nd place!

Assignment Details

Code


Autonomous MarioKart Racer

Deep Learning Assignment - UT Austin
October 2019 - November 2019

I created a driverless MarioKart! We were tasked with using a fully-connected, convolutional neural net in PyTorch to predict the location of a target driving point in PyTuxKart (an open source alternative to MarioKart). We were given gameplay images with circular labels on the correct driving point on the horizon, and we had to design and implement a convolutional neural net to predict the location of the ideal driving point while the game is being played. In other words, we built a model to predict where the kart should drive frame-by-frame. We also had to implement a custom controller to steer and accelerate the kart appropriately based on the location of your prediction. The video on the right shows my software predicting a driving point (the green circle) based on training from images labeled with the the ideal driving point (the red circle). My custom controller drives the kart to quickly follow my predicted ideal driving point. I think it’s ready for Rainbow Road!

Assignment Details

Code


Cart Pole Problem - OpenAI Gym

Reinforcement Learning Assignment - UT Austin
March 2020 - April 2020

I solved the classic Reinforcement Learning problem called the Cart Pole Problem. “A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright” (OpenAI Gym). I solved it by implementing the REINFORCE with baseline algorithm from scratch, where the baseline was the predicted value of the agent’s continuous state. Both the state-value estimation and the policy were learned through two separate convolutional neural nets implemented in PyTorch. The result to the right was recorded while the agent was about 200 iterations into training.

Code


Mountain Car Problem - OpenAI Gym

Reinforcement Learning Assignment - UT Austin
March 2020 - April 2020

I solved two classic Reinforcement Learning challenges. The first is called the Mountain Car problem. “The goal is to drive up the mountain on the right; however, the car’s engine is not strong enough to scale the mountain in a single pass. Therefore, the only way to succeed is to drive back and forth to build up momentum” (OpenAI Gym). The cart also has a set time limit for it to make it to the top of the mountain before the episode ends, and it’s reset at the bottom of the hill. I implemented the True Online Sarsa Lambda algorithm in Python and combined it with a tiling approach to map the continuous state space to a discrete environment. After about ~2,000 training episodes, the result is the GIF to the right: the trained agent repeatedly and efficiently accomplishing the task.

Code


Wizard's Chess

Senior Capstone Project - University of Virginia
August 2016 - December 2016

For our senior capstone design project, we were given the liberty of choosing our own design. All circuits had to be custom designed, and we only had three months to take it from concept to completion. Our group chose to design and implement “Wizards Chess” from Harry Potter (without piece destruction). When a player made a move, the circuit boards on each space detected the magnet on each piece. A microcontroller managed the game state and would communicate any changes to a laptop using the UART communication protocol. The laptop would hit the Stockfish chess API to compute the computer’s next move. We could vary the difficult of this algorithm via an integer parameter that varied from 1-100… no one could beat level 5! A motorized XY table received the move command from the microcontroller via the laptop and moved a magnetic arm in order to shift the computer’s physical pieces. The project drew upon our knowledge of software engineering, microcontroller programming, and circuit board design.

Full Video


16-bit MIPS Processor

Computer Architecture and Design - University of Virginia
January 2016 - May 2016

Computer Architecture and Design (ECE 4435) included a semester-long project that required implemented a MIPS processor in VHDL and run an assembly program on it (description below).

The processor included five stages: Fetch, Decode, Execute, Memory, and Writeback. The Fetch stage fetches instructions from memory and processes the program counter. The Decode stage decodes each instruction and passes on the control bits and components necessary for the instruction to be executed, which is done in the Execute stage. The Memory stage interfaces with memory and reads/writes if necessary. Lastly, the Writeback stage writes to one of the 16 general-purpose registers stored in the Register File. A Register Tracker was implemented to mark registers that are being written to or read from in future cycles and mitigate data hazards. A Memory Arbiter was used to mitigate structural hazards when it comes to reading/writing from memory.