Simultaneous Localization and Mapping (SLAM) Mining Agent

    Below are recordings of a mining agent that I'd programmed to use an online graph slam algorithm to maintain its position in space while discovering its surroundings using sensor data. This is the final project of the "Artificial Intelligence for Robotics" graduate course at Georgia Tech.

    The red point is the agent's position. The grey points are gems to be mined, with the gem types (letters) above them. The black point is the agent's next destination. The agent needs to explore the space (which it does using an achimedian spiral) until finds and extracts all the gems of specific types that it requires. When it finds any gem, it can use as a reference point for its position for future movement.

    The agent gets data from noisy movement and landmark distance sensors and updates a graph of Omega (weights) and Sigma (scalars) matrices with the information. With each movement step, the online algorithm extends the matrices to integrate the information and then removes the old movement data, reducing the matrix sizes back to its prior size. This ensures that the graph will never grow too large regardless of the number of timesteps.

Examples of Omega and Sigma used to calculate Mu position estimates vector

    Similar to a Kalman filter, the graph slam algorithm works via the gaussian equation, however graph slam extends the equation to use a matrix to handle multiple constraints. This allows the agent to explore the space, adding new information about landmarks and movement (mapping) to the matrix while using prior and current instrument measurements to maintain where the agent is in this mapped space (localization).

Gaussian equation 

Particle Filter to Determine 3D Position from Noisy Height Sensors

    Below are recordings of a "Mars Rover" agent that I'd programmed to determine its position and successfully navigate itself to the goal point using a particle filter. This is part of the "Artificial Intelligence for Robotics" graduate course at Georgia Tech.

    The agent builds a collection of particles with randomized parameters based on sensor noise and environment variables. It then weights the viability of each particle using a probability density function with the absolute height and radar to ground height. It resamples from the particle pool with replacement based on the weights, propagates them forward in time according to the agent's movement model, and randomizes some of the variables further.

    Once the particles have converged to a point that the standard deviation between their locations is low enough, the agent uses a weighted average of the particles to estimate its position. With this position, the agent calculates the turning angle to get to the goal point.

    The code for the mars rover and rendering was provided by GaTech; the code for the particle filter and ship's navigation strategy was written by me.

Add caption

Detection of the agent's position and navigation to goal point. 

Agent is red, particles are green/black, estimation is purple.

Detection of the agent's position, showing terrain height visualization. 

Agent is red, particles are green/black, estimation is purple.

Hyperparameter tuning based on trail runs (blurred for Academic Honesty Policy)

Kalman Filter to Navigate Obstacles Based on Noisy Senor Data

I've implemented a Kalman filter that takes noisy sensor data regarding the x and y positions of the asteroids in this simulation to infer the velocity and acceleration based on the bivariate normal distribution. This data is used to navigate a ship from the bottom to the top of the screen, avoiding all of the asteroids by using their predicted future positions. This is part of the Artificial Intelligence for Robotics course for the CS Master's program at Georgia Tech.

The Kalman filter is ideal for this application because of its ability to handle noisy data by using the Kalman gain to gradually transition from relying on the sensor to the internal model and because of its ability to infer correlated variables such as velocity and acceleration using a transition function that can define the physical constraints of the system. Because this system uses 6 variables it requires the use of linear algebra matrix operations vs the more 1D application of the Kalman filter.

The ship uses the predicted future positions of the asteroids to, at any given timestep, determine if steering left, right, or center, would bring to nearer or father from the squared euclidean distance of the future position of the nearest asteroid.

Below are samples of the ship using the Kalman filter to avoid the asteroids and reach the goal. The code for the asteroid field and rendering was provided by GaTech; the code for the Kalman filter and ship's navigation strategy was written by me.

Machine Learning Decision Tree w Bagging Implementation

https://docs.google.com/document/d/1dw_kCWJv_WjaM9C-3Hvuna2rREuLzHWOajnhppyGpp0/edit?usp=sharing

This week I've implemented a decision tree algorithm that splits upon correlation creates a regressor based upon an input set of training data. I've also implemented bagging that uses randomized datasets with replacement to smooth out potential overfitting problems over many trees.

Machine Learning Stock Portfolio Optimization in Python

This week in Machine Learning for Trading I've implemented an optimizer using SciPy that takes any number of stock symbols and most effectively allocates one's portfolio among the provided options based upon previous data in a given date range. The metric for evaluating the profitability of a portfolio is based upon the Sharpe Ratio, which adjusts a stock's income against its risk.

This work correlates to work I'd done using optimizers to find the optimal behavior for a multi-agent system in the Reinforcement Learning class.

Machine Learning Water Turret Pest Deterrent

Here are some pictures of the 3D printed turret I've been working on. It can identify and track targets by adjusting the servos and then activate a water jet by using the solenoid valve controller. It uses Tensorflow using ImageNet custom compiled for Raspberry Pi 3B+. The hull was printed with a hand-constructed Anet A6 3D printer. The hull was designed in FreeCAD. More soon!

ProGab ESL Simulated Lecturer App

16:52 - Running App in Editor

19:20 - Running App on Android Device

Final Paper:

https://docs.google.com/document/d/11P64Je_q1hVFvL8bsmwNBfCxVxAFehfDip5gW0IsLlQ/edit?usp=sharing

Final Presentation Video (shown above):
https://www.youtube.com/watch?v=0qn-9gx6D7k

Final Presentation Slides:
https://docs.google.com/presentation/d/1G4cxJu8j1GPFE8tThRYGPBhdVQyBBw_DrxXB_hP_ftU/edit?usp=sharing

Final Project Files:
https://drive.google.com/open?id=10VWUoTwekPijgbiV8MwLxMCkxK8vWHYp

OMSCS - Educational Technology, Project Research

As I'm wrapping up my project for Educational Technology for the OMSCS program at Georgia Tech, I wanted to post my research regarding my project in the hope that it is of use to others. The project as a simulated lecturer that uses generated content to assist with learning English, specifically targeting students in India. I will post the deliverables and final presentation when they are finished.

Assignment 1 
https://docs.google.com/document/d/17IWZhqfhN9KaeHZVU0XKeNfCq0tV7d8d_4j9skfifho/edit?usp=sharing

Assignment 2 
https://docs.google.com/document/d/1XQKQe04Wub0cYCEeeiBtoK2jXZz52G0trxBNk-OHGSc/edit?usp=sharing

Assignment 3 
https://docs.google.com/document/d/1_LQFnL4CozpUGqbpTHQuP6wAWByt9LH3MSvNSyF2vvA/edit?usp=sharing

Qualifier Question
https://docs.google.com/document/d/1alkwrZklGnZF75-c9fdAzm1gbFRLasT0H5tZm2n9WTk/edit?usp=sharing

Project Proposal
https://docs.google.com/document/d/1z5dj-Oj7_XNZ-msM6uADsBPHFSECPVWRvZcOo9V6QGM/edit?usp=sharing

Milestone 
https://docs.google.com/presentation/d/1q25F9xtQep55eEW4MWptUFcpkXNt2CXOnS7ZwqnPdY8/edit?usp=sharing

VBT Update 3/7/18

(Click to enlarge)

The tree's editor has been connected to the backend and is at last, functioning! The agent loads the tree, and runs the given behavior.

Completed:

• Serializes/deserializes tree to/from JSON
• Rewrote Blackboard to serialize correctly
• Make agents use singular static reference to given tree, using their own blackboard variables
• Manager determines loading and unloading of tree instances
• Rewrote much of tree editor
• Wrote and tested logic for composites, decorators, and actions. Subtree action included and correctly loads and uses tree within tree.

Up Next:

• More actions and composites
• Testing in builds
• More editor debugging features
• Improve UI / usability
• Ultimately, use to design AI for an indie game

VBT Update 2/24/18

I've been hard at work getting the editor linked up to the agent tree. 

• The tree itself will be a static reference that all agents can use a single instance of- so you can have 1000 agents and you'll only have to duplicate their local variables, not all the tree stuff.
• I've focused on doing some neat reflection magic in editor to make adding new behaviors easy.
• The editor and agent behavior is still totally separated so that when building to platforms, all of the editor based code is stripped. In editor however, they link into each other so you can make changes to agent behavior on the fly.
• I've switched JSON from CSV, and to serializing/deserializing classes using the Newtonsoft .NET JSON lib. This solution seems far more sustainable and easier once the initial groundwork is laid. Serialize() & Deserialize(). ez.

I'm currently in that dark tunnel of code wherein you depart with a map and hope to emerge on the other side with the glorious treasure of a new set of features. Wish me luck ; )

In the meantime, I'd dug up the bits of the OST that I'd done for the roguelike Deathstate (on Steam) and realized I'd never posted that so give it a spin!

Deathstate OST

VBT Update 1/22/18

I've added serialization to the behavior tree. You can see a gif of saving and loading a tree out below. (Click to view full size)

I've opted to store the tree in a space separated value sheet with key value pairs, to keep the file size small. Other contenders are XML and JSON, but SSVs are easy to parse and there are no JSON parsing libraries included with Unity, and the overhead with XML doesn't seem worth it here.

Next time I'll be working on having the agent correctly run through the tree in-game, and perform what actions are specified. The "brains" of the operation. This will be tricky so I'll start with running through the tree each update loop, but later state-saving can be added to increase efficiency. 

VBT Update 1/16/18

- Hooked up the VBT editor window with the inspector to both display the current Agent's blackboard
-The values carry over between them
-I've also added support for saving and loading files
- A few visual tweaks / usability

 

Next:
- Serializing/deserializing tree data

VBT Update 1/15/18

Blackboard support has been added to VBT. You can select a data type, name, and value to but used with the agent. A blackboard is a public collection of variables to be used and modified by an agent via it's behavior tree.



Next I will be integrating the blackboard with the behavior tree before hooking up the editor and backend.

"Cause" Game Update

New features:

Tanks can attack units of other teams
Units have health bars than show health
Units go away when out of health
New tank art
Mouse controls
Drag selection of units
Units have teams + colors for teams
Scaling unit selection boxes
Minimap w/ UI in place on right w scaling
Particle effects on tank firing
Damage particles on unit hit
Units aim top cannon at target when attacking, resets when done
Custom mouses that change when unit selected

"Cause" RTS Game

I've been working on an RTS game in Unity for fun. The idea is that you'll try to convert civilians to your side using various means of propaganda and subversion, and then create a force to crush the forces of the other ideologies.

Here's a screenshot thus:

More soon!

Magic the Gathering Web Scrape

When building decks for Magic the Gathering, I like to use gatherer.wizards.com to find the cards, tappedout.net to build the deck and tcgplayer.com for pricing info. Unfortunately this can be time consuming, especially copying, pasting, and formatting each card name from gatherer.wizards.com to tappedout.net.

Fortunately I've written a python script that takes a gatherer.wizards search url as a string and then copies every card name to your clipboard, formatted correctly for mass entry on tcgplayer. Uses python 3.5.4 and BeautifulSoup which you'll need to install with pip.

Check it out on github

One weird quirk is that the string only stays copied to the clipboard while the script is looping for a few seconds (and thereafter if you paste it during this time), though the script gives you a window to paste at this point. Let me know anyone knows a better way to approach clipboard commands.

GameDevMap Scraping with Python

GameDevMap.com is an excellent resource for finding game companies across the world. I wanted to see which companies had jobs that I might be interested in but doing it manually would be insurmountable- cue Python.

You can find the script on github

The script uses BeautifulSoup and urllib in Python 3.5.4. It gets all of the info listed on GameDevMap as well as a search for the company on duckduckgo.com (it has a nice sans-js mode). The search (ie: "careers Westwood Studios") is then combed for keywords, in this case, "jobs", "careers", "programmer", "engineer", "developer", "unity". Unity was a little harder because words like "opportunity" and "community" gave false positives. The search runs two versions and when it's done, writes to csv. The csv can then be imported into Google Sheets where it can be viewed and sorted.

Check out my sheets import here

A nice future improvement would be multithreading which would speed things up quite a bit. Feel free to do what you like with the script, let me know if it helps.

OpenGL Game Engine - Part 1

I've been working on putting together a game engine using OpenGL, SDL and GLM. I've got textures loading, batching and displaying as 2D elements on screen. Input works so you can move around as Mario. 

I'd used this video series to get things off the ground and have been modifying the code once I got a good starting point. Things are very well explained and I highly recommend them.

Next steps are to get the architecture for nested elements and collision detection.

Here's a video of my progress thus:

Here's a link to download the source project (VS 2013)

Flappy Bird Lesson Using Scratch

I've made a version of the game "Flappy Bird" using the drag and drop coding environment, Scratch. Kids seem to respond well to this environment. On Saturday at the library, I'll be doing a class that teaches how to create this game using included assets. Hopefully I can include video of the lesson.

Here's a video of the game: https://drive.google.com/open?id=0B5AauB4MKwieRHhqYVRrU21uQzg

Here's the project: https://scratch.mit.edu/projects/119594189/

Dayz 3D Map Updated

Click here for the 3D Map

I've made a number of changes and updates. Let me know if it runs ok or if there's anything you'd really like to see!

Navigation - I'm not entirely confident in the accuracy of the heightmap data so I've opted not to include pathfinding / navigation. I've included the assets in the source if you want to turn it on and mess around with it though. I'd timed the running speed of the character on the airstrip in-game, then timed the speed of the agent in Unity to make sure the agent speed is accurate.

Movement - I've changed the movement to use a free-fly approach. Thanks to the Unity community for some help there. You can change movement speed with the mousewheel.

Textures - I've included a normal map in the textures and altered the base material to negate some rendering issues. I've also updated the terrain to display a higher res version of the texture at further distances

Lighting - I've included a sun that goes up and down for day/night and also rotates for each day, changing it's position in the sky. Use Z,X,C to alter sun movement speed

Misc - Added town names in English and Cyrillic and made them face the camera and alpha out when getting too close. Added some controls for resetting the scene (Space) and cleaned up lots of little things that I don't immediately recall. Added screen text for controls and credits. I've also tweaked the terrain a bit since it didn't seem to match the ingame map 1:1.

Here's the source project (uses Unity 5.4.0b15). Feel free to modify it as you like. Please let me know if so and please credit me if you do. Thanks to rhennigan for the height map.