Monocular Visual-Inertial Odometry with Planar Regularities
This is the open sourcing of our real-time monocular visual-inertial odometry (VIO) system which leverages environmental planes. It includes an efficient robust monocular-based plane detection algorithm which does not require additional sensing modalities such as a stereo, depth camera or neural network, and ability to perform real-time regularization of point features to environmental planes which are either maintained in the state vector as long-lived planes, or marginalized for efficiency. Github Link
|
Indoor AR Table Visual-Inertial Datasets
In this dataset we collected a series small-scale indoor table AR visual-inertial datasets ranging from 1-2 minutes in length. A Intel RealSense Depth Camera D455 is equipped with groundtruth Optitrack markers and provides color monocular image at 30Hz and a 400Hz IMU inertial feed from its Bosch BMI055 IMU. Additionally, the depth projected into the color image by the realsense driver is also recorded (untested). Groundtruth for the IMU of each platform is found through the vicon2gt utility package developed by our group. Github Link
|
C++ Pytorch Pixel-wise Segnet
This project demonstrates training on the comma10k using Pytorch C++ "libtorch" library all without python. Goal is to provide training and prediction with Pytorch C++ "libtorch" library for others to use as a reference. The specific network is a simple U-Net which outputs a [N, classes, W, H] size tensor. To find the classification for a given pixel, the argmax of the classes responses is calculated for each and correspond to the class. Github Link
|
OpenVINS: Platform for Visual-Inertial Navigation Research
The OpenVINS project houses some core computer vision code along with a state-of-the art filter-based visual-inertial estimator. The core filter is an Extended Kalman filter which fuses inertial information with sparse visual feature tracks. These visual feature tracks are fused leveraging the Multi-State Constraint Kalman Filter (MSCKF) sliding window formulation which allows for 3D features to update the state estimate without directly estimating the feature states in the filter. Inspired by graph-based optimization systems, the included filter has modularity allowing for convenient covariance management with a proper type-based state system. Please take a look at the feature list below for full details on what the system supports. Github Link
|
Twitch VOD Creator
The goal of these scripts is to enable both the automatic downloading and fast quick rough editing of twitch vods for archival on youtube. For all of these we can render the chat such that the twitch experience is preserved along with the original stream on the video-only youtube platform. Then we can perform specific "cutting" of downloaded videos to select game playthroughs or segments which others might want to watch. Both the original vod and chat are then render into the same video which can be uploaded to youtube. Additionally, a transcript of the video can be generated using vosk for local use when editing. Github Link
|
Manga Recommendation Engine
This repo has both the scraping and matching utilities to find mangas which are close in content to others. The idea is to create a recommendation system outside MangaDex since there isn't one and thus allow for users to discover other content. Currently this calculates the similarity between each manga and all others by using term frequency–inverse document frequency (Tfidf) to score how much the title + description match. We also score based on the intersection between each manga's labels (genres, themes, demographic, etc) and combine it with the Tfidf score. The found matches can then be leveraged in Neko for users. Github Link
|
vicon2gt: Vicon-IMU fusion for groundtruth trajectory generation
This utility was created to generate groundtruth trajectories using a motion capture system (e.g. Vicon or OptiTrack) for use in evaluating visual-inertial estimation systems. Specifically we want to calculate the inertial IMU state (full 15 dof) at camera frequency rate and generate a groundtruth trajectory similar to those provided by the EurocMav datasets. Please take a look at the report for specific details and a more formal description. Github Link
|
Home ESP8266 Thermostat
This came out of the desire to have both remote and finer grain control of our heating and cooling system. Specifically for multi-floor houses with a single heating and cooling system it might be desirable to cool in respect to the top, ground, or basement floors. This of course isn't achievable in a single thermostat system, and thus a multi-sensor system that reports to central controller architecture was born. Github Link
|
LiDAR-Inertial 3D Plane Simulator
This repository has a LiDAR-inertial 3D plane simulator in it that allows for custom trajectory through 3D environments to be created, and a sensor suite to be sent through it at a given rate. Does ray intersection with floorplan planes to simulate a LiDAR sensor moving through the environment. This was used in our LIPS paper to evaluate the system experimentally. I also have some quick start guides for people wanting to use it and those that want to simulate new enviroments/floorplans. Github Link
|
Pointgrey Ladybug ROS Driver
Nice small package that handles publishing of pointgrey ladybug images onto ROS. This was originally from the Autoware repository, but has been cleaned up and exposes a few more variables (framerate, exposure, gain) to the user. Github Link
|
Continuous Preintegration Source
This is the open sourcing of our continuous preintegration method so that others can leverage it. It includes the simulator used to compare agaist discrete integration and should be pretty easy to run. We use the GTSAM optimization framework and its fixlag smoother implementation which allows for realtime optimization in a sliding window fashion. The repository includes MATLAB scripts to generate the Monte-Carlo simulation, based on trajectories and groundtruth IMU readings from the GAZEBO simulator. Github Link
|
Reach RTK ROS Node
This is a driver that handles publishing of NMEA messages onto the ROS framework. Unlike other drivers, this one will also publish the uncertainty of the GNSS readings alongside the lat,lon,alt measurements. This was tested on the original Reach RTK device by Emlid, which has since replaced with the Reach M+ module. Github Link
|
Android ROS Sensor Driver
Main motivation was getting data off of a Google Tango Yellowstone tablet into ROS for dataset recording. This android app allows for connecting to an external ROS master that one can stream images, IMU, GPS, pressure, magnetometer, and temperature data to. This was a tough project since the OpenCV compiling on android is a a bit complex and not well documented, while working with android is always a handfull. You can check out the repository here. Github Link
|
Image Imu Record
This was a quick ROS node that allows for the recording of ROS image and imu messages to a flat file format to disk. This can be used to convert a ROS bag to individual images, and two data files containing the imu data and image stats. Very simple and easy to use helper node. This was created to allow for consistent offline running of both real time and non-real time algorithms. You can check out the repository here. Github Link
|
Bibtex Publication List
This was a modification of a great original portfolio site created by @vkaravir. It takes in a bibtex file or string and then creates a dynamic table and bar chart to visually illustrate the readings. I worked on updating, commenting, display formatting, and doing a general overhaul on the library. The large contribution was implementing a "reading group" function that allows for an additional due date field to be added to the bibtex entries and displayed in the online table. Github Link
|
Microstrain Communication Driver
This driver originally ran on the LCM framework. I worked on porting over the "pod" to the ROS catkin build system, along with the ROS event system. This was a really fun project to work on, as it was a lot of detective work to understand what the original author tried to code. Along with this, the amount of dependencies where minimized to allow for a more self contained package. Github Link
|
ORB SLAM Modification
As part of the university's undergraduate Summer Scholars program for the summer of 2015, I worked on a modification of the ORB Slam system. This modification was a multi-map approach with a goal of minimizing the loss of data. Through the collection of multiple maps, these smaller maps could be merged into a single large map, thus preventing a large loss of data. Can see the source code here Github Link
|