Garuda's Grand Gaze

 Our conversations over the payload vision for our rocket AgniAstra began on March 28th, 2023. A culmination of years of consuming knowledge and a hunger to now apply it, Garuda was born. The idea was simple, well it still is. How can we use cameras onboard high-speed aircraft and make them intelligent with Computer Vision and Deep Learning to perform something useful in real-time and in situ, using nothing except what the camera sees? How hard could it be, right? It seemed like an exciting challenge at the outset and a year later we have the answer:

 

Chapter 1: The dimensions are what?

iterating the platform

The challenge truly began when we recognized the stringent dimensional constraints we had to navigate. Designing for the 30K category is a completely different endeavor compared to the 10K category, something we anticipated from our previous years with thrustMIT. Specifically, the Nosecone CanSat structure required us to fit everything within a 9cm diameter and a 30cm length. This constraint forced the team to essentially "start from scratch."

We aimed to develop a solution that would remain robust even beyond the specific dimensions of deployment. Consequently, we devised an innovative platform mechanism. This mechanism, featuring a system of ball joints and links, was designed to swing the camera on a platform inside the payload. This setup increased the degree of vibrations and random displacements, thereby making the system more challenging to stabilize and broadening its potential applications.

Numerous iterations were undertaken to perfect the design. Each iteration was meticulously adjusted to achieve the ideal balance of inertia and degree of randomness. The final design required careful calibration to ensure it met all our performance and reliability criteria. This rigorous process ultimately led us to a robust and versatile solution.

 

Chapter 2: We're going to need some serious parts

mounting the hardware

Garuda's outcomes needed to be generated in real-time and executed entirely onboard the system. We also faced the challenge of operating in the total absence of light inside the CanSat, coupled with its extremely compact dimensions. This set us on a quest to find suitable components to address these complex issues.

Our journey involved numerous consultations with professors, late-night brainstorming sessions, and months of meticulous calculations. Eventually, we identified the necessary components to piece together our solution. The compact, GPU-enabled Jetson Nano board provided the processing power, while a 160-degree field-of-view infrared camera served as our vision system. We also integrated a power distribution board (PDB) to manage power efficiently, and developed a mount to fit everything compactly within the CanSat. After assembling these components, the puzzle was finally complete.

Chapter 3: How high is the IQ?

annotating our dataset

The problem statement was to create a highly compact and efficient software system that takes frames from the camera, processes them in real-time to perform 3D tracking and pose estimation, and, using a clever algorithm, determines the angle between the object and the camera. The challenge? Developing a real-time model for such a specific task, with limited hardware constraints, deployed in a harsh environment.

 Months of planning and research led to the development of a custom-made dataset and a Deep Learning CNN model. Annotating a thousand images over weeks was no easy feat, but it was thrilling to imagine the possibilities. Efficient code was crafted to interpret context from pose estimates, and a neat solution—a polynomial regression model—was designed to deduce angles from the visible surface area. The script was then reinforced with fail-safes and redundancy. After a year of dedicated work, we finally had a cutting-edge computing system in place.

Chapter 4: What's Next?

vision system results

Garuda represents an impressive technical demonstration of what becomes achievable when cutting-edge technologies seamlessly integrate, paving the way for incremental progress towards a much larger vision. From intelligent aerial surveillance and high-speed filmmaking to the implementation of missile locking redundancy systems, the range of possibilities is extensive and profoundly impactful. With our endeavors, we aim to actively contribute to shaping this promising future.

 

Written by-

Joshvir Narula