Skip to content

ORION

ORION

Flight Segment CI Documentation

Orbital Real-time Inference and Observation Network

An autonomous LEO satellite triage system built using:

  • a fine-tuned Liquid AI LFM2.5-VL-1.6B vision-language model (for inference),
  • NASA's F-Prime (for flight software),
  • SimSat (to simulate real payload sensors - GNSS and a camera),
  • a Raspberry Pi 5 (to act as satellite's OBC).

ORION solves the orbital bandwidth bottleneck: roughly 71% of Earth's surface is featureless ocean, yet a traditional satellite downlinks every captured frame. By running a Q4-quantized VLM on-board, ORION classifies each image as HIGH, MEDIUM, or LOW priority and only transmits the most strategically valuable observations in real time.

Motivation

This project was born from a real problem flagged on a real mission. Being the Flight Software subsystems lead on EPFL Spacecraft Team's CHESS mission (part of ESA's Fly Your Satellite! Design Booster programme), I received a Review Item Discrepancy (RID) during our Final Design Review (FDR) from an ESA expert:

"From experience I recommend thinking about pre-loading software that allows you to check that a picture is worth downloading before you do it."

Earth observation satellites generate far more data than their limited comm windows can downlink, and most of it is open ocean, empty desert, cloud cover, which is scientifically worthless. Our mission's approach was to downlink everything and sort on the ground, which would have wasted precious bandwidth and pass time.

I was looking for a solution and then Hack #05: AI in Space, a hackathon co-organised by Liquid AI and DPhi Space, happened. DPhi provided SimSat (the orbital simulator that feeds ORION GNSS coordinates and Mapbox imagery), and Liquid provided the LFM2.5-VL-1.6B vision-language model (the base model for the fine-tuned VLM performing on-board triage). ORION is what came out of that month.

ORION runs a vision-language model directly on the satellite's on-board computer, classifies each frame in real time, and only downlinks what matters. The "what matters" can change mission-to-mission, which would require fine-tuning the VLM on specific data, but ORION serves as a prototype, showing that this approach is technically viable. Furthermore, the local experiments (and the math) proves that this approach does cut-down downlink data and runs in an orbital environment without much issues.

Build and deployment

ORION builds natively on macOS/Linux for development, and cross-compiles for Raspberry Pi 5 via Docker. The flight segment uses CMake + F-Prime's fprime-util toolchain, and the ground segment uses uv for Python dependency management.

Flight segment:

Ground segment:

  • Data generation: Generate the training dataset from SimSat
  • Training: QLoRA fine-tuning of LFM2.5-VL-1.6B
  • Quantization: Convert to GGUF Q4_K_M for Pi deployment
  • Receiver: Run the ground station image receiver
  • Studies: Validation and ablation evaluation

Architecture

The system is split into a flight segment (6 F-Prime components on Pi 5) and a ground segment (receiver, training pipeline, dataset). The flight segment runs an FPP state machine governing four mission modes, with autonomous comm window detection via Haversine distance to the ground station at EPFL.

  • System overview: Component inventory, rate groups, ground segment
  • State machine: IDLE / MEASURE / DOWNLINK / SAFE transitions
  • Data flow: Capture to downlink pipeline, ORIO frame protocol

Usage

The following section goes through the basic usage of this prototype. Refer to the SDD files for more commands, telemetry, and data handling.

Prerequisites

What each part maps to on a real satellite

ORION Component Real Satellite Equivalent
EventAction (C++) OBC mode manager / FDIR logic
NavTelemetry (C++) GNSS receiver payload
CameraManager (C++) Earth observation camera payload
VlmInferenceEngine (C++) On-board AI co-processor
TriageRouter (C++) On-board data handling unit
GroundCommsDriver (C++) X-band radio transmitter
BufferManager (F-Prime) On-board mass memory
comDriver (F-Prime) UHF radio transceiver
Raspberry Pi 5 On-board computer
SimSat GNSS receiver hardware
SimSat Mapbox API Earth observation camera payload hardware
receiver.py Ground station X-band receiver
F-Prime GDS Mission control software

Start SimSat and connect GDS

SimSat provides position data and Mapbox imagery. Start it on your ground station machine (default port 9005).

Launch the ORION binary on the Pi (or locally for development):

./Orion -a <gds-host-ip> -p 50000

Connect GDS from the ground station:

fprime-gds -n --ip-address 0.0.0.0 --ip-port 50000

Open http://localhost:5000: you should see SimSatPositionUpdate events arriving every 5 seconds. The satellite starts in IDLE mode (charging).

Enter MEASURE mode

SET_ECLIPSE true

IDLE transitions to MEASURE (eclipse = imaging on battery). You will see:

  • ModeChanged: IDLE -> MEASURE
  • ModelLoaded: VLM loads into RAM (~15s on Pi)
  • AutoCaptureEnabled every 65 seconds

Observe autonomous capture and inference

Every 65 seconds, CameraManager fetches a Mapbox satellite tile, fuses GPS, and dispatches to the VLM. Watch for:

  • ImageDispatched: image captured at Lat/Lon
  • InferenceComplete: VLM result: HIGH, MEDIUM, or LOW with reasoning and inference time

After inference, TriageRouter routes the frame:

  • HIGH: HighTargetDetected, frame forwarded to GroundCommsDriver for downlink
  • MEDIUM: MediumTargetStored, image saved to disk
  • LOW: LowTargetDiscarded, buffer recycled

When the satellite passes within 2000 km of the ground station (EPFL Ecublens), EventAction auto-transitions to DOWNLINK:

  • CommWindowOpened (distance XXXX km)
  • ModeChanged: MEASURE -> DOWNLINK
  • GroundCommsDriver flushes queued HIGH frames
  • FrameDownlinked for each transmitted frame

Start the ground receiver to accept frames:

python ground_segment/receiver.py

To bulk-download MEDIUM images during the comm window:

FLUSH_MEDIUM_STORAGE

Files are queued to F-Prime FileDownlink at 1 file/sec. Rejected if not in DOWNLINK.

Return to IDLE

SET_ECLIPSE false

Sun is visible: satellite returns to IDLE (charging). Model unloads, captures stop.

Safe mode

Suspend all operations from any state:

ENTER_SAFE_MODE

All components halt. The satellite stays in SAFE until ground commands:

EXIT_SAFE_MODE

Returns to IDLE and re-evaluates conditions (comm window, eclipse) to auto-transition.

Manual overrides

Force specific transitions for testing:

GOTO_IDLE          # From MEASURE or DOWNLINK
GOTO_MEASURE       # From IDLE only
GOTO_DOWNLINK      # From IDLE only

Rejected with GotoRejected if the transition is not allowed from the current state.

API reference

Auto-generated API documentation for both the C++ flight segment (via Doxygen) and the Python ground segment (via mkdocstrings).

  • C++ API: Classes, namespaces, and source files
  • Python API: Receiver, training, and data modules

Contributing

ORION uses clang-format for C++, ruff for Python, and pre-commit hooks for automated formatting. CI runs a native clang-tidy build and a Docker ARM64 cross-compile on every push.

Why "ORION"?

A little easter egg: I grew up watching the Orion constellation every winter from my home in East Delhi. I would know winter is approaching when I could spot the belt before bedtime. There was not much light or air pollution back then, so I have very fond memories of looking up to find constellations in the night sky. I was always an Astronomy kid, but I never knew I would work on a real satellite mission (CHESS at EST) when I grow up. The name felt right for a project that looks down at Earth from the orbit.