KDEAmbi/docs/hyperion_processor_algorithm.md

Hyperion Processor Algorithm

This document outlines the algorithm implemented in the HyperionProcessor module, which aims to replicate the core image processing pipeline of Hyperion.ng for generating LED colors from screen content.

Overall Goal

The primary goal of the HyperionProcessor is to transform a captured screen image into a set of colors suitable for driving an LED strip, creating an Ambilight-like effect. It focuses on accurately sampling colors from relevant areas of the screen while ignoring irrelevant parts like black borders.

Processing Steps

The HyperionProcessor follows a multi-step pipeline to achieve this:

1. Black Border Detection

Before sampling colors, the processor identifies and accounts for black borders (letterboxing or pillarboxing) in the input image. This ensures that the LEDs only react to the actual video content and not to the black bars.

• Component: BlackBorderDetector
• Mechanism: The BlackBorderDetector analyzes the edges of the input image. It samples pixels at strategic locations (e.g., 1/3, 2/3, and center of the image dimensions) to find the first non-black pixel. This helps determine the extent of the black borders.
• Output: A BlackBorder struct containing the horizontalSize and verticalSize of the detected borders. If no borders are detected, these sizes will be zero.

2. LED Area Mapping (buildLedMap)

This is a crucial step where the relationship between the LEDs and specific regions of the screen content is established. Unlike simple pixel sampling, Hyperion maps each LED to a collection of pixels within a defined area.

• Component: HyperionProcessor::buildLedMap
• Mechanism:
  1. It first calculates the dimensions of the content area by subtracting the detected black border sizes from the overall image dimensions.
  2. For each LED defined in the LedLayout (e.g., bottom, right, top, left LEDs), it determines a corresponding rectangular region within this content area.
  3. It then iterates through all the pixels within that calculated region and stores their coordinates (x, y) in an internal _ledMap (a QVector<QVector<QPoint>>). Each inner QVector<QPoint> represents the set of pixels associated with a single LED.
• Rebuilding the Map: The _ledMap is rebuilt only when the input image dimensions change or when the detected black borders change. This optimizes performance by avoiding unnecessary recalculations.

3. Color Calculation (calculateLedColors)

Once the _ledMap is established, the processor calculates the final color for each LED based on the pixels mapped to it.

• Component: HyperionProcessor::calculateLedColors and HyperionProcessor::getAverageColor
• Mechanism:
  1. It iterates through each LED's entry in the _ledMap.
  2. For each LED, it retrieves all the associated pixel coordinates.
  3. It then calls getAverageColor to compute a single representative color for that LED.
• Current Color Algorithm (Mean Color): Currently, the getAverageColor function calculates the simple arithmetic mean of the Red, Green, and Blue components of all pixels within an LED's mapped region. This provides a smooth and representative color for the area.
  • Future Enhancements: Hyperion.ng offers other color calculation methods (e.g., mean squared, dominant color, k-means clustering) which could be implemented here for different visual effects.

What We Are Trying to Achieve Here (with grabberconfigurator)

The grabberconfigurator tool serves as a visual debugging and testing utility for this HyperionProcessor pipeline. Its purpose is to:

1. Verify Screen Capture and Scaling: Confirm that the WaylandGrabber and HyperionGrabber are correctly capturing and scaling the screen content.
2. Validate Black Border Detection: Visually inspect if the BlackBorderDetector is accurately identifying and cropping the black borders from the input image. The current visualization in grabberconfigurator shows the image after the black borders have been removed.
3. Test LED Mapping: Observe how the _ledMap is generated and how the LED regions are defined on the screen content. The visualization outlines these regions.
4. Evaluate Color Calculation: Ultimately, this tool will allow us to see the final calculated LED colors in real-time, helping us fine-tune the color calculation algorithms and LED layout for the best Ambilight experience.

By breaking down the complex process into these observable steps, we can systematically develop and refine our Hyperion-like image processing capabilities.