Overview
Gekkota is a piece of software which generates audio data using algorithms to determine the sound wave frames using linear algebra. It uses C/C++/HIP to harness the power of CPU's and GPU's to build and create electronic music from algorithms and patterns. The application features both a live performance interface and a recording/sequencing user interface.
The engine multiprocesses between foreground and background activities to create music. Incoming ui or midi events trigger algorithms. These algorithms influence the data used on the CPU and GPU to output sound to the ALSA driver. The application leverages platforms on Linux, such as Wayland, the compositor and window manager which is the successor to Xorg, ALSA which is the userland driver for the Linux kernel, Blend2D which is a high performace 2D vector graphics engine, and the AMD ROCM/HIP platform for processing the sound generation on the GPU.
Objectives
The objective of Gekkota is to be a system that can synthesize transitions in sound shapes to create unuque mosaic effects for music. Further uses in synthesizing audio from 3D models, or generating analysis plugins is possible, though not in active development.
Architecture
Gekkota is written in C/C++ using HIP for GPU processing in a Linux environment. The main application is broken up into subfolders to handling the inputs, outputs, and processing of the system. Within the Gekkota application, there are seperate threads for managing the ui/midi instruments or processing the sound data on the CPU/GPU. These threads call, listen, or call out to their respective devices and gather information.
The sound algorithms are packed into a packet structure so they can easily be shared as memory to the GPU for processing. Most of the arithmetic/vector calculations take place on the GPU in C code using the struct gka_entry arrays (see audio-segment.h for details). These segment arrays are configured to be navigated by both the CPU/GPU for ease of sharing in order to both compose sounds (CPU) and render them to frames (GPU).
GPU calculation stages
In order for sound to be calculated in parallel, the time sensitive parts must be considered, sounds can happen in absence of each other, but sound waves are periodic and thereby require a component of in series processing. Because of this, the GPU computation process happens in three stages (see hip-calculations.hpp for more details on how the kernels run).- Calculate the steps for a given frame. This happens as a frame per thread (at 192khz per second, for 1 1/100th of a second this is 1920 GPU threads). These are the steps of difference in the sine wave phase positions from the last time it was rendered to a frame. This changes with frequency becuase if a higher frequency is used the sine wave will be smaller.
- Calculate the phase of the sound for the given frame. This is dependent on the frames preceeding it, so this parallel processing is done one thread per sound, not one thread per frame.
- Generate frames and fold them all into one value per frame for the specified period. In this calculation, using the outcome of steps 1 and 2, we again calculate using one thread per frame.
In the below example, four sounds are used to generate frames, each frame represents one value in the PCM output. For example, in the output format 24bit 192khz, the final frame cells would be converted from doubles to be one of 192,000 24 bit integers that would play per second for each left or right audio channel.
| sounds | frames | ||||
|---|---|---|---|---|---|
| melody1 | 0.12 | 0.13 | 0.14 | 0.15 | ... |
| melody2 | 0.00 | 0.00 | 0.00 | 0.10 | ... |
| kick | -0.01 | -0.00 | 0.01 | 0.02 | ... |
| snare | 0.23 | 0.25 | 0.30 | 0.28 | ... |
| final frame | 0.34 | 0.38 | 0.45 | 0.55 | ... |
Current state
The present state of Gekkota is that the calculation and memory architectures are in place along with most of the base ui infrastructure. The engine successfully processes sounds according to the algorithms for volume and frequency, using the GPU to expedite the calculations. Visualization of the data is in place, but utilization of the events to propogate information is limited. MIDI divices are used directly as binary packets for playing live.
Roadmap
- Visualization of live segment patterns and audio data beyond a single period
- Shape transformations, using linear algebra to morph the output of the sine functions into weighted phase positions (particularly useful for percussion instuments or anything that sounds sharp).
- Segment memory management using custom free schedule
- Recording and persistence of sounds generated from MIDI instruments