Tools for Data Acquisition and Empirical Research

Signal Analysis Tool I

Audio Signal Analysis based on meyda.js library.

Signal Analysis Tool II

Audio Signal Analysis based on js-xtract.js library.

Spectral Features

Spectral Feature Analysis based on essentia.js library.

Frequency Bands

Spectral Bands Feature Analysis with essentia.js library.

Formant Estimator

Formant Estimation based on formantanalyser.js library.

Laplace Transformator

Formant Detection with Laplace Transformation.

Roughness Estimator

Gather the roughness of sounds using algorithms in the time and frequency domain.

Masking Score Estimator

Score for masking and timbral blending of a masker vs target sound.

Pitch & Tempo Analyser

Pitch, Chroma and BPM Analysis based on essentia.js library.

Pitch & Onset Tool

Pitch and Onsets analysis based on aubio.js library.

Audio File Analyser

Sound/Noise Analysis and making curves of audio features audible.

dB Analyser

Analysing dB SPL, dBA, dBC, Phon, Sone etc. from an audio file.

MIDI Analyser

MIDI file analyser including harmonic tension estimation.

Tempo Tapper

Find out BPM via tapping the space bar.

Harmonic Tension Analyser

Based on meyda.js for chroma calculation.

Clubbernaut Harmonic Tension Analyser

Pitchogram chroma calculation; works better than the previous version (recommended).

Recommendation

Room Acoustics Tool

Calculate the acoustic properties of a room according to ISO 3382.

Modulation Power House

Modulation Power Spectrum Analysis for detecting repeating patterns like vibrato, tremolo, rhythms, roughness, fluctuations, syllable rates etc.

Spectral Editor

Selectively edit spectral artifacts and remove background noise

Pitch Estimator

Pitch Estimation based on CREPE model and ml5.js library.

Violin Intonation (VIP)

Check intonation while playing (Tuning, Temperament, Timbre).

Firefox Only

Viola Intonation (VIP)

Check intonation while playing (Tuning, Temperament, Timbre).

Firefox Only

Cello Intonation (VIP)

Check intonation while playing (Tuning, Temperament, Timbre).

Firefox Only

Genre Estimator

Music genre estimation with MusiCNN model and essentia.js.

Mood Estimator

Emotional mood estimation with MusiCNN model and essentia.js.

Valence Arousal Estimator

Valence/Arousal estimation with MusiCNN model and essentia.js.

Speech Emotion Estimator

Emotions Estimation and voice features with WebSpeechAnalyser.js.

Gender/Speech Emotion Estimator

Gender and Emotions Estimation with transfomers.js and ONNX-Models.

Speech Emotion Classifier (CLAP)

AI based emotional speech analysis detects tension, nervousness, joy etc. in speech.

Extreme Metal Vocals (EMV)

Classification of the own Metal Voice.

Phoneme Recognizer

Detect and map individual phonemes (IPA) across a temporal timeline.

Sound Classifier

Detect and map individual sounds, noises and timbres across a temporal timeline.

Instrument & Ensemble Recognizer (CLAP)

Detect and map musical textures, genres, and specific instruments across a temporal timeline.

Image Batch Analyser

Extract information out of picture collections (AI based).

Bark Band Vocoder

Adjust sound spectra to each other and/or vocode signals.

Spectral Resynthesizer

Select the strongest partials of a sound and resynthesize it.

Spectral Cross-Synthesizer

Apply the timbre of one (target) sound to another (source) sound.

Modulation Transfer

Analyse vibrato, tremolo and micro modulations and transfer it to other sounds.

Harmonic/Percussive Separator

Calculate and download the harmonic and percussive parts of a spectrum.

Harmonic Frequency Extractor

Isolates the F0 Frequencies and its multiples (harmonics) of a sound.

Harmonicity Analyser

Separate harmonic and noisy parts and calculate the Harmonicity (HNR) of an audio file.

Time Stretcher

Extend the length of an audio file without artifacts even over long durations.

Formant Shifter

Change voices and instruments sounds by shifting and reshaping their spectral envelope.

Pitch & Formant Shifter

Shift pitch and formants independently while keeping the duration of an audio file.

Spectral Gator

Keep/remove partials or spectral frequencies depending on their amplitudes.

Spectral Stamper

Apply the spectral envelope of an audio file as a dynamic 3D filter to another audio file.

Spectral Morpher

Audio morphing with time warping, pitch tracking and formant correction.

Spectral Warper

Shift partials from harmonic to inharmonic relationships.

Squeeze & Stretch Partials

Turn consonances into dissonances and vice versa.

Tempodrom

Multi-Band Rhythm Lab to extract BPMs and resynthesize Groove Patterns according to Human Perception.

Emotion Analysis

Including Mindfield Esense SCR / Temperature / Respiration Measurement and Audio Signal Analysis.

Facial Expression Analysis (recommended)

Frame by Frame Video Facial Expression Analysis.

Face Expression Tracker with Audio Analysis

Video Face and Expression Tracking incl. Audio Signal Analysis.

Facial Muscle Tracking

Frame by Frame Video Facial Muscle and Expression Analysis.

Heartbeat Tracker

Heartbeat, mood and age tracking via Webcam.

2D Pose Tracker (MoveNet)

Frame by Frame 2D Video Pose Tracking based on MoveNet Model.

2D/3D Pose Tracker (recommended)

Frame by Frame Video Pose Tracking based on BlazePose Model.

2D/3D Pose Tracker with Audio Analysis

Video Pose Tracking including meyda Audio Signal Analysis.

Chrome Only

2D/3D Hand Tracker (recommended)

Frame by Frame Video Hand Tracking based on MediaPipe Model.

2D/3D Hand Tracker with Audio Analysis

Video Hand Tracking including Audio Signal Analysis.

Chrome Only

2D Face/Body/Hand Tracker

Frame by Frame Body, Hands, Face, Gaze, Age and Emotion tracking with Vladmandic human.js library.

Point Tracker

Track up to 8 points of interest in videos, based on OpenCV.js with Kanade-Lucas-Tomasi Feature Tracker (KLT).

3D Pose Tracking Visualiser (Avatar)

Visualise results from BlazePose Pose Tracking tools with 3D Avatar and Movement Analysis.

2D Pose/Hand/Face-Tracking Visualizer

Visualise results from Human.js Tracking Tools with 2D Stickfigure and Movement Analysis.

3D Pose Tracking Visualiser (Stick Figure)

Visualise results from BlazePose Pose Tracking tools with Audio Features Correlation.

3D Hand Tracking Visualizer (Stick Hands)

For the results of the 2D/3D Hand Tracker with Audio Features Correlation.

Columns Correlator

Find correlations in Excel data.

Time Series Analyser

Analyse time series like audio data, physiological data and/or motion data for correlation.

t-Tester

Find differences in Excel data.

Interrater Calculator

Calculate Fleiss' Kappa from Excel or .csv files.

Similarity Sorter

Sort Excel rows according to the similarity of one row.

Excel Sorter

Sort table rows and columns according to certain patterns.

Z-Transfomer

Generate z-transformed values of each column.

Moving Averager

Run a moving average with adjustable window size.

Excel Synchroniser

Synchronise Excel and .csv tables with different lengths.

Gaussian Mixture Model

Run independent GMMs on two columns and visualise results.

Excel ClusterBuster

Run K-Mean Cluster Analysis incl. Elbow method.

SupANOVA

Run ANOVA and PCA to find patterns in multidimensional data.

Excel Visualiser

Generate charts and plots for Excel or .csv data visualisation.

Excel Boxplotter

Generate box/violin plots or radar charts for Excel or .csv data.

Excel Categorizer

Generate Progression Charts over several categories (points of time, situations, conditions) from Excel or .csv tables

Heatmapper

Generate heatmaps from Excel or .csv tables.

Excel to JS Array

Get Javascript arrays out of Excel and .csv files.

Movie to MP3/MP4

Convert mov, avi, mpg, etc. to audio (mp3) or video (mp4) files.

Batch Audio Channel Separator

Upload audio files - stereo or multi-channel -, separate their channels and download them as individual *.wav files.

Audio Recorder

Record audio examples incl. calibration support and download them as *.wav files.

Sound Segmenter

AI Sample Slicer - Segment and slice audio samples by events and pitches and download them as individual *.wav files.

Perceptual Color Comparator

Calculating human perception of colour similarity with delta ΔE .

Muse 2 EEG Datalogger

4 Channel EEG Recording for Muse S Head Band incl. Brain Synthesizer.

Chrome Only

Eye Tracker

Heatmaps and Gaze Paths from Eye Movements.

Will this Mosquito Bite me?

Find out from the buzzing whether the mosquito is going to bite.

Dynamic Timbre Maps

Formants of Orchestral Instruments in Motion.

MAD - Motion & Audio Feature Display

Conference contribution
Reuter, C., Bertsch, M., Czedik-Eysenberg, I., Cui, A.-X., Roos, M., Ambros, S., Jewanski, J., Eder, M., Mühlhans, J., Klooss, F., Popovic, D., Weber, V., Stickler, A., Feller, G., Steurer, P., Yang, J., Oehler, M. (2024). Grooving & Moving, Blowing & Bowing - Typische Bewegungen beim Saxophonspiel und ihre klanglichen Auswirkungen. Body, Mind & Music, 11.-13.4.2024. Österreichische Gesellschaft für Musik und Medizin (ÖGfMM), Musikphysiologie der mdw - Universität für Musik und darstellende Kunst. Wien.

MoCap vs. PoseTracking vs. RITMO Video Analysis

Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.

Sound Radiation and other Audio Features of the Guitar

Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.

SHAPED - Sound radiation, Hand, Anatomy, Pitch & Expression Display

Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.

HandBoX - Hand Body eXpression Visualizer (2D)

Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.

(E)MOTiVATe - (E)Motion Over Time Valence Arousal Template

Conference contribution
Kilicci, J., Reuter, C. (2024). "The tingling sensation on my skin". Posterbeitrag, 40. Jahrestagung der Deutschen Gesellschaft für Musikpsychologie (DGM), 6.-8.9.2024, Hochschule für Musik und Theater München.

Music/video synchronous step visualisation of dancers (Chachacha)

Conference contribution
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.

Music/video synchronous step visualisation of dancers (Boogie)

Conference contribution
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.