Audio Signal Analysis
Signal Analysis Tool I
Audio Signal Analysis based on meyda.js library.
Signal Analysis Tool II
Audio Signal Analysis based on js-xtract.js library.
Spectral Features
Spectral Feature Analysis based on essentia.js library.
Frequency Bands
Spectral Bands Feature Analysis with essentia.js library.
Formant Estimator
Formant Estimation based on formantanalyser.js library.
Laplace Transformator
Formant Detection via Laplace Transformation.
Roughness Estimator
Gather the roughness of sounds using algorithms in the time and frequency domain.
Modulation Power House
Modulation Power Spectrum Analysis for detecting repeating patterns like vibrato, tremolo, rhythms, roughness, fluctuations, syllable rates etc.
Pitch & Tempo Analyser
Polyphonic Pitch, Chords and BPM Analysis based on essentia.js
library.
Pitch & Onset Tool
Pitch, BPM and Onsets analysis based on aubio.js library.
Clubbernaut Harmonic Tension Analyser
Harmonic Tension Analysis based on clubber.js.
recommended version
Harmonic Tension Analyser
Based on meyda.js for chroma calculation.
depracted
Tempodrom
Multi-Band Rhythm Analysis to extract BPMs and resynthesize Groove Patterns according to Human Perception.
Tempo Tapper
Find out BPM via tapping the space bar.
dB Analyser
Analysing dB SPL, dBA, dBC, Phon, Sone etc. from an audio file.
Room Acoustics Tool
Calculate the acoustic properties of a room according to ISO 3382.
Spectral Editor
Selectively edit spectral artifacts and remove background noise
Audio File Analyser
Sound/Noise Analysis and making curves of audio features audible.
Masking Score Estimator
Score for masking and timbral blending of a masker vs target sound.
MIDI Analyser
MIDI file analyser including harmonic tension estimation.
AI Based Estimations
Pitch Estimator
Pitch Estimation based on CREPE model and ml5.js library.
Violin Intonation (VIP)
Check intonation while playing (Tuning, Temperament, Timbre).
Viola Intonation (VIP)
Check intonation while playing (Tuning, Temperament, Timbre).
Cello Intonation (VIP)
Check intonation while playing (Tuning, Temperament, Timbre).
Genre Estimator
Music genre estimation with MusiCNN model and essentia.js.
Mood Estimator
Emotional mood estimation with MusiCNN model and essentia.js.
Valence Arousal Estimator
Valence/Arousal estimation with MusiCNN model and essentia.js.
Speech Emotion Estimator
Emotions Estimation and voice features with WebSpeechAnalyser.js.
Gender/Speech Emotion Estimator
Gender and Emotions Estimation with transfomers.js and ONNX-Models.
Speech Emotion Classifier (CLAP)
AI based emotional speech analysis detects tension, nervousness, joy etc. in speech.
Extreme Metal Vocals (EMV)
Classification of the own Metal Voice.
Phoneme Recognizer
Detect and map individual phonemes (IPA) across a temporal timeline.
Sound Classifier
Detect and map individual sounds, noises and timbres across a temporal timeline.
Instrument & Ensemble Recognizer (CLAP)
Detect and map musical textures, genres, and specific instruments across a temporal timeline.
Image Batch Analyser
Extract information out of picture collections (AI based).
Audio Synthesis
Bark Band Vocoder
Adjust sound spectra to each other and/or vocode signals.
Spectral Resynthesizer
Select the strongest partials of a sound and resynthesize it.
Spectral Cross-Synthesizer
Apply the timbre of one (target) sound to another (source) sound.
Modulation Transfer
Analyse vibrato, tremolo and micro modulations and transfer it to other sounds.
Harmonic/Percussive Separator
Calculate and download the harmonic and percussive parts of a spectrum.
Harmonic Frequency Extractor
Isolates the F0 Frequencies and its multiples (harmonics) of a sound.
Harmonicity Analyser
Separate harmonic and noisy parts and calculate the Harmonicity (HNR) of an audio file.
Time Stretcher
Extend the length of an audio file without artifacts even over long durations.
Formant Shifter
Change voices and instruments sounds by shifting and reshaping their spectral envelope.
Pitch & Formant Shifter
Shift pitch and formants independently while keeping the duration of an audio file.
Spectral Gator
Keep/remove partials or spectral frequencies depending on their amplitudes.
Spectral Stamper
Apply the spectral envelope of an audio file as a dynamic 3D filter to another audio file.
Spectral Morpher
Audio morphing with time warping, pitch tracking and formant correction.
Spectral Warper
Shift partials from harmonic to inharmonic relationships.
Squeeze & Stretch Partials
Turn consonances into dissonances and vice versa.
Body, Hands, Point & Emotion Tracking
Emotion Analysis
Including Mindfield Esense SCR / Temperature / Respiration Measurement and Audio Signal Analysis.
Facial Expression Analysis
Frame by Frame Video Facial Expression Analysis.
recommended version
Face Expression Tracker with Audio Analysis
Video Face and Expression Tracking incl. Audio Signal Analysis.
depracted
Facial Muscle Tracking
Frame by Frame Video Facial Muscle and Expression Analysis.
Heartbeat Tracker
Heartbeat, mood and age tracking via Webcam.
2D Multiple Person Pose Tracker (MoveNet)
Frame by Frame 2D Video Pose Tracking for up to 6 persons based on MoveNet Model.
2D/3D Pose Tracker
Frame by Frame Video Pose Tracking based on BlazePose Model.
recommended version
2D/3D Pose Tracker with Audio Analysis
Video Pose Tracking including meyda Audio Signal Analysis.
depracted
2D/3D Hand Tracker
Frame by Frame Video Hand Tracking based on MediaPipe Model.
recommended version
2D/3D Hand Tracker with Audio Analysis
Video Hand Tracking including Audio Signal Analysis.
depracted
2D Face/Body/Hand Tracker
Frame by Frame Body, Hands, Face, Gaze, Age and Emotion tracking with Vladmandic human.js library.
Point Tracker
Track up to 8 points of interest in videos, based on OpenCV.js with Kanade-Lucas-Tomasi Feature Tracker (KLT).
Video Motion Analysis Tool
Track Quantity of Motion, Motion Centroid, Motion Periodicity, and Optical Flow based on OpenCV.js and RITMO Video Analysis (Jensenius 2005).
Video Motion Amplification Tool
Magnify subtle sub-visual movements and color differences in videos (for modal analysis, human pulse visualization etc.).
Tracking Visualizers
3D Data Analyzer
Visualising/Analysing/Editing 3D Qualisys MoCap, BlazePose (Body) and MediaPipe (Hands) Data (including motion transfer to 3D avatars).
2D Data Analyzer
Visualising/Analysing/Editing 2D MoveNet, BlazePose
(Body) MediaPipe (Hands) face-api (Face) and PointTracking Data (for up
to 6 Persons, including openPose export and FaceMesh Warping)
3D Pose Tracking Visualiser (Avatar)
Visualise results from BlazePose Pose Tracking tools with 3D Avatar and Movement Analysis.
2D Pose/Hand/Face-Tracking Visualizer
Visualise results from Human.js Tracking Tools with 2D Stickfigure and Movement Analysis.
3D Pose Tracking Visualiser (Stick Figure)
Visualise results from BlazePose Pose Tracking tools with Audio Features Correlation.
3D Hand Tracking Visualizer (Stick Hands)
For the results of the 2D/3D Hand Tracker with Audio Features Correlation.
Helpers (Statistical & Excel Tools)
Columns Correlator
Find correlations in Excel data.
Time Series Analyser
Analyse time series like audio data, physiological data and/or motion data for correlation.
t-Tester
Find differences in Excel data.
Interrater Calculator
Calculate Fleiss' Kappa from Excel or .csv files.
Similarity Sorter
Sort Excel rows according to the similarity of one row.
Excel Sorter
Sort table rows and columns according to certain patterns.
Z-Transfomer
Generate z-transformed values of each column.
Moving Averager
Run a moving average with adjustable window size.
Excel Synchroniser
Synchronise Excel and .csv tables with different lengths.
Gaussian Mixture Model
Run independent GMMs on two columns and visualise results.
Excel ClusterBuster
Run K-Mean Cluster Analysis incl. Elbow method.
SupANOVA
Run ANOVA and PCA to find patterns in multidimensional data.
Excel Visualiser
Generate charts and plots for Excel or .csv data visualisation.
Excel Boxplotter
Generate box/violin plots or radar charts for Excel or .csv data.
Progression Charter
Generate Progression Charts over several categories (points of time, situations, conditions) from Excel or .csv tables
Performance Wormhole
Generate Performance Worms from tempo and dynamics data (acc. to Dixon et al. 2002) as well as from other audio features.
Heatmapper
Generate heatmaps from Excel or .csv tables.
CSV Formatter
CSV/Excel Format Converter, convert Excel and csv files between European and American formats
Excel to JS Array
Get Javascript arrays out of Excel and .csv files.
Movie to MP3/MP4
Convert mov, avi, mpg, etc. to audio (mp3) or video (mp4) files.
Videorecorder and Editor
Record videos from webcam/screen and/or upload videos, trim, crop and download them.
Audio Recorder
Record audio examples incl. calibration support and download them as *.wav files.
Batch Audio Channel Separator
Upload audio files - stereo or multi-channel -, separate their channels and download them as individual *.wav files.
Batch Audio Loudness Normalizer
Upload audio files, align their perceptual loudness individually, or match and maximize them as a stimuli group for listening experiments.
Sound Segmenter
AI Sample Slicer - Segment and slice audio samples by events and pitches and download them as individual *.wav files.
Audio Thumbnailer
Create Audio Thumbnails, Timbral Prototypes and Musical Structure Elements using Self-Similarity Matrices according to Paulus/Klapuri 2009.
Perceptual Color Comparator
Calculating human perception of colour similarity with delta ΔE .
Further Tools
Muse 2 EEG Datalogger
4 Channel EEG Recording for Muse S Head Band incl. Brain Synthesizer.
Chrome Only
Eye Tracker
Heatmaps and Gaze Paths from Eye Movements.
Will this Mosquito Bite me?
Find out from the buzzing whether the mosquito is going to bite.
Dynamic Timbre Maps
Formants of Orchestral Instruments in Motion.
Selected Conference Demos
MAD - Motion & Audio Feature Display
Conference contribution
Reuter, C., Bertsch, M., Czedik-Eysenberg, I., Cui, A.-X., Roos, M., Ambros, S., Jewanski, J., Eder, M., Mühlhans, J., Klooss, F., Popovic, D., Weber, V., Stickler, A., Feller, G., Steurer, P., Yang, J., Oehler, M. (2024). Grooving & Moving, Blowing & Bowing - Typische Bewegungen beim Saxophonspiel und ihre klanglichen Auswirkungen. Body, Mind & Music, 11.-13.4.2024. Österreichische Gesellschaft für Musik und Medizin (ÖGfMM), Musikphysiologie der mdw - Universität für Musik und darstellende Kunst. Wien.
Reuter, C., Bertsch, M., Czedik-Eysenberg, I., Cui, A.-X., Roos, M., Ambros, S., Jewanski, J., Eder, M., Mühlhans, J., Klooss, F., Popovic, D., Weber, V., Stickler, A., Feller, G., Steurer, P., Yang, J., Oehler, M. (2024). Grooving & Moving, Blowing & Bowing - Typische Bewegungen beim Saxophonspiel und ihre klanglichen Auswirkungen. Body, Mind & Music, 11.-13.4.2024. Österreichische Gesellschaft für Musik und Medizin (ÖGfMM), Musikphysiologie der mdw - Universität für Musik und darstellende Kunst. Wien.
MoCap vs. PoseTracking vs. RITMO Video Analysis
Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
Sound Radiation and other Audio Features of the Guitar
Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
SHAPED - Sound radiation, Hand, Anatomy, Pitch & Expression Display
Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
HandBoX - Hand Body eXpression Visualizer (2D)
Conference contribution
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
Reuter, C. (2024). Audiosignalanalyse, KI und die 96 Ohren der akustischen Kamera. Summerschool im Musikinstrumentenbau (SUMU), Westsächsischen Hochschule Zwickau, Standort Markneukirchen.
(E)MOTiVATe - (E)Motion Over Time Valence Arousal Template
Conference contribution
Kilicci, J., Reuter, C. (2024). "The tingling sensation on my skin". Posterbeitrag, 40. Jahrestagung der Deutschen Gesellschaft für Musikpsychologie (DGM), 6.-8.9.2024, Hochschule für Musik und Theater München.
Kilicci, J., Reuter, C. (2024). "The tingling sensation on my skin". Posterbeitrag, 40. Jahrestagung der Deutschen Gesellschaft für Musikpsychologie (DGM), 6.-8.9.2024, Hochschule für Musik und Theater München.
Music/video synchronous step visualisation of dancers (Chachacha)
Conference contribution
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.
Music/video synchronous step visualisation of dancers (Boogie)
Conference contribution
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.
Reuter, C. (2024). Data Science with Tools from SInES. Science Day Motion Capture: Exploring Bodies in Motion. Campus: Aula, University of Vienna.
How to cite SInES Tools?
When citing a single tool, the citations should follow the pattern below. Author(s) and year are always listed at the bottom of the application. For example:
- Reuter, C. (2021). SInES Tools: Audio Signal Analysis Tool I. Web Application https://sinestools.univie.ac.at/meyda_signalanalyse.htm
If you are referring to the SInES Tools as a whole the citation should be:
- Reuter, C., Czedik-Eysenberg, I., Cui, A.-X., Ambros, S.,
Klooss, F., Feller, G., Schachner, I., Roos, M., Lindenbauer, H., Li, H.,
& Bertsch, M. (2026). SInES
Tools, Helpers, Visualizers as your Scientific Realizers. DAGA 2026,
52nd Annual Conference of the German Acoustical Society (pp. 1002–1005).
Dresden.