Multimodal Analytics
Computational frameworks for gesture, speech, and visual data in collaborative settings. Co-speech gesture detection and kinematic analysis from video.
Postdoctoral Researcher
Babajide
Alamu Owoyele
Data-driven, theory-guided researcher developing cartographic frameworks for mapping role constellations in sustainability transitions. Leveraging design science for multimodal data analysis across diverse sources, including gesture, speech, and audio-visual recordings.
Current Position
Postdoctoral Researcher
Centre for Language Studies, Radboud University
PI: Mark Dingemanse
Affiliate
Affiliate Researcher
AI & Intelligent Systems, Hasso Plattner Institute
PI: Gerard de Melo
Location
Nijmegen, Netherlands
Potsdam, Germany
Research Focus
Multimodal Analytics
Design Science · Sustainability Transitions
Research Areas
What I work on
Design Science Research
Creating innovative artifacts that bridge academic theory with practical solutions for complex social systems and sustainability challenges.
Sustainability Transitions
Computational methods to understand how actors and concepts enact social systems that accelerate or decelerate sustainability at scale.
AI Ethics & Privacy
Privacy-preserving tools for research data, including face and voice masking strategies that balance open science with personal data protection.
Community Building
Leading collaborative platforms that democratize access to multimodal research tools and foster international scientific communities.
Actor Role Cartography
Mapping role plurality and interconnectedness in multi-actor constellations using network analysis and computational text methods.
Selected Work
Projects
SYNAPSIS
NWO Funded · €250K
PI
2025–2027
Privacy-Preserving Audiovisual Data Sharing for Research
TDCC-SSH Challenge Project enabling researchers to share audiovisual data safely. Combines the MaskAnyone toolkit with SURF cloud infrastructure and researcher training to make privacy-preserving tools accessible across Dutch universities.
Capabilities
- Face & voice masking for GDPR compliance
- Body kinematics extraction
- SURF cloud integration
- Masking School training workshops
Impact
- 100+ trainees across Dutch universities
- 7 institutional partners
- DANS long-term data archiving
- Open science sustainability model
Privacy Tech
Open Science
Training
GDPR
Partners: VU Amsterdam, UvA, EUR, University of Groningen, Tilburg University, SURF, DANS
SMASH
NWO Funded · €50K
2024–2025
Synthesis and Multimodal Analytics System for Humanities
Fully open-source tool that analyzes non-verbal communication — body language, facial expressions, and speech patterns. Integrates privacy-preserving data archiving with behavioral analysis, making multimodal communication research accessible to non-programmers.
Capabilities
- Hand gesture analysis & diversity metrics
- Facial expression & posture tracking
- Voice tone variation analysis
- Automatic speech transcription
Impact
- Cross-cultural speaker databases
- Clinical research applications
- Social sciences & humanities tool
- Privacy-first data archiving
Multimodal AI
Computer Vision
NLP
Privacy Tech
Open Source
Collaboration: Radboud University, Hasso Plattner Institute · Co-investigators: Gerard de Melo, Wim Pouw · Radboud project page
EnvisionBOX
Core Team Lead
Ongoing
Community Platform for Multimodal Data Analysis
Community-driven platform for sharing multimodal data analysis solutions. Leading development of open-source tools that lower entry barriers for newcomers in multimodal signal processing.
Mission
- Increase research transparency & reproducibility
- Lower entry barriers for newcomers
- Foster international collaboration
- Democratize multimodal research tools
Community Impact
- Teaching modules & dynamic dashboards
- Full body tracking with MediaPipe
- International research collaboration
- Educational platform serving globally
Community
Open Science
Education
Platform
Team: Cognitive scientists, AI researchers, linguists across Europe · Partners: Tilburg University, University of Amsterdam, Leibniz Centre Berlin · envisionbox.org
Other Projects
EnvisionHGdetector Framework
2025Computational framework for co-speech gesture detection and kinematic analysis in multimodal communication research.
Python
Computer Vision
Machine Learning
MaskAnyone Toolkit
2024Privacy-preserving strategies for audio-visual data archiving that balance open science with personal data protection.
Privacy
Data Ethics
Research Tools
GitHub
MaskBench
2025Comprehensive benchmark framework for evaluating 2D pose estimators under privacy-preserving masking strategies including blurring, pixelation, and contour overlays. With Tim Riedel, Zainab Zafari, Sharjeel Shaik, Wim Pouw, Gerard de Melo.
Benchmarking
Computer Vision
Privacy
GitHub
InterPerDyn Pipeline
2025Open-source standardized pipeline for equitable observations of interactive behavioral dynamics. Processes video recordings through YOLOv8 pose estimation, signal processing, movement smoothness extraction, and non-linear dynamical systems analysis.
Python
Pose Estimation
Time Series
R
Pipeline notebook
Deep Narratives Africa (DNA)
StealthSocio-semantic analysis of development actor narratives across African institutions using natural language processing.
NLP
Text Mining
Social Analysis
Academic Output
Publications
EnvisionHGdetector: A Computational Framework for Co-Speech Gesture Detection, Kinematic Analysis, and Interactive Visualization
Preprint · 2025 · with Wim Pouw, Bosco Yung, Sharjeel Ahmed Shaikh
Computational framework for kinematic analysis of automatically detected co-speech gestures from single-person videos using TensorFlow and five open datasets.
MaskAnyone Toolkit: Offering Strategies for Minimizing Privacy Risks and Maximizing Utility in Audio-Visual Data Archiving
ResearchGate · 2024
Privacy-preserving toolkit for de-identifying individuals in video and audio while balancing open science with personal data protection.
Exploring Digital Health Solutions: Personalised Medicine and N-of-1 Trials in Ghana — A Scoping Review
Preprint · 2024 · with Godsway Sackey, Frank Kwesi Baiden, Stefan Konigorski
Scoping review of digital health applications and personalized medicine approaches in Ghana, examining N-of-1 trial implementations.
StudyU: A Platform for Designing and Conducting Innovative Digital N-of-1 Trials
Journal of Medical Internet Research · 2022 · DOI: 10.2196/35884
Open source platform for conducting personalized N-of-1 trials with innovative user-centric elements.
StudyMe: A New Mobile App for User-Centric N-of-1 Trials
PMC · 2022 · with Alexander M Zenner, Stefan Konigorski
Mobile application supporting non-experts in conducting multi-crossover self-experiments to evaluate intervention effects on personal health goals.
Exploring Ecosystem Orchestration Patterns and Systemic Intermediaries in Social Media
SSRN · 2022 · with Arash Hajikhani · DOI: 10.2139/ssrn.4042351
Analysis of social media ecosystems using digital trace data to map ecosystem orchestration and communication patterns in European Innovation Systems.
Designing for Value Creation: Principles, Methods, and Case Insights from Embedding Designing-as-Performance in Digital Health Education and Research
Springer Design Thinking Research · 2022 · DOI: 10.1007/978-3-031-09297-8_10
Exploration of value creation principles in healthcare through stakeholder engagement in academia and industry.
Designing as Performance: Bridging the Gap Between Research and Practice in Design Thinking Education
Springer Design Thinking Research · 2021 · DOI: 10.1007/978-3-030-62037-0_3
Research-based training materials for high-performing design teams, bridging design thinking research and practice in education.
Neurodesign Live
Springer Design Thinking Research · 2021 · DOI: 10.1007/978-3-030-62037-0_17
Cross-disciplinary exploration of synergies at the intersection of neuroscience, engineering, and design thinking in collaborative innovation.
Multimodal Mapping of Structure and Temporality of Institutional Logics in Multi-Actor Constellations
ResearchGate · Ongoing Research
Development of proto-ontology for socio-institutional analysis in sustainability transitions using computational literature review and machine learning.
Get in Touch
Contact
Open to research collaborations, speaking opportunities, and innovative projects in multimodal analytics and computational sustainability.
Radboud University
babajide.owoyele@ru.nlHasso Plattner Institute
babajide.owoyele@hpi.deProfiles