Heiko Hübert
Photo: SCIoI
Heiko Hübert is a professor at HTW Berlin since 2015, specializing in embedded systems and with a focus on health electronics. At SCIoI, he collaborated in Project 6 and contributed by creating an open source robotic framework, Sobotify, which integrates AI-/ML-based tools in social robots in education. Currently, he continues his collaboration with SCIoI in the Integration B2 project, extending Sobotify by integrating the findings from the SCIoI projects to demonstrate social intelligence in robots. His research interests include open source platforms for social robots and socially assistive robotics (SAR), also for applications in elderly care and health.
Projects
Heiko Hübert is member of:
Publications +
6984777
Hübert
1
apa
50
date
desc
year
20976
https://www.scienceofintelligence.de/wp-content/plugins/zotpress/
%7B%22status%22%3A%22success%22%2C%22updateneeded%22%3Afalse%2C%22instance%22%3Afalse%2C%22meta%22%3A%7B%22request_last%22%3A0%2C%22request_next%22%3A0%2C%22used_cache%22%3Atrue%7D%2C%22data%22%3A%5B%7B%22key%22%3A%224QMWBBYK%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22H%5Cu00fcbert%20and%20Yun%22%2C%22parsedDate%22%3A%222024%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BH%26%23xFC%3Bbert%2C%20H.%2C%20%26amp%3B%20Yun%2C%20H.%20S.%20%282024%29.%20Sobotify%3A%20A%20Framework%20for%20Turning%20Robots%20into%20Social%20Robots.%20%26lt%3Bi%26gt%3BProceedings%20of%20the%202024%20ACM%5C%2FIEEE%20International%20Conference%20on%20Human-Robot%20Interaction%2C%20HRI%20%26%23x2019%3B24%26lt%3B%5C%2Fi%26gt%3B%2C%20910%26%23x2013%3B914.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1145%5C%2F3610977.3637482%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1145%5C%2F3610977.3637482%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Sobotify%3A%20A%20Framework%20for%20Turning%20Robots%20into%20Social%20Robots%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%5D%2C%22abstractNote%22%3A%22Sobotify%20is%20a%20software%20framework%2C%20which%20aims%20at%20simplifying%20the%20process%20of%20using%20robots%20in%20the%20field%20of%20social%20robotics.%20This%20paper%20delineates%20the%20design%20and%20usage%20of%20the%20framework.%20With%20Sobotify%2C%20even%20non-technical%20people%20should%20be%20enabled%20to%20use%20robots%20for%20their%20specific%20purposes%2C%20such%20as%20teachers%20in%20a%20classroom%2C%20therapists%20in%20a%20physiological%20or%20psychological%20therapy%20or%20childcare%20workers%20in%20kindergarten.%20During%20the%20development%20process%20of%20Sobotify%2C%20feedback%20from%20teachers%20at%20a%20vocational%20school%20have%20been%20taken%20into%20account%20in%20order%20to%20adjust%20the%20tools%20to%20their%20needs.%20The%20framework%20is%20designed%20to%20work%20with%20different%20robots%20including%20both%20humanoid%20%28NAO%20and%20Pepper%29%20and%20non-humanoid%20robots%20such%20as%20toy%20robots%20%28Cozmo%29%20with%20advanced%20abilities%20as%20well%20as%20very%20simple%20toy%20robots%20%28MyKeepon%29.%20The%20framework%20was%20tested%20by%20two%20research%20works%20which%20proved%20that%20Sobotify%20is%20applicable%20in%20different%20setups.%20Further%20development%20is%20already%20planned%20for%20the%20next%20months%2C%20e.g.%20integration%20of%20additional%20robots%20and%20extension%20of%20tools.%22%2C%22proceedingsTitle%22%3A%22Proceedings%20of%20the%202024%20ACM%5C%2FIEEE%20International%20Conference%20on%20Human-Robot%20Interaction%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%22M%5Cu00e4rz%2011%2C%202024%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.1145%5C%2F3610977.3637482%22%2C%22ISBN%22%3A%22979-8-4007-0322-5%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22https%3A%5C%2F%5C%2Fdl.acm.org%5C%2Fdoi%5C%2F10.1145%5C%2F3610977.3637482%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A00%3A35Z%22%7D%7D%2C%7B%22key%22%3A%22PWB8NFLU%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222024%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Pinkwart%2C%20N.%2C%20%26amp%3B%20Hafner%2C%20V.%20V.%20%282024%29.%20Design%20Based%20Research%20of%26%23xA0%3BMultimodal%20Robotic%20Learning%20Companions.%20In%20A.%20M.%20Olney%2C%20I.-A.%20Chounta%2C%20Z.%20Liu%2C%20O.%20C.%20Santos%2C%20%26amp%3B%20I.%20I.%20Bittencourt%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BArtificial%20Intelligence%20in%20Education%20%28AIED%202024%29%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%2097%26%23x2013%3B104%29.%20Springer%20Nature%20Switzerland.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-64312-5_12%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-64312-5_12%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Design%20Based%20Research%20of%5Cu00a0Multimodal%20Robotic%20Learning%20Companions%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Niels%22%2C%22lastName%22%3A%22Pinkwart%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Verena%20V.%22%2C%22lastName%22%3A%22Hafner%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Andrew%20M.%22%2C%22lastName%22%3A%22Olney%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Irene-Angelica%22%2C%22lastName%22%3A%22Chounta%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Zitao%22%2C%22lastName%22%3A%22Liu%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Olga%20C.%22%2C%22lastName%22%3A%22Santos%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Ig%20Ibert%22%2C%22lastName%22%3A%22Bittencourt%22%7D%5D%2C%22abstractNote%22%3A%22This%20paper%20presents%20some%20initial%20findings%20from%20a%20design-based%20research%20study%20on%20multimodal%20robotic%20language%20learning%20companions.%20The%20study%20involved%20designing%20four%20robots%20with%20varying%20physical%20modalities%2C%20including%20two%20humanoid%20%28Pepper%20and%20Nao%29%20and%20two%20non-humanoid%20%28Cozmo%20and%20MyKeepOn%29%2C%20to%20interact%20empathetically%20with%20participants%20during%20language%20learning%20scenarios.%20One%20German%20teacher%20participated%20in%20the%20development%20of%20these%20interactive%20scenarios.%20Pre-%20and%20post-tests%20were%20conducted%20to%20measure%20learning%20gains%2C%20and%20a%20survey%20was%20administered%20to%20gather%20insights%20from%20participants.%20The%20results%20showed%20that%20all%20participants%20exhibited%20improvements%20in%20targeted%20behaviors%20between%20pre-%20and%20post-tests%2C%20specifically%20by%20using%20longer%20and%20more%20grammatically%20correct%20sentences%20in%20their%20responses.%20However%2C%20unlike%20the%20strong%20preference%20for%20NAO%2C%20the%20survey%20revealed%20that%20Cozmo%20had%20the%20most%20characteristics%20of%20a%20learning%20companion.%22%2C%22proceedingsTitle%22%3A%22Artificial%20Intelligence%20in%20Education%20%28AIED%202024%29%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222024%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.1007%5C%2F978-3-031-64312-5_12%22%2C%22ISBN%22%3A%22978-3-031-64312-5%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A01%3A23Z%22%7D%7D%2C%7B%22key%22%3A%22IFC4RRS5%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222023%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Sardogan%2C%20A.%2C%20Pinkwart%2C%20N.%2C%20Hafner%2C%20V.%20V.%2C%20%26amp%3B%20Lazarides%2C%20R.%20%282023%29.%20%26lt%3Bi%26gt%3BHumanoid%20Robot%20as%20a%20Debate%20Partner%26lt%3B%5C%2Fi%26gt%3B.%20HCI%20International%202023%20Posters.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-36004-6_74%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-36004-6_74%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22presentation%22%2C%22title%22%3A%22Humanoid%20Robot%20as%20a%20Debate%20Partner%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Abdullah%22%2C%22lastName%22%3A%22Sardogan%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Niels%22%2C%22lastName%22%3A%22Pinkwart%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Verena%20V.%22%2C%22lastName%22%3A%22Hafner%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Rebecca%22%2C%22lastName%22%3A%22Lazarides%22%7D%2C%7B%22creatorType%22%3A%22contributor%22%2C%22firstName%22%3A%22Constantine%22%2C%22lastName%22%3A%22Stephanidis%22%7D%2C%7B%22creatorType%22%3A%22contributor%22%2C%22firstName%22%3A%22Margherita%22%2C%22lastName%22%3A%22Antona%22%7D%2C%7B%22creatorType%22%3A%22contributor%22%2C%22firstName%22%3A%22Stavroula%22%2C%22lastName%22%3A%22Ntoa%22%7D%2C%7B%22creatorType%22%3A%22contributor%22%2C%22firstName%22%3A%22Gavriel%22%2C%22lastName%22%3A%22Salvendy%22%7D%5D%2C%22abstractNote%22%3A%22In%20this%20paper%2C%20we%20describe%20our%20design%20and%20development%20process%20of%20a%20humanoid%20robot%2C%20Pepper%2C%20as%20a%20debate%20partner%20by%20integrating%20open-source%2C%20offline%20Generative%20Pre-trained%20Transformer%20models%2C%20namely%20GPT-J%206B%20and%20BLOOM.%20We%20used%20our%20tool%20flow%2C%20which%20integrates%20open-source%20offline%20tools%20to%20animate%20a%20robotic%20agent%20to%20listen%2C%20speak%20and%20gesture%20in%20a%20human-like%20manner%2C%20to%20create%20a%20debate%20partner%20in%20a%20humanoid%20robot.%20This%20paper%20provides%20a%20technical%20description%20of%20the%20integration%20of%20GPT%20models%20into%20our%20tool%20flow.%20The%20first%20implementation%20was%20presented%20to%20a%20group%20of%20teachers%20and%20as%20a%20second%20prototype%2C%20several%20questioning%20schemes%20and%20post%20filtering%20were%20implemented%20to%20attain%20better%20responses%20from%20a%20robotic%20agents.%20The%20developed%20second%20prototype%20will%20be%20used%20for%20a%20pilot%20study%20with%20prospective%20users.%22%2C%22date%22%3A%222023%22%2C%22sessionTitle%22%3A%22%22%2C%22DOI%22%3A%2210.1007%5C%2F978-3-031-36004-6_74%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A00%3A28Z%22%7D%7D%2C%7B%22key%22%3A%22HRMQHITI%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222023%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Cheval%26%23xE8%3Bre%2C%20J.%2C%20Pinkwart%2C%20N.%2C%20Hafner%2C%20V.%20V.%2C%20%26amp%3B%20Lazarides%2C%20R.%20%282023%29.%20Analyzing%20Learners%26%23x2019%3B%20Emotion%20from%26%23xA0%3Ban%26%23xA0%3BHRI%20Experiment%20Using%20Facial%20Expression%20Recognition%20Systems.%20In%20P.%20Zaphiris%20%26amp%3B%20A.%20Ioannou%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BLearning%20and%20Collaboration%20Technologies%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%20396%26%23x2013%3B407%29.%20Springer%20Nature%20Switzerland.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-34550-0_29%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-34550-0_29%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22Analyzing%20Learners%5Cu2019%20Emotion%20from%5Cu00a0an%5Cu00a0HRI%20Experiment%20Using%20Facial%20Expression%20Recognition%20Systems%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Johann%22%2C%22lastName%22%3A%22Cheval%5Cu00e8re%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Niels%22%2C%22lastName%22%3A%22Pinkwart%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Verena%20V.%22%2C%22lastName%22%3A%22Hafner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Rebecca%22%2C%22lastName%22%3A%22Lazarides%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Panayiotis%22%2C%22lastName%22%3A%22Zaphiris%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Andri%22%2C%22lastName%22%3A%22Ioannou%22%7D%5D%2C%22abstractNote%22%3A%22In%20this%20study%2C%20we%20analyzed%20the%20facial%20expressions%20of%20the%20participants%20from%20a%20study%20where%2063%20students%20were%20learning%20about%20Climate%20Change%20using%20Betty%5Cu2019s%20Brain%2C%20half%20with%20robotic%20agents%20and%20the%20other%20half%20with%20on-screen%20based%20agents%20using%20a%20Facial%20Expression%20Recognition%20%28FER%29%20system.%20Among%20FER%20systems%2C%20upon%20a%20review%20of%20existing%20offline%20open%20source%20solutions%2C%20we%20chose%20HyperExtended%20LightFace%20%28more%20known%20as%20deepface%29%20to%20extract%20emotions%20from%20the%20participants%5Cu2019%20facial%20expressions.%20Based%20on%20the%20extracted%20emotions%20from%20the%20FER%20system%2C%20we%20compared%20the%20resulting%20emotions%20between%20two%20groups%20to%20investigate%20if%20there%20are%20differences%20in%20displayed%20emotions%20due%20to%20human%20robot%20interaction.%20The%20result%20of%20this%20first%20analysis%20shows%20that%20learners%20who%20interacted%20with%20robotic%20agents%20express%20fear%2C%20happy%20and%20neutral%20emotions%20more%20than%20the%20students%20who%20interacted%20with%20on-screen%20based%20agents.%20In%20addition%2C%20neutral%20emotion%20was%20detected%20more%20in%20the%20robotic%20condition%20with%20a%20large%20effect%2C%20fear%20with%20moderate%20effect%20and%20happy%20with%20small%20effect%20size.%20Fear%20and%20happiness%20was%2060%25%20correlated%2C%20which%20indicates%20that%20the%20FER%20system%20may%20not%20distinguish%20well%20between%20these%20two%20emotions.%20The%20future%20work%20entails%20filtering%20and%20validation%20of%20face%20detection%20results%20in%20case%20of%20false%20detection%20and%20utilizing%20different%20pre-processing%20methods%20to%20detect%20the%20face%20and%20using%20different%20FER%20algorithms.%22%2C%22proceedingsTitle%22%3A%22Learning%20and%20Collaboration%20Technologies%22%2C%22conferenceName%22%3A%22%22%2C%22date%22%3A%222023%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.1007%5C%2F978-3-031-34550-0_29%22%2C%22ISBN%22%3A%22978-3-031-34550-0%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T12%3A00%3A21Z%22%7D%7D%2C%7B%22key%22%3A%22VRSKEESF%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222022-05%22%2C%22numChildren%22%3A0%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Taliarona%2C%20V.%2C%20%26amp%3B%20Sardogan%2C%20A.%20%282022%2C%20May%29.%20%26lt%3Bi%26gt%3BUtilizing%20Machine%20Learning%20based%20Gesture%20Recognition%20Software%2C%20Mediapipe%2C%20in%20the%20Context%20of%20Education%20and%20Health%26lt%3B%5C%2Fi%26gt%3B%20%5BPoster%5D.%20AI%20Innovation%20Summit.%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22presentation%22%2C%22title%22%3A%22Utilizing%20Machine%20Learning%20based%20Gesture%20Recognition%20Software%2C%20Mediapipe%2C%20in%20the%20Context%20of%20Education%20and%20Health%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22H.%20S.%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22V.%22%2C%22lastName%22%3A%22Taliarona%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22A.%22%2C%22lastName%22%3A%22Sardogan%22%7D%5D%2C%22abstractNote%22%3A%22%22%2C%22date%22%3A%222022-05%22%2C%22sessionTitle%22%3A%22%22%2C%22DOI%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22language%22%3A%22Englisch%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-09T12%3A08%3A40Z%22%7D%7D%2C%7B%22key%22%3A%22NF2ZHF8D%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222022%22%2C%22numChildren%22%3A1%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Taliaronak%2C%20V.%2C%20Mayet%2C%20R.%2C%20Kirtay%2C%20M.%2C%20Hafner%2C%20V.%20V.%2C%20%26amp%3B%20Pinkwart%2C%20N.%20%282022%29.%20AI-Based%20Open-Source%20Gesture%20Retargeting%20to%26%23xA0%3Ba%26%23xA0%3BHumanoid%20Teaching%20Robot.%20In%20M.%20M.%20Rodrigo%2C%20N.%20Matsuda%2C%20A.%20I.%20Cristea%2C%20%26amp%3B%20V.%20Dimitrova%20%28Eds.%29%2C%20%26lt%3Bi%26gt%3BArtificial%20Intelligence%26%23xA0%3B%20in%20Education.%20Posters%20and%20Late%20Breaking%20Results%2C%20Workshops%20and%20Tutorials%2C%20Industry%20and%20Innovation%20Tracks%2C%20Practitioners%26%23x2019%3B%20and%20Doctoral%20Consortium%26lt%3B%5C%2Fi%26gt%3B%20%28pp.%20276%26%23x2013%3B279%29.%20Springer%20International%20Publishing.%20%26lt%3Ba%20class%3D%26%23039%3Bzp-DOIURL%26%23039%3B%20href%3D%26%23039%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-11647-6_51%26%23039%3B%26gt%3Bhttps%3A%5C%2F%5C%2Fdoi.org%5C%2F10.1007%5C%2F978-3-031-11647-6_51%26lt%3B%5C%2Fa%26gt%3B%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22conferencePaper%22%2C%22title%22%3A%22AI-Based%20Open-Source%20Gesture%20Retargeting%20to%5Cu00a0a%5Cu00a0Humanoid%20Teaching%20Robot%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Volha%22%2C%22lastName%22%3A%22Taliaronak%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Ralf%22%2C%22lastName%22%3A%22Mayet%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Murat%22%2C%22lastName%22%3A%22Kirtay%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Verena%20V.%22%2C%22lastName%22%3A%22Hafner%22%7D%2C%7B%22creatorType%22%3A%22author%22%2C%22firstName%22%3A%22Niels%22%2C%22lastName%22%3A%22Pinkwart%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Maria%20Mercedes%22%2C%22lastName%22%3A%22Rodrigo%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Noburu%22%2C%22lastName%22%3A%22Matsuda%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Alexandra%20I.%22%2C%22lastName%22%3A%22Cristea%22%7D%2C%7B%22creatorType%22%3A%22editor%22%2C%22firstName%22%3A%22Vania%22%2C%22lastName%22%3A%22Dimitrova%22%7D%5D%2C%22abstractNote%22%3A%22Gestures%20and%20speech%20modalities%20play%20potent%20roles%20in%20social%20learning%2C%20especially%20in%20educational%20settings.%20Enabling%20artificial%20learning%20companions%20%28i.e.%2C%20humanoid%20robots%29%20to%20perform%20human-like%20gestures%20and%20speech%20will%20facilitate%20interactive%20social%20learning%20in%20classrooms.%20In%20this%20paper%2C%20we%20present%20the%20implementation%20of%20human-generated%20gestures%20and%20speech%20on%20the%20Pepper%20robot%20to%20build%20a%20robotic%20teacher.%20To%20this%20end%2C%20we%20transferred%20a%20human%20teacher%20gesture%20to%20a%20humanoid%20robot%20using%20a%20web%20and%20a%20kinect%20cameras%20and%20applied%20a%20video-based%20markerless%20motion%20capture%20technology%20and%20an%20observation-based%20motion%20mirroring%20method.%20To%20evaluate%20the%20retargeting%20methods%2C%20we%20presented%20different%20types%20of%20a%20humanoid%20robotic%20teacher%20to%20six%20teachers%20and%20collect%20their%20impressions%20on%20the%20practical%20usage%20of%20a%20robotic%20teacher%20in%20the%20classroom.%20Our%20results%20show%20that%20the%20presented%20AI-based%20open-source%20gesture%20retargeting%20technology%20was%20found%20attractive%2C%20as%20it%20gives%20the%20teachers%20an%20agency%20to%20design%20and%20employ%20the%20Pepper%20robot%20in%20their%20classes.%20Future%20work%20entails%20the%20evaluation%20of%20our%20solution%20to%20the%20stakeholders%20%28i.e.%20teachers%29%20for%20its%20usability.%22%2C%22proceedingsTitle%22%3A%22Artificial%20Intelligence%20%20in%20Education.%20Posters%20and%20Late%20Breaking%20Results%2C%20Workshops%20and%20Tutorials%2C%20Industry%20and%20Innovation%20Tracks%2C%20Practitioners%5Cu2019%20and%20Doctoral%20Consortium%22%2C%22conferenceName%22%3A%22AIED%202022%22%2C%22date%22%3A%222022%22%2C%22eventPlace%22%3A%22%22%2C%22DOI%22%3A%2210.1007%5C%2F978-3-031-11647-6_51%22%2C%22ISBN%22%3A%22978-3-031-11647-6%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22ISSN%22%3A%22%22%2C%22language%22%3A%22en%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-22T11%3A59%3A52Z%22%7D%7D%2C%7B%22key%22%3A%22C8E3VJQG%22%2C%22library%22%3A%7B%22id%22%3A6984777%7D%2C%22meta%22%3A%7B%22creatorSummary%22%3A%22Yun%20et%20al.%22%2C%22parsedDate%22%3A%222022%22%2C%22numChildren%22%3A0%7D%2C%22bib%22%3A%22%26lt%3Bdiv%20class%3D%26quot%3Bcsl-bib-body%26quot%3B%20style%3D%26quot%3Bline-height%3A%202%3B%20padding-left%3A%201em%3B%20text-indent%3A-1em%3B%26quot%3B%26gt%3B%5Cn%20%20%26lt%3Bdiv%20class%3D%26quot%3Bcsl-entry%26quot%3B%26gt%3BYun%2C%20H.%20S.%2C%20Taliaronak%2C%20V.%2C%20Kirtay%2C%20M.%2C%20Chevel%26%23xE8%3Bre%2C%20J.%2C%20H%26%23xFC%3Bbert%2C%20H.%2C%20Hafner%2C%20V.%20V.%2C%20Pinkwart%2C%20N.%2C%20%26amp%3B%20Lazarides%2C%20R.%20%282022%29.%20%26lt%3Bi%26gt%3BChallenges%20in%20Designing%20Teacher%20Robots%20with%20Motivation%20Based%20Gestures%26lt%3B%5C%2Fi%26gt%3B%20%5BPoster%5D.%20Workshop%20on%20Robot%20Curiosity%20in%20Human-Robot%20Interaction%20%28RCHRI%29.%26lt%3B%5C%2Fdiv%26gt%3B%5Cn%26lt%3B%5C%2Fdiv%26gt%3B%22%2C%22data%22%3A%7B%22itemType%22%3A%22presentation%22%2C%22title%22%3A%22Challenges%20in%20Designing%20Teacher%20Robots%20with%20Motivation%20Based%20Gestures%22%2C%22creators%22%3A%5B%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Hae%20Seon%22%2C%22lastName%22%3A%22Yun%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Volha%22%2C%22lastName%22%3A%22Taliaronak%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Murat%22%2C%22lastName%22%3A%22Kirtay%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Johann%22%2C%22lastName%22%3A%22Chevel%5Cu00e8re%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Heiko%22%2C%22lastName%22%3A%22H%5Cu00fcbert%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Verena%20V.%22%2C%22lastName%22%3A%22Hafner%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Niels%22%2C%22lastName%22%3A%22Pinkwart%22%7D%2C%7B%22creatorType%22%3A%22presenter%22%2C%22firstName%22%3A%22Rebecca%22%2C%22lastName%22%3A%22Lazarides%22%7D%5D%2C%22abstractNote%22%3A%22%22%2C%22date%22%3A%222022%22%2C%22sessionTitle%22%3A%22%22%2C%22DOI%22%3A%22%22%2C%22citationKey%22%3A%22%22%2C%22url%22%3A%22%22%2C%22language%22%3A%22%22%2C%22collections%22%3A%5B%5D%2C%22dateModified%22%3A%222026-06-04T14%3A41%3A54Z%22%7D%7D%5D%7D
Hübert, H., & Yun, H. S. (2024). Sobotify: A Framework for Turning Robots into Social Robots. Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’24, 910–914. https://doi.org/10.1145/3610977.3637482
Yun, H. S., Hübert, H., Pinkwart, N., & Hafner, V. V. (2024). Design Based Research of Multimodal Robotic Learning Companions. In A. M. Olney, I.-A. Chounta, Z. Liu, O. C. Santos, & I. I. Bittencourt (Eds.), Artificial Intelligence in Education (AIED 2024) (pp. 97–104). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-64312-5_12
Yun, H. S., Hübert, H., Sardogan, A., Pinkwart, N., Hafner, V. V., & Lazarides, R. (2023). Humanoid Robot as a Debate Partner. HCI International 2023 Posters. https://doi.org/10.1007/978-3-031-36004-6_74
Yun, H. S., Hübert, H., Chevalère, J., Pinkwart, N., Hafner, V. V., & Lazarides, R. (2023). Analyzing Learners’ Emotion from an HRI Experiment Using Facial Expression Recognition Systems. In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies (pp. 396–407). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-34550-0_29
Yun, H. S., Hübert, H., Taliarona, V., & Sardogan, A. (2022, May). Utilizing Machine Learning based Gesture Recognition Software, Mediapipe, in the Context of Education and Health [Poster]. AI Innovation Summit.
Yun, H. S., Hübert, H., Taliaronak, V., Mayet, R., Kirtay, M., Hafner, V. V., & Pinkwart, N. (2022). AI-Based Open-Source Gesture Retargeting to a Humanoid Teaching Robot. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium (pp. 276–279). Springer International Publishing. https://doi.org/10.1007/978-3-031-11647-6_51
Yun, H. S., Taliaronak, V., Kirtay, M., Chevelère, J., Hübert, H., Hafner, V. V., Pinkwart, N., & Lazarides, R. (2022). Challenges in Designing Teacher Robots with Motivation Based Gestures [Poster]. Workshop on Robot Curiosity in Human-Robot Interaction (RCHRI).
For the Public -
Science of Intelligence Fair 2023
