A framework for recognizing symbols (characters, numbers) drawn on touch devices and transmitting them wirelessly.

Added support in the Interactex IDE for mining information out of sensor data. The IDE now supports the development of the KneeHapp bandage.

Identified functionality needed for the development of eTextiles and wearable devices and developed an IDE to for the visual programming of smart textiles.

Implemented a framework based on the MQTT communication protocol to enable cyber-physical systems including wearable devices to communicate and share data over a back end server with regard to privacy concerns. Demonstrated it with a T-Shirt able to measure pulse from the user’s ears.

A smart ring with an integrated IMU, which learned the gestures performed by users. It enabled users to map actions to each gesture such as inputting text into a computer device, or turning on and off the music.

Added functionality to the Interactex environment to classify data (such as gestures performed by users) and trigger actions when event or gesture occurred. Demonstrated the idea with a smart glove with flexion sensors.

Created a bandage for rehabilitation of knee injuries with pressure sensors and algorithms for measuring performance of one-leg hops and side hops.

Created a testbed to support developers in the creation of software for e-Textiles by automatically finding the proper algorithm parameters.

A solution to reusing functionality in the development of eTextiles by hierarchically aggregating lower level functionality components in the Interactex environment. Demonstrated the idea with a smart glove with flexion sensors.

Development of a new visual programming paradigm in the Interactex IDE.

-

Studying different methods to visually detect the damages on metallic surfaces, from simple background subtraction to supervised and unsupervised machine learning approaches, and performing an experiment to compare the results. 

Study the current approaches to get the right dataset for deep learning processes and implementation of an application for automatic data augmentation to enrich the training dataset

 

By making use of flow-based programming paradigms and visual programming, we developed an automation system that is easy to learn and use by people without prior programming knowledge. Automation rules created with our system are represented graphically and modifying rules is as simple as dragging-and-dropping and connecting basic building blocks provided by the system. The system also features a plug-in infrastructure that makes it independent from appliance-vendor-specic protocols used to gather data from sensors or control appliances. Plug-ins can be written by third-party developers to make the system adaptable to every environment that allows access and control via a computer network. It is not tied to a specic appliance manufacturer and can be deployed on all major operating systems and hardware platforms. An informal usability evaluation conducted with the prototype we implemented over the course of the project shows, that our approach to graphical programming is easy to learn by people newly introduced to the concept. People can use the system to easily implement automated behaviour for their smart environments to increase their comfort and safe energy through intelligent, on-demand appliance control.

Building occupants have a considerable interest in the current state of their environment. Studies have shown that making building data transparent helps occupants to understand their energy consumption. In addition, transparent data can help to increase the Indoor Environmental Quality (IEQ) and therefore increase the occupants’ health. However, such data is often not accessible, intransparent and mostly not easy to understand for a typical user. Modern buildings equipped with various sensors tend to collect massive amounts of data, but often fail to provide information in an accessible, user-friendly and meaningful way. To give an example, a level of particulates at 170.000 ppcm might not be a helpful answer to understand the air quality. I pose the research hypothesis that an effective visualization of IEQ-related data in a smart building helps its occupants to develop an awareness about their IEQ. Furthermore, many smart environments offer diverse proprietary and therefore incompatible controlling systems. In order to interact with these or to gather information, its occupants have to switch between multiple interfaces. As a solution, I combine comprehensive and meaningful IEQ information with the possibility to control and improve the environment using a mobile application. The system’s usability is evaluated in iterative cycles with application domain experts and by means of two user studies. The first study serves the purpose of illustrating the user’s comprehension of the design concept whereas the second study evaluates the resulting system’s performance in terms of usability. The studies show that the mobile application enables users to get a better understanding for IEQ-relevant data.

In our research we propose a portfolio of serious games for home-based stroke rehabilitation. The game approach enriches a patient’s training experience and thus establishes a higher level of compliance to prescribed exercises, while maintaining a supportive training environment as found in common therapy sessions. Our system monitors the patient’s performance while exercising and provides clinicians with an interface to personalize the training for their patients. In order to achieve an increased compliance to home-based exercising, we provide a collection of mini games based on rehabilitation exercises used in conventional physical therapy. All games are part of a joint storyline that encourages a patient to accomplish tasks on a treasure island. Playable games change over time to adapt to the patient’s state of rehabilitation and drive diversification. This research was inspired by a previous project at Carnegie Mellon University that provides a non-gamified way of self-governed exercising. While this prototype was a major breakthrough in the domain of computer-aided rehabilitation, patients experienced difficulties using the complex interface resulting in a lack of compliance and motivation to use the system. However, to offer a non-gamified alternative of exercising to our solution, the previous system was integrated as an independent component, giving patients the choice between the two systems.

A manual-procedural activities (MPA) involves following the steps of a given workflow for manipulating the physical world. Examples include manual assembly, repair and maintenance, different crafts, cooking, etc. To learn an MPA the trainee needs to master both the steps of the procedure and the hand skills required for manipulating physical objects and the using the tools. TUMA: An Intelligent Tutoring System for Manual-Procedural Activities supports trainees in learning an MPA. 

Multiple topics available in context of the TUMA project. For details of the topics please see my chair web page. 

Design and development of a smart baton. The baton consists of an IMU and the controller boards that send the motion data in different formats via BLE to the desktop computer. An interface to other systems and the the program for configuration of the baton is developed.