The scope of this master thesis is to build an (semi)automatic system to train a Siamese net to find the similarity of the fasteners (namely bolts, nuts, screws and washers). Using the output of the Siamese net, the system should group and sort the fasteners in similar groups (in a fine-grained manner. For example it should put all the AS1800 bolts in one group and all the MS2650 bolts in another one). To successfully undertake this thesis, you need to have solid knowledge on machine learning and convolutional neural networks, together with experience in Python programming and working with at least one deep learning framework (Keras, PyTorch, TensorFlow).

 

 

In this thesis, a flexible and extensible framework for communication with fixtures, using a variety of communication protocols is proposed. The solution is based on the REST architectural style and the broker pattern that is used for decoupling the clients from fixtures. The broker provides a uniform protocol for controlling all fixtures in the environment, while hiding the complexity of different fixture communication protocols being used. The protocol, provided by the broker, is a HTTP based protocol which can be easily implemented in any programming language that supports the HTTP protocol. Using this feature, a variety of mobile or web based smart environment applications can be developed.

The master thesis studies how Radio-frequency Identification (RFID) along with Cloud-based computing can be used to improve the user experience of household appliances. Bringing RFID and cloud computing together provides a modern solution enabling the users ubiquitous real time control and management of a wide range of household appliances. This thesis describes the development of a scalable platform named "Cloud oriented Push Platform" (CoPP). The main purpose of CoPP is to connect and control various smart devices. These functions are enabled by storing and retrieving user information, and are supported by push notification services. Furthermore, CoPP may facilitate the development of a low-cost home automation system that is easy to deploy and maintain. Finally, this thesis displays a development and implementation process of a cloud based RFID solution for a content-aware smart fridge. Additionally, in order to determine the best solution for the implementation of the smart fridge, this thesis also evaluates available cloud computing services, RFID technologies, and different micro-controllers from several perspectives, keeping economic, legal, and technical restrictions in mind.

The current thesis proposes a gamified approach that encourages the office workers to start saving energy in a collaborative way. They can define sequences of actions to be performed immediately or at a later time and send tailored recommendations to their colleagues, in the form of action sequences. By actively using the application, controlling their workplace in an energy-efficient way and encouraging their colleagues to have a pro-environmental behavior, the users enter a competition. The gamification strategy used in the current application makes use of game mechanics elements such as points, badges, levels and high scores. The pro-environmental behavior of the users is rewarded with points and/or stars. By accumulating points, they can climb in the high scores. Each player starts the game with five empty American flags, without stars on the canton. By accumulating stars, they can fill in the flags one by one and advance to the next level.

A solution is proposed that aims at leveraging smart meters and delivers insight into consumption as well as control to office occupants using a mobile application. Occupants receive smart meters for tracking their appliances and are personally in control of registering these. The application gives them access to current, monthly and weekly data as well as the possibility to switch appliances on and off. Additionally, consumption comparisons are presented which express Wh (watt hours) in terms of another appliance with the goal to increase the occupant’s understanding of the consumption of their appliances. This prototype of an iPhone application was built at the Intelligent Workplace at the Carnegie Mellon University. The Intelligent Workplace is a living and lived-in laboratory providing an instrumented environment with sensors and controls to developers and occupants who can test applications in their daily use. The application was created in a user-centered and scenario-based approach. Continuous delivery ensured a highly iterative process with weekly deployments to the occupants. The design of the application was reviewed weekly by experts in building controls of the Intelligent Workplace. At the end of the project a survey was conducted, in which occupants were asked about whether the application helped to increase their understanding of the consumption of the appliances and reduce the consumption. The results show that, among the implemented features, presenting the current consumption, split by appliances and giving occupants the possibility to control them has most impact when this functionality can be accessed fast and conveniently.

By making use of flow-based programming paradigms and visual programming, we developed an automation system that is easy to learn and use by people without prior programming knowledge. Automation rules created with our system are represented graphically and modifying rules is as simple as dragging-and-dropping and connecting basic building blocks provided by the system. The system also features a plug-in infrastructure that makes it independent from appliance-vendor-specic protocols used to gather data from sensors or control appliances. Plug-ins can be written by third-party developers to make the system adaptable to every environment that allows access and control via a computer network. It is not tied to a specic appliance manufacturer and can be deployed on all major operating systems and hardware platforms. An informal usability evaluation conducted with the prototype we implemented over the course of the project shows, that our approach to graphical programming is easy to learn by people newly introduced to the concept. People can use the system to easily implement automated behaviour for their smart environments to increase their comfort and safe energy through intelligent, on-demand appliance control.

In this thesis, I propose NICE (Natural Intuitive Camera-based Environment) control which allows occupants to use freehand-gestures, speech commands and a multimodal combination of both in order to interact with individual devices in a smart building. The usage of gestures and speech is a natural user interface that allows for an easy and intuitive interaction. Backed by a 3D camera, users can interact from various positions in a room without any additional personal device. The development of NICE control was complemented by two dedicated user studies as well as a continuous quasi-experiment in a user-centered approach. The first, a wizard-of-oz study, provided essential findings for the design of gestures and speech commands, whereas the second user study evaluated the real-world performance of NICE control. Participants stated they enjoyed natural user interfaces for controlling a smart room and were able to complete about 90% of the given tasks. Speech showed the fastest overall completion times for the tasks. Notably, participants ranked gesture commands as fastest and most fun.

This thesis introduces a way of combining any number of location providers into one centralized location provider. By using the input of multiple location sources the accuracy of every single source can be improved and the availability of location information can be increased. Simple scenarios where two sources’ location information overlaps will be addressed as well as more complex ways of combining the information. The system introduced is flexible and extensible in terms of fusion algorithms or sources used. It’s ability to provide indoor location information more accurately and with higher availability makes it a better alternative to existing location based services.

The central component of this research is an editor for the definition and customization of multimodal interactions and controls in smart buildings.

Building occupants have a considerable interest in the current state of their environment. Studies have shown that making building data transparent helps occupants to understand their energy consumption. In addition, transparent data can help to increase the Indoor Environmental Quality (IEQ) and therefore increase the occupants’ health. However, such data is often not accessible, intransparent and mostly not easy to understand for a typical user. Modern buildings equipped with various sensors tend to collect massive amounts of data, but often fail to provide information in an accessible, user-friendly and meaningful way. To give an example, a level of particulates at 170.000 ppcm might not be a helpful answer to understand the air quality. I pose the research hypothesis that an effective visualization of IEQ-related data in a smart building helps its occupants to develop an awareness about their IEQ. Furthermore, many smart environments offer diverse proprietary and therefore incompatible controlling systems. In order to interact with these or to gather information, its occupants have to switch between multiple interfaces. As a solution, I combine comprehensive and meaningful IEQ information with the possibility to control and improve the environment using a mobile application. The system’s usability is evaluated in iterative cycles with application domain experts and by means of two user studies. The first study serves the purpose of illustrating the user’s comprehension of the design concept whereas the second study evaluates the resulting system’s performance in terms of usability. The studies show that the mobile application enables users to get a better understanding for IEQ-relevant data.

In our research we propose a portfolio of serious games for home-based stroke rehabilitation. The game approach enriches a patient’s training experience and thus establishes a higher level of compliance to prescribed exercises, while maintaining a supportive training environment as found in common therapy sessions. Our system monitors the patient’s performance while exercising and provides clinicians with an interface to personalize the training for their patients. In order to achieve an increased compliance to home-based exercising, we provide a collection of mini games based on rehabilitation exercises used in conventional physical therapy. All games are part of a joint storyline that encourages a patient to accomplish tasks on a treasure island. Playable games change over time to adapt to the patient’s state of rehabilitation and drive diversification. This research was inspired by a previous project at Carnegie Mellon University that provides a non-gamified way of self-governed exercising. While this prototype was a major breakthrough in the domain of computer-aided rehabilitation, patients experienced difficulties using the complex interface resulting in a lack of compliance and motivation to use the system. However, to offer a non-gamified alternative of exercising to our solution, the previous system was integrated as an independent component, giving patients the choice between the two systems.

According to current studies intuitive ways of controlling are for instance gestures, the use of voice-recognition or mobile phone applications. In this thesis the focus is set to the use of voice-recognition and the combination with existing controls. A possible way of integrating voice-recognition in the current system is using Apple’s HomeKit. This framework provides the possibility of using Siri, Apple’s voice assistance, to control accessories. Furthermore, users can exchange the application with any other HomeKit supporting application in the App store according to their personal preferences.

The aim of the thesis is the develop a framework for context-aware recipe recommendation systems. In order to address user-specific recommendations, the system is using three different information sources. First, it analyzes ratings from the community that is using the application (that is developed as a part of the thesis). Second, it extracts the current context of the user by using measurements of the user’s smartphone sensors. The context information includes location, season and activity. Finally, the system uses the data of an activity tracker information to draw inferences about the current mood of the user, which is also part of the current context.

In this thesis, a flexible and extensible framework for maintaining supermarket content and recommending products to supermarket customers, has been proposed. The system is composed of three distinct subsystems namely: Client Applications, Supermarket Content Management System (SCMS) and a Product Recommendation Engine (PRE).

Deployment and publication of software can be painful and time consuming. In this thesis we address the release management of software related to continuous integration and delivery. As collaboration is a central point in agile software development, we aim to promote collaboration in the area of release management by improving the release process. We focus on the release process, as the Rugby process model describes it. This thesis deals with the usage of releases in the context of Rugby and examines their impact. We conduct an evaluation to address the use of releases in meetings. This leads to an idea that can improve the application of releases as communication models.

Regular user feedback is important, as it helps to validate requirements and improves user satisfaction. However, it fails to unfold its full potential in many projects. Common challenges include low user motivation, fragmented feedback channels, and missing information, making it difficult for the development team to extract the underlying requirements of the user. We aim to address these problems by proposing a system for Context-Aware Feedback (CAFE), which integrates user feedback into the software evolution process. The objective is to provide a channel for users to give feedback with as little effort as possible in order to bridge the gap between users and developers. CAFE stores feedback directly in the issue tracker enriching it with recorded usage context and visualizes the evolution of requirements through user feedback. In a formative evaluation with controlled user experiments we found that both developers and users prefer CAFE to other feedback channels and that automatically gathered context data is useful. Both the fact that user feedback is saved in the issue tracker and the added value of Feedback Traceability are regarded as beneficial.

Continuous delivery allows software developers to deliver applications to the customer with a few clicks and therefore helps them to gather feedback fast. We present a continuous delivery process model and describe how we integrated this process model into the iOS Praktikum. We explain how we thought the concepts to the student using different teaching resources like a cross-project release-team. We evaluate the course using an online questionnaire after the end of the projects. Our observations show that participating students understood and applied the concepts of continuous delivery and are convinced about the benefits. Continuous delivery increased the amount of releases increased the speed of the delivery process compared to earlier versions of the Praktikum.

Mobile applications are of increasing relevance to businesses in all industries. While companies are progressively adopting agile methodologies to facilitate faster development, there is a lack of suitable workflows that accommodate all requirements and restrictions of a corporate environment. There are, however, promising approaches to continuous delivery for mobile projects in the academic field. Our goal is to transfer a well-proven process model and system from Technische Universität München (TUM) to the mobile business line at Capgemini, a multinational provider of IT consulting, outsourcing, and professional services. We describe a modified and extended process model that includes optional as well as variable subprocesses to provide a flexible solution. By allowing each project to adapt the workflow to their specific needs, we simultaneously provide a way of standardizing how continuous delivery is applied to mobile projects in the company. After evaluating our solution in eight deliberately chosen projects, results show increased frequency and thoroughness of integration, resulting in more immediate feedback about changes. Delivery is simplified and accelerated, saving both time and resources.