An approach to enhancing contemporary handmade products with historic narratives

Writer : Nikolaos Partarakis, PhD, Effie Karuzaki (Mrs.), Paraskevi Doulgeraki (Mrs.), Carlo Meghini (Mr.), Cynthia Beisswenger (Mrs.), Hansgeorg Hauser (Mr.), Xenophon Zabulis, PhD
Year : 2021


Traditional crafts involve craft artefacts, materials and tools while encompassing craftsmanship as a form of intangible cultural heritage (ICH). In modern societies, the production of traditional crafts (TCs) has decreased due to the lack of interest and the reduction in the number of practitioners. In this context, we explore modern digital technologies to enhance a contemporary TCs object with digital stories that connect it to the history, society, traditions and values of Europe. Such stories can support new interest in TCs objects by enhancing their meaning, value and uniqueness. In this research work, we have created a handbag that functions both as an accessory and as a contemporary craft that can be experienced to reveal its unique history and identity.


In consumer societies, the availability of affordable, mass-produced goods has seriously reduced the market for handmade products. This has placed traditional crafts (TCs) in a vulnerable position. The factors that influence the decline of TCs can be summarised as the reduction of market demand for traditional handmade goods, competition with imported mass-produced products from third-world countries, regression of the workforce as skilled craftspeople get older, the globalisation of the economy and the rise in the cost of raw materials (European Policy Brief of the RICHES project)]. At the same time, the globalisation of the economy, branding policies, advertisement and product presentation in media may lead to the false perception that craft products are associated with the past, thus making them less attractive to young people. This problem has financial dimensions as well: because of the lack of interest in and support for craft-related entrepreneurship, fewer people are exploring it.

According to the European Policy Brief of the RICHES project, however, some isolated positive signs may give rise to an increase in handmade products in the future. For example, do-it-yourself culture, the emergence of digital fabrication at home (e.g. 3D printing) and hybrid forms of making, which merge the traditional with the contemporary, are giving life to several small businesses. At the same time, the rising need for individuality in consumer societies has increased the demand for unique, customised products. In this context, a repositioning of craft skills, techniques, patterns and materials is possible by augmenting them through the integration of digital technology. In this article, we argue that this may lead to a new form of craft-based innovation, which can also be promoted through ethical approaches to sustainability, ecologic use and local production.

According to Kurin (2007), ‘Intangible Cultural Heritage (ICH) – a loose English translation of the Japanese mukei bunkazi, is broadly defined in terms of oral traditions, expressive culture, the social practices, ephemeral aesthetic manifestations, and forms of knowledge carried and transmitted within cultural communities’. Heritage crafts, a form of ICH, are strongly bound to the history and traditions of the communities in which they are practised (Kurin 2007), but they are also linked to the history and tradition of humanity as a whole, through common links and craft manifestations across the globe.

In this work, we focus on the craft of textile weaving and the presentation of the historic patterns, symbols and motifs through storytelling. Like many craft products, in traditional textiles, one can often recognise the origin of textile from its decorative patterns or its visual style. The pattern of a woven product may contain an aesthetic, decorative or artistic dimension. Weaving a patterned fabric is a tedious process that requires additional skill to encode specific multicoloured patterns. The invention of mechanical looms increased the automation of the tedious task of patterned weaving and resulted in the ability to manufacture highly detailed patterns more quickly and with fewer errors.

The relation of weaving to storytelling and music is surprisingly old. For example, Euripides’s consistent pairing of weaving with the recitation of mythological stories is intriguing:

As the texts suggest, women retell stories while weaving, and this means of retelling is differentiated from other modes of storytelling. If the processes of relating such myths were directly associated with patterning in the textiles as they are produced, then the recitation would necessarily embed numerical information about the thread counts of the woven pattern. (Tuck 2009)
It remains a mystery today whether storytelling provided thematology or acted as a numeric code corresponding to groups of knots of a given colour or the number of warp threads overshot by those of the weft. Regardless of the kind of connection, it is still important today to revive memories and stories carried by traditional motifs, patterns and symbols used in TCs (Heckman and Heckman 2003).

To support this vision, this work presents the design, implementation and augmentation of a handmade handbag, to act as a storyteller for the historic patterns of Haus der Seidenkultur (HdS), in the city of Krefeld, Germany. From a technological point of view, this research work exploits the evolution of augmented reality (AR), virtual reality (VR) and mixed reality (MR), which tries to identify and design new forms of interaction with cultural heritage artefacts, thus providing an alternative approach for experiencing handmade products through storytelling and mobile devices.

Background and related work Social and historical dimensions of textile manufacturing in Krefeld: Silk and the history of Krefeld

Pure silk, one of the oldest known natural fibres, is still highly fashionable even after thousands of years. This beautiful, elegant fabric fascinates humankind with its precious radiance, gossamer touch and strength. As long ago as antiquity, the incomparable haptic inspired the powerful in this world to such a degree that they even weighed the fibres in gold. Kings, emperors and the clergy wore splendid silken garments; the wealthy did not want to forgo silk clothes.

The history of Krefeld is closely linked to this magical material. Today, there is a small museum, the HdS, which shows how the history of silk has shaped the development of the town over the past three centuries. Krefeld is a town with a Roman past. Excavations in neighbouring Gellup revealed the existence of at least six Roman camps up to the fourth century AD. Burial sites dating to the time of the Franks indicate that the area was settled continuously. Starting in the late Middle Ages, Krefeld belonged to the County of Moers. The Counts of Moers made every effort to establish the Reformation early in their territory; the first Reformist preacher in the area took up his post in 1561. At the beginning of the 17th century, Krefeld came under the rule of the Netherlands, and the town became an island of religious tolerance. Consequently, in a period in which the denomination of the population was determined by the denomination of the ruler, Mennonites from near and far came to Krefeld and settled there. This immigration had far-reaching consequences that have shaped the profile of the town to the present day.

The religious refugees brought with them linenprocessing skills, and, because they were also successful business people, they laid the foundations for economic growth and prosperity. The Von der Leyen family,
immigrants from Radevormwald, also contributed significantly to the development of the ‘Town like Silk and Velvet’. Originally linen weavers, the family increasingly changed the emphasis of their business to silk weaving. In 1702, Krefeld came under Prussian rule, and silk weaving became the most important economic factor: sales to the Prussian court in Berlin flourished. During this period, the silk weavers were out workers who received orders from merchants and traders to weave fabrics. The looms were set up in front of the light window in the typical small cottages, some of which still exist today. The head of the household was normally the weaver, and other family members helped with tasks such as reeling the thread onto the bobbins for the shuttle. On one of the main avenues of the town, there is a monument to the weaver ‘Meister Ponzelaar’. He wears his Sunday best: a frock coat (in the local dialect, a ‘Laakesserock’), a high-necked waistcoat, a small collar with a silk scarf and a ‘Jraduutkapp’ (a black cap). At the end of the week, he takes the finished fabric on the beam, together with a bag containing any leftover thread, to the merchant’s office. There, he was paid and received a new prepared warp beam and thread for the week ahead. Such weavers were a typical sight in the town until the beginning of the 19th century. Their craft required rapid comprehension and rhythmic movement of hand and foot.

In 1785, Edward Cartwright invented his first mechanical loom and continued to make improvements to it. The enhanced looms became available on the market in 1820. With the advent of mechanisation, the silk entrepreneurs started to build factories where all the machines were powered by one source of energy and the workers were responsible for more than one loom.

Historic patterns employed in this research work

Since antiquity, craft communities have been embedding cultural elements in weft products, for decorative or emblematic purposes. Like many craft products, patterned fabrics have a dual purpose. They are  usable items, and they include heritage that represents a region and its people. In terms of traditional textiles, one can often recognise the origin of a piece of textile from its decorative patterns or visual style. Thus, patterns, motifs and shapes of a craft object can relate to historical ones that are still reproduced today or new ones that appear in the craftsmanship of younger, contemporary generations. Furthermore, some creations are determined by craft locality: these are unique to the craft community, and they might depend on the social, religious or environmental characteristics of the specific area.

The revival of patterns, symbols and motifs by the modern textile industry

According to Anderson (2012), the emergence of the contemporary maker movement has been empowered by digital technology, and especially the internet, which has allowed the connection of communities of makers, buyers and sellers, transforming the maker movement into a global phenomenon. The impacts on the craft economy are directly connected with the recalibration of the retail industry by granting makers direct access to the globalised market, reviving interest in craft skills and craft products, and contributing to the democratisation of design, with patterns and techniques inspired by culture and cultural heritage. In Figure 1, some examples of contemporary designers reaching the market through the revival of traditional techniques, patterns, symbols and motifs are presented.

Digital storytelling technologies

Digital technology has empowered the maker movement to create original, contemporary, handmade products inspired by art, culture and tradition and to revive patterns, symbols and motifs. In this changing world, in this work, we aspire to revive and present the cultural significance of patterns, symbols and motifs as bearers of tradition, collective memories and stories.

Our approach is rooted in the idea that humans exhibit a limited capacity to memorise inventories, such as a list of events as opposed to stories imbued with meaning or causal dependencies (Straub 2008). In this work, an approach is proposed that binds contemporary craft products with the cultural context of their depicted patterns, symbols and motifs through stories that explain their elements. As such, among the available technologies, augmented reality (AR), virtual reality (VR) and mixed reality (MR) can empower experiences in multiple contexts of use and can be gracefully used to augment physical items or their digital representation with information.

AR for overlaying information on the real world

According to Chatzopoulos, Bermejo, Huang and Hui (2017), the evolution of the capabilities and features of mobile devices such as smartphones, combined with ubiquitous and affordable internet access and the advances in the areas of networking, computer vision and cloud computing, has transformed AR from science to reality. In 2016, the mobile game Pokémon Go, which essentially made AR technology popular throughout the world, became a great example of how mobile gaming can happen in an augmented world (Rauschnabel, Felix and Hinsch, 2019). Today, AR is a global trend with a market estimated to grow from USD 10.7 billion in 2019 to USD 72.7 billion by 2024, and its applications include games, training, assistance providing, design and modelling, education, tourism, marketing and many more.

However, one of the earlier manifestations of AR regards its exploitation in the context of cultural heritage objects and sites, aiming to improve the visitors’ experience in these spaces. For example, mobile museum guides use AR technology to enrich exhibits with information (Wojciechowski et al. 2004; Schavemaker et al. 2011; Schmalstieg and Wagner 2005; Sidyawati et al. 2013). This approach has also been applied at archaeological sites, giving rise to new forms of digitally augmented tours (Stricker et al. 2001; Vlahakis et al. 2001; Kourouthanassis et al. 2015). Virtual restoration of religious heritage objects through 3D AR technology content has also been explored (Gîrbacia et al. 2013; Narciso et al. 2015). In the same vein, mobile AR has been used to enhance archaeological sites with more than static information. For example, virtual and real scenes at the site of ancient Pompeii have been mixed, so virtual avatars can present moments from everyday life in Pompeii (Papagiannakis et al. 2002; Papagiannakis et al. 2005; Papagiannakis et al. 2006). One of the major opportunities for the penetration of mobile AR technologies in the museum context is the low cost of such solutions because no equipment is required from the museum side (Angelopoulou 2011). According to the same research, museums have started to realise that AR can be an effective way to build user interest in museum collections and exhibits. AR is considered to be a way to preserve history, enhance visitor satisfaction, generate positive word-of-mouth, attract new target markets and contribute to a positive learning experience. AR also has an economic, experiential, social, epistemic, cultural, historical and educational value from stakeholders’ perspectives (Tom Dieck and Jung 2017).

Virtual reality

VR is ‘a very powerful and compelling computer application by which humans can interface and interact with computer-generated environments in a way that mimics real life and engages all the senses’ (Burdea and Coiffet 2003). As such, VR has gained the attention of the cultural heritage sector for reviving the past (Burdea and Coiffet 2003) and thus opening a window in time and space, so users can travel and experience the past with all of their senses. VR achieved this through combining computer graphics, interaction and novel approaches towards the digitisation of cultural heritage components, such as artefacts and monuments of material heritage.

3D models and 3D reconstructions are useful to preserve information about historical artefacts. However, the potential of this digital content is not fully realised, because it is not used to communicate information to museum visitors (Gonizzi Barsanti et al. 2015). A recent example is the exploitation of 3D reconstruction in the context of 3D games that facilitate exploration and learning (Partarakis et al. 2020). In this context, VR can be exploited to bridge the gap between the virtual representation of an artefact and the visitor, allowing multiple forms of interaction and storytelling with cultural heritage in the virtual or physical museum setting (Bruno et al. 2010; Carrozzino and Bergamasco 2010).

Mixed reality

MR refers to the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects coexist and interact in real time. According to Bekele et al. (2018), several studies demonstrate that the use of new and combined media enhances how culture is experienced. In this regard, cultural heritage uses such technologies for different purposes, including education, exhibition enhancement, exploration, reconstruction and virtual museums. Among the multiple manifestations of MR for cultural heritage, most relevant to this work is the use of augmented artefacts to access and interact with information and artefacts. Previous approaches include multimodal interfaces to augment physical cultural heritage artefacts with information (Partarakis et al. 2016; Partarakis, Antona and Stephanidis 2016), art-inspired games that employ physical items (Partarakis et al. 2015; Vayanou et al. 2019), informative art displays (Partarakis et al. 2011; Redström, Skog and Hallnäs 2000; Holmquist and Skog 2003; Ferscha 2007) and immersive mixed reality environments for cultural heritage (Zidianakis 2021).


This work is about combining history and traditions with contemporary consumer goods. As such, interest in the commercial part of AR is vivid in this research work, with the potential to exploit its outcomes in MR settings. Overall, AR can be beneficial for companies and brands contributing to the creation of customer satisfaction, as other forms of experiential marketing have done (Bulearca and Tamarjan 2010). However, its use remains rather limited in the realm of heritage education (Anderson, 2012). In this vein, this work experiments with the use of AR technology to enhance the capacity of the contemporary cultural heritage sector to deliver stories and, through stories, enhance their meaning – transforming them from aesthetic objects to objects that are bearers of stories and memories. With this approach, we expect that new forms of bonding with cultural heritage artefacts will be made possible, thus empowering the capacity of companies exploiting traditional crafts to innovate and reach new target audiences.

Case study: A physical handbag design and handcrafting Research context

TCs involve craft artefacts embedding patterns, symbols and motifs that can be used to create engaging cultural products and storytelling experiences.

The idea of a contemporary accessory

The idea of creating a contemporary accessory was born during the discussions and co-creation workshops with the craft community of HdS that took place at the HdS in Krefeld, Germany. During these sessions, it became apparent that both the textile manufacturing and the history and tradition of Krefeld were important aspects of its cultural heritage. In this context, during knowledge collection, the historical patterns were documented and translated into text-based narratives representing the socio-historical context of the patterns used by HdS. At the same time, the understanding of craft revealed the hidden artistic dimensions of the craft and made available a detailed representation of the crafting process.

Transmitting stories to new generations

One of the most important goals of HdS is to raise awareness of the unique cultural heritage possessed by the museum and to connect its legacy with European history and tradition. Through representation and presentation of the craft, HdS wishes to ensure the transmission of memories, values and identity to the next generation. But how can stories be transmitted effectively? Much research on storytelling has been done and many approaches have been tested. In the search for something new – that is both a tangible artefact and at the same time a carrier of the history and intangible dimensions of the community, tradition and place – the idea of a handbag, the most popular women’s accessory, emerged.

Designing the handbag

The design of the handbag was a challenging task. By closely studying the HdS patterns, the following issues were encountered, which required a careful design approach: (a) the patterns are almost monochromatic, (b) patterns are woven in bright yet intense colour variations that require careful combinations, (c) due to the intense iconographic elements of the patterns, the risk of creating a cluttered visual result is high.

Taking into account the above issues, a careful study of visual aesthetics was needed to decide upon the design approach. Initially, it was decided on the shape of the bag. Taking into account that the handbag should contain several patterns, the size of the bag was decided to be approximately 35 by 40 centimetres. The size was selected following the size of the weaved patterns. Then, based on the size, several sketches of the bag were made to further discuss the concept and help determine the spatial arrangement of patterns. Some preliminary sketches from various angles are presented in Figure 2. As shown in these sketches, it was decided to make the bag minimal in terms of design and ensure that enough negative space was left on both sides to host patterns. Furthermore, this minimal approach would minimise the possibility of visual clutter between the artefact and the artistic qualities of the textile fabrics. The objective was to make the bag ‘invisible’ in terms of aesthetics to transmit the aesthetic qualities of the subject matter depicted by the patterns. Additionally, it was decided that the bag should have a big, solid base, to ensure that it would stand when placed on a surface, and a long strap, so the bag could be worn on a shoulder but still be visible.

Then, based on the preliminary sketches of the bag, it was decided that the patterns should be constrained to a strict geometric approach, so rectangular pieces of fabric would be selected to maintain a common visual feature among the patterns. Furthermore, it was decided that some form of neutral separator between the patterns was required to provide some form of visual arrangement. A black stripe, for example, could provide a good separator between colourful patterns. Third, it was decided that negative space (visually empty space) would further differentiate areas of interest from ones not visually interesting.

Based on the above decisions, it was decided that a concept rooted in geometric abstraction, as manifested in 20th-century art history, could provide a structural abstraction and thus a base for the aesthetic design of the object. This resulted in the selection of Piet Mondrian’s abstract style. In his paintings, primary colours are orchestrated in strong geometrical abstraction, providing time and space for the eye to experience the patterns and colours.

Building a story around the bag

The outcomes of the aforementioned activities provided the idea of the artefact, the kind of visual information to be embedded in the artefact and the geometric and artistic principles to be followed. What was missing was a marketing story to make the handbag appealing to the market.

To do so, first, the market was identified as one of the contemporary handcrafted products that have both an artistic and historical dimension. People that select such products are not searching just for aesthetics but also stories. In this context, the story of this bag can be marketed under the following catchphrase: ‘A handbag to carry and to experience’.

Building a digital concept around the bag

The main questions during the brainstorm sessions to establish a digital concept around the bag were the following:
how do you experience a handbag? How can a handbag transmit stories?
During the discussions, input from the technical personnel provided an overview of the knowledgerepresentation capabilities of Mingei and the narrativeauthoring environment to help further elaborate on the concept.

The outcome of this process is summarised as follows:
the handbag is a practical accessory to carry things. But when the bag is not being carried, can it be experienced? The answer is that, in principle, yes, but through which technology? What is the most common digital accessory placed in a handbag? And the answer was apparent: the smartphone!

Following this rationale, the digital concept was conceptualised. We started from the principle that the handbag contains stories waiting to be discovered with your mobile phone. The mobile phone gives life to the handbag via an AR application. The idea was to create a piece of craftsmanship that is not only aesthetically pleasing but also a storyteller.

Crafting the handbag

Crafting the handbag was, in itself, an interesting process considering the identified requirements and concepts. One of the major challenges was the selection of the appropriate material for the handbag. On the outside, a neutral material that can be easily painted by hand and that can provide a mental connection to art artefacts was preferred. To this end, it was decided that unpainted cotton canvas would be used. Cotton canvas is a strong fabric used in painting and can be painted in various media. At the same time, it is a material that can support the overlaying of delicate textile fabrics, such as the ones provided by HdS.

The interior of the handbag should provide stiff support to the exterior to be a bag that can stand on its own when placed on a surface and provide a smooth surface on the exterior to eliminate possible deformations of the patterns. It is important that the patterns remain visible and with limited deformation to support the recognition algorithms. Another requirement for the interior was that it be waterproof. This is a practical provision to support the actual usage of the artefact as a handbag. For the interior, a stiff, waterproof, synthetic material was used. The same combination of canvas and synthetic material was used for the handbag straps to give a consistent appearance and feel as the rest of the handbag.

After the selection of material, the actual craft creation was initiated. Initially, the pieces of the handbag were designed in semi-transparent paper and then transferred to the fabric layers. Then, the abstraction of the selected Mondrian composition was sketched on both sides of the exterior fabric in water-soluble pencil. The black stripes were then painted by hand, on top of the sketch, with a black textile marker to create the outline of the composition.

In this phase, the creation process was paused to proceed with the selection of patterns in conjunction with the abstract design of the handbag. Although the design does not exhibit the vividness of the pure primary colours used by Mondrian, the selection was made to create maximum visual contrast and thus remain in the spirit of the original creation. An example of pattern placement evaluation is presented in Figure 3. The rationale of the selection of pattern placement is twofold. First, we were interested in visual stimuli; thus, placement should produce stimulus input to the eye of the viewer, and this is done through contrast variations. In the case of patterns with limited contrast to each other, negative space was integrated to support the visual differentiation of patterns. Second, we were interested in the visual contrast of the greyscale representation of patterns. Such visual contrast is needed to assist the image-processing algorithms in differentiating patterns that are not separated through negative space and thus could coexist in the visual field of an application. Finally, to ensure the patterns were not curled on the sides of the bag, spacers had been integrated on the sides of the composition to ensure that visible patterns remained on the main planes of the bag.

After the selection of patterns, their cropping was decided based on their placement to ensure that the patterns would remain visible after cropping. Then, the patterns were sewed on both sides of the bag and the bag straps, and the bag was assembled to produce the final artefact as shown in Figure 4.

Digital augmentation of the physical handbag Mobile AR

In parallel to the design and crafting of the bag, technical experts prepared the transfer of the bag to the digital world. To build the AR application, ARCore was used, which is a Google software development kit that supports AR applications for both Android and iOS using Unity3D, which is a powerful cross-platform game engine. After the installation and the proper set-up of the ARCore kit in Unity, a database of reference images for the patterns had to be provided, so they could be recognised by the ARCore in the real space via the camera embedded in the mobile device (ARCore augmented-image database).

According to ARCore specifications, every image in this database should meet the following guidelines:

1\. Images should avoid repeating motifs and large numbers of geometric features – or very few features\. 2\. The images should have a relatively high contrast when turning into greyscale because no colour information is used by the identification algorithms of ARCore. 3\. The images should be flat \(for example\, not wrinkled or wrapped around a bottle\)\. 4\. All images should feature a resolution of at least 300 x 300 pixels\. Using images with high resolution does not improve performance\.

Selection of patterns

To meet the first guideline, fabric pieces for the creation of the bag were carefully selected so as not to repeat motifs. Furthermore, extra care was dedicated to avoiding fabric parts that contain many geometric features. For example, Figure 5 shows two fabrics provided by the HdS. The left motif contains a large number of geometric features, which would hinder the ability of ARCore to recognise it in the real space. Fabric pieces that would hinder the ability of ARCore to recognise it in the real space were not selected for the creation of the bag.

Image contrast evaluation

After the pre-selection of patterns presented in the previous section, photos of the remaining fabric pieces, from various angles, distances and lighting conditions, were acquired and turned into greyscale to check their visual contrast. Fabric pieces that had a low contrast value in all variations were rejected.

The AR app implementation

The implementation of the AR app started with the collection of photos of patterns as they were placed on the bag. Each image was then tested with the arcoreimg tool2 to get a quality score between zero and one hundred. The closer the score to one hundred, the higher the possibility that the ARCore will successfully recognise the image in the real space. The images were taken using the camera of a Samsung Galaxy Tab S5e and again from various angles and distances.

Initially, however, the application was not able to recognise most of the motifs. This was because ARCore uses a fixed focus, so the camera image in the ARCore application was blurry unless the phone was held at a specific distance from the bag. This problem was partially solved by adding a <ARCoreSession> module to the camera that enables setting the camera focus mode to ‘auto’. Using autofocus, the camera could focus on the target motifs on the bag better, and the application was able to identify the patterns a little better than before – but it still was not good enough. After a lot of trials, it was clear the reason for the low recognition rates was the quality of the images provided to the database. Indeed, guideline four explicitly states that ‘using images with high resolution does not improve performance’. Using images with high resolution hindered the performance of the ARCore recognition algorithms. To solve this problem, software was used to lower the images’ quality, although ensuring that the shortest side of the image would be 300 pixels. For example, one image was transformed from 3024 x 4032 pixels to 300 x 400 pixels. In this way, the ARCore performed significantly better in successfully recognising images; however, the application needed more than a second to do so. When an image is initially detected by ARCore, and no expected physical size has been specified, the ARCore will try to specify its size automatically. However, when providing the library with the actual physical dimensions of the target images, the recognition of the image is much faster. In this vein, the physical width of the motifs was measured, and these values were fed to ARCore. Indeed, this action boosted the ARCore performance, and the application successfully recognised the patterns relatively quickly.

Knowledge representation and knowledge access

Part of the charm of the handbag is its ability to tell stories regarding patterns motifs and symbols. To achieve this, a systematic representation of social and historic knowledge was employed in the bag. This representation of knowledge and its transformation into narratives is achieved through the systematic application of the Mingei online protocol (Zabulis et al. 2019) to represent the craft of textile weaving in Krefeld. To this end, the Mingei online platform (MOP) was used to semantically represent cultural and socio-historical context encompassing a given focal topic of interest, such as a heritage object, collection, site or practice. The achieved socio-historical representation through the usage of the MOP was employed in the practice in the contextualised presentation of the given topic, through narratives that support its explanation to diverse audiences. More specifically, the platform facilitates the formation of knowledge elements for persons, events, objects, enterprises and communities. Each element category is accompanied by a data collection form that allows users to add, modify and delete the respective elements. This makes possible the formulation of a space-time continuum for the social and historical context of a given craft (Zabulis et al. 2020). Figure 6 presents some examples of represented knowledge of textile weaving in Krefeld.

Sequences of events and their presentation provide the means for the definition of (a) fabulae that are considered the sequence of historic events contributing to a story, (b) narratives that are stories based on a fabula, (c) narrations that are different forms of telling the same story and (d) presentations that define how the story will be told according to the target audience and technology used. Historical narratives can be exported and presented in a textual form by the AR app or previewed as multimodal documents through MOP. An example of the latter case is presented in Figure 7 and regards the story of textile weaving in Krefeld.

Formulation of links between textiles and presentations

Each motif used for the formulation of the handbag has to be connected with a presentation of the story. The presentation is authored on the online platform and contains presentation segments. Each presentation segment is linked to media objects presented by the mobile app. Examples of media objects include images, 3D reconstructions and videos. The following table (Table 2) presents an example of a pattern connected to a 3D reconstruction of a garment and thus is linked with a presentation segment.

User experience

Each motif has its history – a series of symbols that evolved in time and were artistically woven together to form traditional ecclesiastical fabrics. A few years ago, some of these motifs were still being manually woven on wooden Jacquard handlooms, which were up to two hundred years old. Our AR application provides three layers of pattern recognition. The first layer regards the bag, which has its own story and creation. The second layer regards multiple patterns, which reveal the story of textile weaving in Krefeld. The third layer encompasses the stories of the patterns (see Figure 8).

The AR app augments the virtual space with a canvas to present those stories. Furthermore, as these stories relate to the history of Europe and the social and historic dimension of textile weaving, the bag is also a portal to a web of information provided through the MOP. To reach this information, a button with the words ‘view more info’ navigates the user to the respective webpage of the MOP – where the journey in European history begins.

Technical validation

To validate the combination of a bag and AR app, an experiment was conducted on the premises of FORTH- ICS AmI research facility. In this experiment, three set-ups were selected: (a) one with standard lighting from the ceiling-mounted lights, (b) one with ambient lighting from augmented wall projections and (c) one with a combination of standard and environmental lighting. During the evaluation, the recognition rate of the AR app was tested in these three environments and at various distances from the artefact. Several iterations were made to adjust the image database to various illumination scenarios and thus to enhance the recognition accuracy and reduce the recognition time. A demonstration can be accessed through the video presented in Figure 9.

Conclusions and future work

In this paper, we presented an approach for the augmentation of a handcrafted contemporary handbag that integrates historical patterns used by HdS in Krefeld, Germany. The design of the bag was challenging in terms of aesthetics and pattern integration; its design was supported by theory on geometric abstraction and visual aesthetics. At the same time, the AR app was an interesting development in terms of pattern recognition, multilayered information provision and the extension of the experience to wider sources of information. The outcomes were technically tested in an experiment held in various internal spaces with lighting variation to validate recognition accuracy and speed.

Overall, the authors argue that this process has proven their hypothesis that digital technology can be gracefully combined with traditional handmade artefacts to provide the possibility for a new form of entrepreneurship concerning the history and tradition of TCs and, at the same time, acknowledgement of the role of digital technologies in modern societies. Currently, the final artefact is intended to act as a physical exhibit at the museum, where it will be experienced by visitors as a storyteller. Furthermore, elaboration of the AR app will be done through the user-based evaluation, which will accompany the pilot instantiation of Mingei technology in the museum.


This work has been conducted in the context of the Mingei project, which has received funding from the European Union’s Horizon 2020 research, and an innovation programme under grant agreement no. 822336. The handbag was created voluntary by Constantina Doulgeraki, who is a computer scientist that enjoys sewing as a hobby.