Home/Research

The Neuramod project aims at assessing a new modeling approach in the early design phases involving human and artificial intelligence to generate meaningful architectonic articulations from visual inputs and their correlated neural responses. Its goal is to contribute to the next generation of grammar-less modeling methods, which should allow for more significant variance and fluency in design explorations and iterations. By researching BCI for generative architectural modeling, this project elaborates progressively on a technical infrastructure, allowing for the production and evaluation of visuospatial and physical experiments.

As a Use-Inspired and Basic Research project, it aims to establish significant basic theoretical and technical knowledge, which, by its modalities, might also address a much larger audience than expert practitioners and enlarge the accessibility and inclusiveness spectrum of digital design. Its development scope allows for many use cases and creates an ideal ground for near-future implementations through other technology transfer mechanisms. The project’s schedule is organised in two main phases progressively and iteratively. While the first phase involves integrating multidisciplinary technical and theoretical knowledge from the scientific fields in CAAD and Cognitive Science into a modeling software prototype, the second phase focuses on implementing found methods for modeling architectural prototypes at a small scale.

A significant part of BCI research has, so far, been focusing on interactions using exogenous responses under selective attention with synchronous and reactive methods for clear relation with controlled stimuli and the widespread applicability of the methods. Over the past two decades, ERP have become increasingly investigated by a broader community of researchers due to the relatively little training necessary for a system to perform and their detection across diverse acquisition modalities to correlate with the sensory discrimination of dedicated stimuli. However, the relatively simple design of experiments for the sake of controlled results has also been a barrier to advances in further applications.

The proposed study aims to observe the detection of ERP components and correlated neural phenomena under the visual presentation of complex stimuli and devise processing methods that would generalize their classification and cardinality for applications in CAAD, where visual complexity becomes an intrinsic feature of the tasks. Its objective is exploratory and twofold: evaluating data processing and stimulus presentation methods and the evolution of similar responses in repeated measures intra- and inter-subjects in non-clinical states.

Covert responses in physiological data may allow for a more significant variance than behavioural ones to maintain polysemy in creative design processes. However, a systemization of the comparative process is necessary for evaluation. To bring more variance, an interactive and generative model that involves and leverages two kinds of intelligence (artificial and human) should frame the informational flows to create an adaptive and mutually beneficial loop that modulates each other’s beliefs towards particular tasks. The GRAD quadrium model (Generation, Rendering, Acquisition, Discrimination) represents such intent in conjugating artificial intelligence and intelligent humans.

An inverse model is required in the form of Vision-As-Inverse-Graphics to generate complex scenes from basic perceptual features, as in the efficient processing of natural scenes found in vision. While an early visual model represents the generative processing of natural scenes synthetically, its inferential mode represents the analytic portion. This principle is generalized to connect natural and synthetic compositions. It constitutes an adequate computational model for synthetically generating scenes from perceptual features that would be processed naturally at a fast pace.

For operative reasons, a necessary reductionist approach must occur at the articulation of sensory inputs digitized into signals and the classification of found features by AI methods to establish a fluent interaction. Inputs from the visual domain involve complexity and dimensionality, making it hard for steady reproducibility in BCI experiments except when brought down to reduced controllable features. However, a handful of separable classes should be involved in particular sensory discrimination under a logical framework of primary order once provided with a standardized presentation of these stimuli. Their cardinality should remain independent from the informational complexity of the stimulus despite being factored by cognitive parameters.

The Neuramod project is fully funded by the Swiss National Science Foundation  SNSF Project Funding Div. 1-2 for Humanities, Social sciences, Mathematics, Natural and Engineering sciences. It is hosted at the Institute of Technology for Architecture (ITA), Chair of Digital Architectonics (CAAD) of ETH Zürich. Its two main individual partners are Dr. Ricardo Chavarriaga from the Center for Artificial Intelligence (CAI) of ZHAW Zürich, and Dr. Marco Congedo from the Vision and Brain Signal Processing Group (GIPSA) of CNRS Grenoble. Further secondary partnerships might be sought along the project for complementing advisory, scientific or industrial expertises.

Throughout the title: Neurodesign, Modeling with Neural Potentials, the term neural potentials means twofold: potentials as in electric potentials, vessels of information to be decoded, and potentials as in capacity held by such information. The thesis comprises three main chapters, its conclusion and an appendix containing publications and research materials produced along the research. Its general intent reflected throughout the chapters is to balance the research between empirical and theoretical findings to mutually inform each other on the prototyping of potentially novel ways to approach architectural modeling and its encoding.

Interfacing the human brain with computers requires a synthetic approach to technical and theoretical advances in cognitive science and the necessary understanding of the appropriate methods to contribute to novel compositional approaches in architecture, which intrinsically embeds informational complexity at its core. To support and feed the research with continuous experiments, a platform-and-hardware-agnostic software prototype has been developed along with several custom sensors. Programmed within a modular and integrative framework, it gathers the entire technical pipeline necessary for the interactive and generative approach of design modeling. From the rendering and presentation of generative design solutions as visual stimulations to its synchronized data streaming and acquisition, its processing and analysis, the concurrent training and use of discriminative and generative estimators to finally identify significant discriminative neural patterns and iteratively generate design objects, this software allows for continuously adding and evolving more features along the research.

Stargazer, Two Columns From Neural Potentials, is a sculpture about digital architectonics from an extensive series of physical aggregates produced from the technology developed along the research. It employs techniques and theoretical arguments to propose operative means to articulate parts using combined human and machine intelligence. The produced mixed model involves human intelligence for discriminative capacities found in neural phenomena and artificial neural networks as perpetual generators of new potential design solutions to explore. Doing so challenges the syntactical tradition of the generative in architecture while reappraising information and communication theories and permuting the idea of parts with tokens, grammars with beliefs, and assemblies with aggregates. The two columns manifest a significant step in this new modeling approach. They can be seen as a dialectic between the potential of human intelligence to create meaning by indexing things in the world to categories and concepts in the mind and the potential of machine intelligence to decode latent and manifold structures.

This article will focus on abstracting and generalising a well-studied paradigm in visual, event-related potential-based brain-computer interfaces for spelling characters forming words into the visually-encoded discrimination of shape features forming design aggregates. After identifying standard technologies in neuroscience and neuropsychology of high interest for integrating fast cognitive responses into generative design processes and proposing the machine learning model of an ensemble of linear classifiers in order to tackle the challenging features that electroencephalography data carry, it will present experiments in encoding shape features for generative models by a mechanism of visual context updating and the computational implementation of vision-as-inverse-graphics, to suggest that discriminative neural phenomena of event-related potentials such as P300 may be used in a visual articulation strategy for modeling in generative design.

This paper focuses on applying ERP in better design and architectural modeling generalisations. It uses previously built techniques and trained models on EEG signals of a singular individual. It observes the robustness of advanced classification models to initiate the development of presentation and classification techniques for enriched visual environments by developing an iterative and generative design process of growing shapes. The pursued interest is to observe if visual ERP as correlates of visual discrimination can hold in structurally similar but semantically different experiments and support the discrimination of meaningful design solutions. Following Bayesian terms, we will coin this endeavour a Design Belief and elaborate a method to explore and exploit such features decoded from human visual cognition.

This research investigates BCI technologies in the broad scope of CAAD applications exploiting early visual cognition in computational design. Firstly, it introduces the pursued interest and contribution that visual-ERP EEG-based BCI application for generative design may provide through a synthetic review of precedents and BCI technology. Secondly, it describes selected BCI components and presents a methodology to provide an appropriate framework for a CAAD software approach. It distinguishes two critical aspects of discrimination and generation in its design and proposes a new model for modulated adversarial design. Emphasis is on the explicit use of inference loops integrating fast human cognitive responses and their capitalisation over time to reflect on the generation of design and architectural features.

This paper describes the work in progress of research on Neurodesign and a previously described process entitled Augmented Iterations (Cutellic & Lotte, 2013). Within this broader field of research and design explorations, we will focus on describing and commenting on results obtained in the neuro-selection of shapes among a continuously varying flow of visual stimuli. Eventually, these results will lead to identifying a non-linear and non-convergent potential for the Human-Based Computation of iterative and generative designs.

The principle of Augmented Iterations is to create shapes of progressively higher complexity, thanks to a fast neuronal selection of shapes among several possible evolving designs. This process is made possible using a brain signal known as P300, which appears when a user perceives a rare and relevant stimulus and can be used for intricate pattern recognition and human computation systems. We aim to use this P300 signal to identify the (re)cognition of shapes or designs that a user finds almost instantaneously relevant and noticeable when exposed to a rapid visual flow of variations of such shapes or designs. Using evolutionary algorithms, the shapes that trigger a P300 in the user’s EEG signals are selected and combined to give rise to geometrical aggregations of higher complexity. These new shapes replace the previous ones in the rapid flow of variations presented to the user, hence iterating the evolutionary design.

This text describes a series of simultaneous events both in computer science and architecture in the 70’s. From Las Vegas and its historical urban study derives a comparison of common concerns and a contemporary speculation on machine learning for human cognition.