One Scene

Sarah B, Alice C, and Eric F

slides

Concept:

  1. To create a non-human subjectivity through the assemblage of semantics [human] and syntactic [machine] components.
  2. The human curates material and interprets meaning.
  3. Machine generates and learns to manipulate human.

Neural network system as a submissive-responsive controller of the virtual space

  • A default scene -> transition to different states
  • “Branching” of the state based on user gaze, head movement
  • Similar to recommendation system
  • User-feedback data -> generative 3D visuals and sounds

Implementation

Model/Technique

Unsupervised Learning Deep Q Learning / reinforced learning?

Input

  • Form: head direction (coordinate in 3d-space), head movement (acceleration data), verbal cues, questionnaire
  • Dataset: have to run experiments, create one

Output

New state:

  • Generate/eliminate object
  • Change density/mood of sound
  • Change timing/pace of transition
  • Change mood of the background
  • Change time-space orientation

Platform

Unity + C#?

Modules

Generative module

  • Visual synthesis
  • Sound synthesis
  • Either using ML technique or rule-based (procedural)

Training module: clustering

  • Input: Viewer’s action/behavior
  • Output: New state

End Goal:

Complete self-alienation!

Evaluation

Time spent in the VR space, User comments