← Index · Applied research
Applied research / 06 MIT Multimodal capture BERTopic · NLP Design science

How engineers actually collaborate inside shared CAD.

A study with MIT and the University of Toronto that reads how two engineers collaborate on one 3D model, not just from what they say, but from where they look, how they feel, and where their cursors go.

My role
Co-lead
With
MIT · University of Toronto
Methods
NLP · eye-tracking · facial affect · mouse logs
Study
40 participants · paired CAD

On this page. Based on published academic research with MIT and the University of Toronto. The visuals here are illustrative recreations of the study's analysis.

BERTopic projection · paired-CAD transcripts
Illustrative recreation

Discovered topics

Two collaboration styles

Shared CAD↑ quality
Designers take turns on the file. More coordination talk, fewer errors, better outcomes.
Parallel CAD↑ throughput
Both edit at once. Faster, but communication overhead rises and styles diverge.

Each point is an utterance from a paired-CAD session, embedded and projected with UMAP, then clustered with HDBSCAN under a semi-guided BERTopic model trained on the CAD corpus, turning raw conversation into legible collaboration topics.

01 · The problem

Synchronous CAD arrived. Nobody knew how to collaborate in it.

Synchronous CAD promised the effortless co-editing of Google Docs for mechanical engineering, two people shaping one 3D model in real time. But adapting design workflows to actually exploit it proved hard, because the language of CAD collaboration is new and the coordination is subtle. There was no clear picture of what good collaboration even looks like in a paired session.

02 · The method

Four signal streams, not just a transcript.

We ran an exploratory study with 40 participants in paired CAD, and we instrumented far more than the chat. Alongside the spoken transcript we captured eye tracking, facial-affect (emotion) detection, and mouse and cursor logs, so we could see attention, friction and hand-offs, not just words.

  • Speech / NLP, a semi-guided BERTopic model (UMAP + HDBSCAN) trained on the CAD corpus, surfacing the topics designers actually talk about.
  • Eye tracking, where each designer's attention went, and the moments the two converged on the same feature or pulled apart.
  • Facial-affect detection, the confusion and frustration that never make it into the words.
  • Mouse & cursor logs, the rhythm of who was driving the model and when control changed hands.
Speech / NLP Eye tracking Facial affect Mouse / cursor fuse + model collaboration patterns shared vs. parallel
Multimodal capture. Words, gaze, affect and cursor activity fused into one model of how a pair actually works together.

"Parallel work yields greater throughput; shared work yields higher-quality outcomes. The overhead, coordination and communication, had been almost entirely underexplored."

summary of the central finding
03 · The finding

Speed and quality come from different ways of working.

Shared CAD significantly increased communication frequency and surfaced identifiable collaboration styles. The clusters told a clear story: when designers take turns (shared), quality rises; when they work in parallel, throughput rises, at the cost of coordination overhead and diverging styles.

The nuance is in the task. For novel, open-ended design, shared work won clearly, the friction of coordinating was worth it, and the signals agreed: converging gaze and calmer affect on the hard problems. For routine, well-understood designs, parallel work won, the coordination cost simply wasn't earned, and the cursor logs showed two people moving fast and independently. One mode does not fit every task.

The practical output is a hybrid recommendation: let teams switch fluidly between shared and parallel modes by task type, support multimodal communication, and let users dial their own collaboration intensity, rather than forcing one mode on every job.

04 · Why it matters

This research became my industry playbook.

The deeper value was personal leverage. The same instinct, using domain-specific NLP to find structure in messy, specialized communication, is exactly what I carried into industry, extending domain topic models toward foundational language models for the factory floor. The CAD work is the academic root of the generative-AI systems I now build in production: a direct line from a research question to a deployed product.

It also speaks straight to industry tooling. As engineering goes remote, the findings argue for hybrid CAD that switches between shared and parallel by task type, attention-aware presence so teammates can see where focus is, and affect cues that flag when a session is quietly going sideways.

05 · Talks & publication

Where this work was shared.

ASME
Semi-Guided Topic Modeling of Synchronous CAD Collaboration
ASME IDETC/CIE, Design Automation Conference
Conference
MIT
From Domain NLP to Foundation Models
MIT, invited research seminar
Seminar
CSCW
Seeing is Believing: Communication & Collaboration Strategies in CAD
ACM CSCW 2023 · co-authored with MIT & University of Toronto
Publication
Next · Industry / 01
Manufacturing Copilot