README.md



HMI-remote
Universal electromechanical remote control for touchscreen user interfaces (UIs)
HMI stands for Human-Machine Interface; it's a term from the controls industry.
A well-documented parallel effort from Manu's group: https://docs.google.com/document/d/1Z44YpbulrfZ0txfAJZJqTHFGj67I7Y-cv655t-TL5Mc/edit

status
On hold as of 4/19/2020 due to lack of need from local hospitals; many are now using mfr-supplied extension cables for ventilator HMIs.

design notes

requirements

security/reliability

cannot interfere with manual operation
no chance of accidental actuation
inherently secure -- i.e. requires line-of-sight, not just a password


adaptable to a variety of ventilators (and eventually, patient monitors/etc)

critical care ventilators on the market, starting from this list and adding:


Medtronic Puritan Bennett 980

15" touch screen (4:3)
one rotary encoder, bottom center


Maquet Servo-i

12" standard TFT LCD -- not a touchscreen (skipping for now)


Maquet Servo-u

15" touch screen (4:3)
14.4" x 11.8" x 2.0" display size overall
no other controls


Drager Evita Infinity V500

17" touch screen (16:9)
one rotary encoder, bottom right


Hamilton-G5

15" touch screen (4:3)
one rotary encoder, bottom right
several touch buttons below screen


GE Carescape R860

15" touch screen (4:3)
one rotary encoder ("trim knob"), bottom right


Philips Respironics V60

screen size unclear; likely <12", 16:9, portrait touchscreen
one rotary touch interface with button, top right


others?


common UI characteristics

relatively thin panel above ventilator unit
large touchscreens
zero or one rotary controls (occasionally more)
zero or several buttons (type? capacitive, membrane, tactile, etc?)


usability

if video feed is included (debatable), high enough resolution/refresh rate/clarity to view UI
fast-fast-fast response: needs to hit desired button and get out of the way

any delay/latency is worse than IRL HMI use and must be minimized
system must "rest" out of frame to allow manual use and clear view of controls


fast/easy pairing with minimal training
should use available devices: iOS/Android phones
no app download, minimal instructions required (should be intuitive)


functionality

remote actuation of all typical UI features required for patient care: touchscreen and any regularly needed hardware controls


fabricatability

design around typical FabLab inventory and fabrication capabilities


HW/FW design thoughts

parallel SCARA

vs cartesian

faster
smaller attachment point to machine
potentially lower cost (no linear actuation/alignment needed)


ESP32+camera

low-cost, low power consumption, ubiquitous
probably can't handle video bandwidth, maybe good for a no-camera option
alternative: RPi ZeroW


connectivity

Bluetooth

secure
pairing infrastructure exists


WiFi

easy
can act as access point w/ line-of-sight verification


4/15 call notes

Possible Goals

Without relying on hospital wifi
Establishing pairing - simple flow + security
Simple enough / fab-able amenable to simple IRB-approved hospital trial


Interfacing

Likely: low level motor control (ATTiny) → serial → UI / Networking stuff (ESP32-CAM / Pi)


Connecting to outside the room

Phones
Bluetooth (website via WebBT) or app
WiFi AP (website)


Pairing

Line-of-sight
Ultrasonic pairing (Amazon buttons)
Open up AP for pairing
QR code on-device?


Security

Requiring in-person


Hardware constraints based on FabLab inventory: https://docs.google.com/spreadsheets/u/0/d/1U-jcBWOJEjBT5A0N84IUubtcHKMEMtndQPLCkZCkVsU/pub?single=true&gid=0&output=html
Stanford project: https://docs.google.com/document/d/1Z44YpbulrfZ0txfAJZJqTHFGj67I7Y-cv655t-TL5Mc/edit
Q: Solidify what hardware stack looks like
Is ESP-32 sufficient for this?
May need closer to camera at 1080p @ 30fps - Tiny Raspberry Pi?
Q: What does interface framework look like?
How do we get data from web interface to high level controller?