Skip to content
Snippets Groups Projects

HMI-remote

Universal electromechanical remote control for touchscreen user interfaces (UIs)

HMI stands for Human-Machine Interface; it's a term from the controls industry.

A well-documented parallel effort from Manu's group: https://docs.google.com/document/d/1Z44YpbulrfZ0txfAJZJqTHFGj67I7Y-cv655t-TL5Mc/edit

status

On hold as of 4/19/2020 due to lack of need from local hospitals; many are now using mfr-supplied extension cables for ventilator HMIs.

design notes

  • requirements
    • security/reliability
      • cannot interfere with manual operation
      • no chance of accidental actuation
      • inherently secure -- i.e. requires line-of-sight, not just a password
    • adaptable to a variety of ventilators (and eventually, patient monitors/etc)
      • critical care ventilators on the market, starting from this list and adding:
        • Medtronic Puritan Bennett 980
          • 15" touch screen (4:3)
          • one rotary encoder, bottom center
        • Maquet Servo-i
          • 12" standard TFT LCD -- not a touchscreen (skipping for now)
        • Maquet Servo-u
          • 15" touch screen (4:3)
          • 14.4" x 11.8" x 2.0" display size overall
          • no other controls
        • Drager Evita Infinity V500
          • 17" touch screen (16:9)
          • one rotary encoder, bottom right
        • Hamilton-G5
          • 15" touch screen (4:3)
          • one rotary encoder, bottom right
          • several touch buttons below screen
        • GE Carescape R860
          • 15" touch screen (4:3)
          • one rotary encoder ("trim knob"), bottom right
        • Philips Respironics V60
          • screen size unclear; likely <12", 16:9, portrait touchscreen
          • one rotary touch interface with button, top right
        • others?
      • common UI characteristics
        • relatively thin panel above ventilator unit
        • large touchscreens
        • zero or one rotary controls (occasionally more)
        • zero or several buttons (type? capacitive, membrane, tactile, etc?)
    • usability
      • if video feed is included (debatable), high enough resolution/refresh rate/clarity to view UI
      • fast-fast-fast response: needs to hit desired button and get out of the way
        • any delay/latency is worse than IRL HMI use and must be minimized
        • system must "rest" out of frame to allow manual use and clear view of controls
      • fast/easy pairing with minimal training
      • should use available devices: iOS/Android phones
      • no app download, minimal instructions required (should be intuitive)
    • functionality
      • remote actuation of all typical UI features required for patient care: touchscreen and any regularly needed hardware controls
    • fabricatability
  • HW/FW design thoughts
    • parallel SCARA
      • vs cartesian
        • faster
        • smaller attachment point to machine
        • potentially lower cost (no linear actuation/alignment needed)
    • ESP32+camera
      • low-cost, low power consumption, ubiquitous
      • probably can't handle video bandwidth, maybe good for a no-camera option
      • alternative: RPi ZeroW
    • connectivity
      • Bluetooth
        • secure
        • pairing infrastructure exists
      • WiFi
        • easy
        • can act as access point w/ line-of-sight verification

4/15 call notes

  • Possible Goals
    • Without relying on hospital wifi
    • Establishing pairing - simple flow + security
    • Simple enough / fab-able amenable to simple IRB-approved hospital trial
  • Interfacing
    • Likely: low level motor control (ATTiny) → serial → UI / Networking stuff (ESP32-CAM / Pi)
  • Connecting to outside the room
    • Phones
    • Bluetooth (website via WebBT) or app
    • WiFi AP (website)
  • Pairing
    • Line-of-sight
    • Ultrasonic pairing (Amazon buttons)
    • Open up AP for pairing
    • QR code on-device?
  • Security
    • Requiring in-person

Hardware constraints based on FabLab inventory: https://docs.google.com/spreadsheets/u/0/d/1U-jcBWOJEjBT5A0N84IUubtcHKMEMtndQPLCkZCkVsU/pub?single=true&gid=0&output=html

Stanford project: https://docs.google.com/document/d/1Z44YpbulrfZ0txfAJZJqTHFGj67I7Y-cv655t-TL5Mc/edit

Q: Solidify what hardware stack looks like Is ESP-32 sufficient for this? May need closer to camera at 1080p @ 30fps - Tiny Raspberry Pi?

Q: What does interface framework look like? How do we get data from web interface to high level controller?