Iris (🏆 Winner at UCSC CruzHacks 2019)

🏆 Winner at UCSC CruzHacks 2019: 2nd Place Tech Cares, Best Use of Google Cloud Platform. An AI powered virtual assistant that uses facial recognition to help Alzheimer’s patients combat memory loss.

Inspiration

  • We had the idea to use machine learning in order to solve a common problem for people diagnosed with Alzheimer’s: helping them to remember things.
What it does
  • Iris is a virtual assistant (and wearable device) that helps you remember people that you know and things you have to do by providing non-invasive cues through an intelligent voice interface.
How we built it
  • We used a Raspberry Pi connected to a camera, a Flask app as the back end, and an Angular app as the front end. We used the Microsoft Azure Face API for facial recognition as well as an abundance of Google Cloud Platform services to support our application: including Google Compute Engine, Google App Engine, Firebase Realtime Database, Google Cloud Storage, and Google Cloud Text-to-Speech.
Challenges we ran into
  • One problem our team on the backend ran into was learning new APIs and successfully incorporating them into our final product. The time crunch was also an issue, but due to teamwork we overcame this obstacle.
  • Another challenge was training the speaker recognition using snowboy, a DNN based hotword and wake word detection toolkit.
Accomplishments that we’re proud of
  • The final product looks amazing and we’re extremely proud to present a web app in the time allotted.
What we learned
  • We learned how to use the Microsoft Azure Face API, storing data into the realtime database, and images into Google Cloud Storage. We also learned how to perform realtime data updates on a web app using firebase.
What’s next for Iris.io
  • We would like to allow a user to benefit from our app with a mobile device. Being able to incorporate the hardware into a smaller more discrete device, like glasses and an earpiece, is also a future goal.
Check it out

Here’s some photos of our team, including the prizes we won:

 

Leave a Comment