source

Driver assistant app. Can it be done?

I was too optimistic about making this work on Android since it takes more than a couple of seconds to process even single frame. So folks doing what I hoped in this post with OpenCV  is currently not achievable with a mobile phone.

This video which had 30 fps and was 11 seconds long took about 22 minutes to process.

I wonder why is that there is close to none Android or IPhone applications that can in real-time detect erroneous drivers driving on the road before or sideways to you. The technology is there and algorithms, namely Deep Learning is there too. It is possible to run OpenCV based deep learning models in real-time on mobile phones and get good enough performance to detect suddenly stopping car ahead of you. Since mobile phone field of view isn’t that large, I think it will be hard if not impossible to detect erroneous driving on the sides of the car. A good example of OpenCV based object detection and classification using Deep Learning may be Mask R-CNN with OpenCV post by Adrian Rosebrock from PyImageSearch.

Requirements

• Hardware
  • Decent average mobile phone 
• Software
  • Operating system
    • Andorid or IPhone
  • Object detection and classification
    • OpenCV based approach using built-in DL models
• Type of objects classified
  • Car
  • Truck
  • Bus
  • Pedestrian (optional)
• Number of frames per second
  • Depends on the hardware. Let’s say 24. 
• Field of View 
  • About 60 degrees
• Type of erroneous driving detected
  • Sudden stopping
  • Zigzag driving
  • Cutting off from the side (hard to do with single forward facing phone camera)
  • etc.

Then what are we waiting for?

This application can be built quite ‘easily and fast’ if you have an Android developer account, had an experience developing an Android apps. You worked a little bit with GitHub and had a certain amount of experience and fascination with machine learning, namely OpenCV DL based models. To be able to detect some dangerous maneuvering others do there is a need to use a little bit of math to be able to detect them, as well as calculate speed, direction and distance to other cars. The effort is worth investing time into. Even a little helper can have a big impact, unless drivers start staring into the mobile phone screen looking how it’s doing while driving.

A possible plan of action

• Get a laptop that can run OpenCV 4 and Android Studio.
• Install OpenCV and all needed dependencies.
• Run the example from Adrian Rosebrock’s blog post.
• Adapt the Pyhton code into Java using as a template this blog.
• Install Android Studio.
• Create a Android developer account (if you don’t have one, about $25 USD)
• Use the Android app from above blog post as a blueprint.
• Add distance, speed, direction detection to the application (the hard part)
  • This recent paper may be useful too Orthographic Feature Transform for Monocular 3D Object Detection
• Publish the app at Google Play Store (Check that there is no legal problems if someone using the app will get into car accident)
• Share the app.