It was the summer of 2014. I was well into my big data addiction thanks to Splunk. I was looking for a fix anywhere: Splunk my home? Splunk my computer usage? Splunk my health? There were so many data points out there for me to Splunk but none of them would payoff like Splunking my driving…
Rocky Road
At the time, my commute was rough. Roads with drastically changing speeds, backups at hills and merges, and ultimately way more stop and go than I could stomach. But how bad was my commute? Was I having as bad an impact on the environment as I feared? Was my fuel efficiency much worse than my quiet cruise-controlled trips between New York and Boston? With my 2007, I really had no way to know…that is, until I learned about Automatic.
Automatic is a dongle that goes into your On Board Diagnostic (OBD) port. This port is hiding in plain sight – typically right under your steering wheel – but only your mechanic knows to look for it. The OBD port is what the mechanics use to talk to the car during service. It turns out there’s a ton of information available through that USB-like port and there’s a slew of new devices and dongles (for which Automatic is one of) that expose that information to you. Combine that car info from what your phone is capturing of the world around you and you got yourself a delicious data stew!
The Dataman Cometh
Out of the box, Automatic provides really cool details about every trip it records: fuel efficiency, route path, drastic accelerations, sudden brakes, periods of speeding, and so on. This is all accessible for review on your smart phone. A little bit hidden is that Automatic provides a dashboard where all of your trips can be seen in aggregate (http://dashboard.automatic.com). This dashboard allows you to see some basic aggregate statistics (sums and averages) for a selected time period of trips. I’m sure you see where I’m going with this…
I wanted more from this data. I had already been spoiled by the power of Splunk and I knew that if I could Splunk this data, I could do so much more. That’s when I noticed in the bottom right corner of the dashboard an export option. EUREKA! I immediately downloaded the resulting CSV of my trip data and got to work adding it to Splunk.
Getting Dizzy with Vizzies
I created each and every search that immediately came to mind: total miles driven, total time behind the wheel, total number of trips. Then I got into the visualizations: number of trips over time, fuel efficiency trends relative to instances of poor driving (sudden brakes, fast accelerations, or speeding), fuel prices over time, and location of the cheapest gas observed.
Basic aggregate analytics of Automatic data within Splunk.
As I started seeing my data represented visually, I was reminded of something from Dr. Tom LaGatta’s .conf2014 talk “Splunk for Data Science” (http://conf.splunk.com/speakers/2014.html#search=lagatta). He spoke about how the human brain processes data more effectively in a visual manner. I was now seeing exactly what he meant! My data was no longer a table, or a graphic for a single trip. Instead I was able to use my entire trip history to create visualizations that demonstrated my driving trends and the impact of my behaviors on fuel efficiency – things I would never have captured by looking at individual events, nor by reading a spreadsheet of numbers. With this new enthusiasm, I took Dr. LaGatta’s premise to the max and used a D3 Chord Diagram to represent my travel frequency from zip code to zip code.
Density of zip code transits represented with a Chord Diagram
Dynamic Data
After posting this collection of insights as an app on SplunkBase, I met a wise Splunk Ninja named Todd Gow. Todd showed me the ways of the force by using Damien Dallimore’s REST API Module Input to pull my Automatic data perpetually, rather than solely through manual CSV exports from dashboard.automatic.com. Thanks to Todd and Damien’s coaching, as well as some support from the developers at Automatic, the app was now able to pull user’s driving behavior automatically.
This was a game changer. Not only did it ease the import of data, but it meant that users of Automatic and Splunk could now create Splunk alerts based on driving performance! Alerts that highlight everything from erratic driving, to poor fuel efficiency, to notifications that the vehicle has gone outside its typical geographic area (stolen?). The possible insights were growing now that new data could be analyzed against historical performance.
“Where we’re going, we don’t need roads”
Since Back to the Future II’s prediction for 2015 was off, let’s conclude by covering some final insights achieved thanks to Splunk + Automatic:
Besides the metrics mentioned above, I was able to calculate some interesting details about my fuel efficiency. Automatic provides audio tones to alert the driver to speeding, drastic acceleration, and sudden braking. This produced a Pavlovian response for me such that, over time, I could feel myself adjusting my driving behavior in response. So this led me to wonder: Has Automatic saved me money on fuel by adjusting my driving behavior? Thanks to Splunk, I was able to calculate such an answer!
First I calculated my average fuel efficiency from when I started with Automatic till now. Since each trip with Automatic includes fuel consumption and fuel prices (as calculated by prices local to my trip location), I was able to compare the fuel efficiency increase to the average fuel prices and provide an estimate of money saved. Considering the cos of Automatic’s dongle, I concluded that I MADE money by spending less on fuel thanks to my improved driving behavior!
Fuel efficiency data from Automatic represented in Splunk
To learn about the search techniques to draw these, and other, conclusions, download the app from SplunkBase and check out the dashboards for yourself.
“Get Outta My Dreams, Get Into My Car”
With my Automatic driving data in Splunk, my mind won’t stop “racing” with new insights to implement. In addition to the alerts proposed in this post, I plan to provide insights into vehicle usage for wear and tear, route path details, stronger fuel economy calculations, and webhook incorporations.
Before any of that, I have to finish what has become a complete app re-write. I’ve been rewriting the data ingestion modular input to both simplify app configuration as well as take advantage of the new Automatic API and its new field names.
Down the “road”, I hope to collaborate with the Common Information Model team here at Splunk to define a vehicle-based data model. That way, users of any vehicle-related data capture (such as Automatic) could take advantage of the Automatic app on SplunkBase and get the same insights, regardless of the data’s differences in field names and formats.
Of course, if you’ve found cool insights into your driving data, post it below and share your discoveries! Let us know how you used Splunk to make your “machine data accessible, usable and valuable to everyone.” Thanks for reading, drive safe, and happy Splunking!