Working in IT, I’ve been talking about unstructured data for years. We know it’s growing faster than data in databases, we know much of it is machine-generated, and we know there are several emerging technologies to help manage it.
But one of my Achilles’ heels at work is having never been on the customer side of the table in enterprise IT. I’ve been talking with customers for 15+ years about storage and data management, but I have never been the person actually responsible for it.
In the past few weeks, however, I’ve realized that I have a major unstructured data management problem in my own life – and it’s giving me a taste of what IT management struggles with on a massive scale.
Let’s just consider two important types of files in my life: my music, and my photos. For a few years, I’ve been syncing every song and every photo to the cloud (for music, first iTunes, now I’ve moved to Amazon; for photos, Google). So I don’t have a storage space problem, and I don’t have a portability problem – everything is accessible from each of my devices.
The music is organized by artist and by album. And the photos are organized by date (and sometimes by trip). This works ok when I know what I’m looking for and it’s aligned to these structures. For example, these types of things are easy to find:
- Pictures from our trip to Prague and Budapest
- The Tori Amos Little Earthquakes album
- The photo I took of Curt Schilling at the Red Sox Parade in 2004
- Let me Clear my Throat by DJ Kool
But there are times I want other kinds of things, and these are nearly impossible to find:
- The best 5 photos of my daughter since she was born
- All the photos of my dog
- 30 songs I like from different artists that I haven’t listened to lately
- A playlist-style playlist of good songs to work out to.
For music, I do have a few other options – I have Amazon Prime, so there are a lot of playlists of music I may or may not otherwise have in my library that i have access to. So, I recently chose to listen to a playlist called “90’s Alternative Rock” and it was like being transported back to college. It was exactly what I wanted. But that was a kind of lucky hit. And what about something as effective like that, but with my existing library that I have acquired and culled over the years?
I can see why people hone Pandora stations and stop buying individual songs or albums. But even if that worked for me (and I’m not ready to give up my music library just yet) I can’t do that with my photos. Google’s been doing a nice job with “highlights” and with making animated GIFs and collages from my photos, but even those aren’t as focused as what I can get from music.
I could tag every photo (and song, I suppose, or at least put songs on playlists) to help with this. But I don’t always know when I take a photo (or tag a photo) the ways in which I might want to look for it. I could tag all the people in photos, but how would I know that I would have wanted to tag something as “outside” or “old apartment” until I want to retrieve it that way.
I would love any suggestions you readers have about solving either of these problems, but mostly I’m enjoying the parallels between this and enterprise IT.