Arduino datalogger 2012
Lifeloggers, people who wear computers in order to capture their entire lives, including physical movements are not new. My first introduction to lifelogging was around 1976 when St Mary's Hospital, Manchester approached me re a design for a device to record the activities of hyperactive children. The device would attach to the leg and measure and count the activities of the children. I discussed the use of mercury switches and TTL counters, as there were no low cost microcontrollers or small accelerometers. I was fascinating by this, and was spending a lot of time designing dataloggers, particularly for digital audio.
Here is an example of a similar lifelogger using a mercury switch and TTL logic to count from this 1978 patent.
The band wrapped around the leg of the child.
Click to Enlarge
1980, I worked at British Aerospace on noise reduction systems for The Queen's Flight.
Here is an example of the accelerometer used to measure vibration of the out of balance propeller, the size of a tin of food.
Moving forward to 1998, I joined Microsoft Research, Cambridge. They were interested technology for a sensing pen I had designed, SmartQuill, but the patent was later bought by Apple for the iPhone. This was a lifelogging pen, that would record all your written words, and has sensors built in like a camera, tilt sensors to orientate the display and detect the person's physical activity using accelerometers, eyeball tracking, heat and touch sensors.
SmartQuill handheld computer 1997 - click to enlarge
In 2000, I designed SmartMoveX. This was a lifelogging device I wore for 23 hours/day (leather is comfortable) for months to detect and log body movement using an accelerometer, Galvanic Skin Response, light and heart rate. I used it while working, sleeping and cycling. It would send me alerts if it detected inactivity and laziness for more than 30 mins. It was a good way to reduce a person's weight but the novelty of wearing the device wore of after about a year. The device really needed to be smaller. There is more data in the PowerPoint link including some metrics. You can buy devices with similar functionality now from Nike etc.
SmartMoveX 2000
Time to add a camera
At about 2000 I had an inquiry from a friend who was a dentist and wanted a time and motion study recording but also including images of the performance of the surgeon re time sitting down and standing up and minutes spent on surgery. I also wanted to help a friend who was always losing his keys. I needed to design a lifelogging device with a camera that would photograph every room as he walked through the door frame and so capture a sequence of rooms that had been walked through and so trigger his memory. I had also lost my memory for many months, when I was at school, due to a road traffic accident so knew the frustrations of have no recollection of the last few months of my life. (Memory was so bad I could not remembered if I had attended a Rick Wakeman music concert, that had occurred a month before my accident, normally a concert is etched in the memory) I also wanted a device that could predict what events in people's life would trigger illness like migraines. The SenseCam was like a black box data recorder for the human body, recording motion, temperature, location, images. Heart rate was done externally.
Kodak DC20 camera
I dismantled the DC20 camera and checked the electronic shutter circuit. I designed a circuit based on a PIC 8 bit microcontroller to control the shutter using a MOSFET transistor on based on motion interrupts from an Analog Devices accelerometer. Triggering images using sensing enables much more useful interesting images than timelapse which can miss 99% of events. Sensing also allowed the use of limited RAM memory to take more pictures. I connected it to my bike basket and got some fairly unexpected pictures in Cambridge, cars pulling up in front of me and startled babies on pushchairs as mothers pushed them into the road in front of me! The images were only captured with a narrow degree of view, so a lot of pictures of sky or the road. I put the research on hold as no suitable lens.
An early sensecam, no wide angle lens or PIR
For the Sensecam I decided I needed a fish eye lens to increase the field of view. After a lot of experimenting with door peep holes lens and similar I eventually sourced a reasonable wide angle lens (130 degrees) similar to this. A very wide angle lens also has the advantage of a depth of field from around 6 inches to infinity so no focusing needed.
wide angle lens 130 degrees
I dismantled the camera, removed the original lens and replaced it. I removed the infra red filter and so could capture IR that the eye could not see. See below for image of King's College, Cambridge showing the chlorophyll of the green grass but as white. Here is the video.
By 2004 the rather neat Philips wearable camera was on the market so I used this for tests.
Later Sensecams were based on the camera module from inside this Philips Key wearable camera
I also decided that capturing pictures could be triggered by other life events, e.g. a person moves in front of the lens, so I used a 10mm diameter passive infra red detector to detect the heat from a person. A Red Green Blue light sensor was also used to trigger images based on light changes, e.g. walking through a door frame. I tried a temperature sensor but after around 20 mins this was warmed up by the body and showed the user's temperature. I did not want to capture audio as an invasion of privacy, hard to compress the listening experience, and there were also third part devices that did record sound.
I recorded GPS using an external unit as these were low cost and the data could be synced via a time stamp. I also knew there would be problems in battery life, getting a very small GPS antenna in the same case as a camera and power supply without electrical interference. There were other problems to be overcome, e.g. the camera module took 150mA to capture a picture which curiously is curiously just as high as the current Omnivision camera modules. An 1.5V AA battery with voltage converter allowed 5 hours of battery life.
There are cameras today, e.g. Autographer, (uses the above SenseCam patented Microsoft technology) and Memoto, due to ship soon.
It is possible to build you own lifelogging camera, an Arduino can be used as a controller, with sensor triggers, and used to control shutter of any camera. However there are a few small cameras on eBay that are wearable and can be used.
In 2007 I designed and built a life logging lightbulb, SenseBulb, that tracked the heat from a person and inferred their activities from heat movements.
Later Sensecams
I have designed more advanced Sensecam's that use heat sensors to detect the heat a few feet in front of the camera , and this can detect the heat from hands moving in front of the camera, people, food, and so trigger an image. I also use infra red system to capture what the eye cannot see. The device is clipped onto the chest like a nurse's watch and can also capture respiration, (GSR) and heartrate.
Audio recording
There is also a version that records audio but compresses a day's audio into approximately 5 minutes, so just an impression or montage with no invasion of privacy. It will replay doors closing and opening, trains,cars, laughter, birds, cocktail conversation etc and so an audio summary of the day. These can be played in synchronisation with the video or as a stand alone impression of the day.
I am testing this at present, Sept, but some work needed on the speech algorithms.
Below is an example of a BBC radio play which I am processing today with Audacity to compress and remove the dull parts. (The Archers Sept 2013).
click to enlarge
I am also (2013) working on Sentinel, a device to predict or avoid early death caused by illness or accident. The user does not even need to have it close to them to capture measurements. Details under NDA.
Sensecam is a registered Trade Mark owned by Lyndsay Williams.
Lyndsay Williams, sensecam@gmail.com
No comments:
Post a Comment
Comments will be moderated