Mining Sensor Data (Capturing to a File): Part 3

In Creating a Data Logger embedded application using the Intelligent Sensing Framework (ISF 2.1), my colleague Dave Munsinger introduced the Intelligent Sensing Framework (ISF) to create an embedded sensor application quickly with a Freedom development board. The next step in mining sensor data is the subject of today’s blog – capturing raw sensor data and metadata to a file. This will allow us to experiment with signal processing and machine learning algorithms later.

Our embedded sensor application handles configuring the accelerometer device and sampling raw accelerometer data on a periodic interval by making use of ISF’s I2C driver, bus manager (timer), and sensor adapter components. It also provides a well-defined serial communication protocol for a host device, in this case a PC, to receive raw sensor data using ISF’s command interpreter component. OpenSDA provides a serial-to-USB adapter to bridge communications with our PC.

Now that we have everything configured on the embedded side, we need something on the PC to communicate with the serial port and log data to a file. A serial terminal application, such as RealTerm, is always useful for an initial test to make sure you can open/close the port, send a command and receive a response. Most serial terminals can also log to a file, but this is not enough for our mining sensor data application…we need metadata!

Metadata is “data about data”. We want to capture the conditions under which the raw sensor data was collected. For example, in an intelligent stove top we want to detect if water in a pot is boiling. In addition to the raw sensor data, we’d like to know:

• What was happening (water in the pot was boiling, the pot was empty, etc.)
• Where the board was located (on the stove top, on a pot handle, etc.)
• Who collected the data
• What time the data was collected
• What board (FRDM-K64F) and sensor (FXOS8700) were used

We have too many parameters to try to encapsulate them in a complicated filename, so instead we can capture them in an XML file and bundle it with the raw sensor data files in a .zip. To automate this process, we developed our own PC application using Microsoft Visual Studio and .NET Framework. The .NET Framework’s support for XML serialization is particularly useful.

Now, when I want to log data, all I have to do is fill in the various fields and click “Record”. The user interface makes use of combo boxes to prevent invalid values from polluting the metadata. In the above figure, I’ve decided to monitor a stock pot filled with boiling water on the front right burner, which I’ve named Burner A. I want to see the difference between mounting a sensor on the pot handle vs. mounting a sensor on the stove top, therefore I’ve configured two sensor boards to record simultaneously.

The result is a .zip file containing the raw sensor data in CSV format and the metadata in XML format.

And finally, here’s an excerpt from the XML file:

  <?xml version="1.0" encoding="utf-8"?>
  
  <StoveTop 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  
  <TechnicianName>Maureen Helm</TechnicianName>
  
  <Output_Folder>C:Usersmstengl1DocumentsCooking</Output_Folder>
  
  <Timestamp>20150408113135</Timestamp>
  
  <Requested_Recording_Length>5</Requested_Recording_Length>
  
  <Actual_Recording_Length>4.999</Actual_Recording_Length>
  
  <SensorBoards>
  
  <StoveTopBoard>
  
  <Port>COM5</Port>
  
  <BaudRate>230400</BaudRate>
  
  <IsMonitored>true</IsMonitored>
  
  <Status>Ok</Status>
  
  <Board>FRDM-K64F</Board>
  
  <Sensor>FXOS8700</Sensor>
  
  <SW_Version>1.2</SW_Version>
  
  <Location>Stove Top</Location>
  
  <StoveTopBurnerName>Burner A</StoveTopBurnerName>
  
  </StoveTopBoard>
  
  <StoveTopBoard>
  
  <Port>COM6</Port>
  
  <BaudRate>230400</BaudRate>
  
  <IsMonitored>true</IsMonitored>
  
  <Status>Ok</Status>
  
  <Board>FRDM-K64F</Board>
  
  <Sensor>FXOS8700</Sensor>
  
  <SW_Version>1.2</SW_Version>
  
  <Location>Pot Handle</Location>
  
  <StoveTopBurnerName>Burner A</StoveTopBurnerName>
  
  </StoveTopBoard>
  
  </SensorBoards>
  
  <Stove_Type>Gas</Stove_Type>
  
  <Length_Units>in</Length_Units>
  
  <Front_to_Back>0</Front_to_Back>
  
  <Back_Left_to_Right>0</Back_Left_to_Right>
  
  <Front_Left_to_Right>0</Front_Left_to_Right>
  
  <StoveTopBurners>
  
  <StoveTopBurner>
  
  <Name>Burner A</Name>
  
  <Pot_Type>Stock Pot</Pot_Type>
  
  <Pot_Sides>Straight</Pot_Sides>
  
  <Pot_Material>Stainless Steel</Pot_Material>
  
  <Pot_Surface>None</Pot_Surface>
  
  <Pot_Diameter>10</Pot_Diameter>
  
  <Pot_Depth>6</Pot_Depth>
  
  <Liquid_Type>Water</Liquid_Type>
  
  <Liquid_Depth>3</Liquid_Depth>
  
  <State>Simmer/Gentle Boil</State>
  
  <Burner_Diameter>5</Burner_Diameter>
  
  </StoveTopBurner>
  
  </StoveTopBurners>
  
  </StoveTop>
  

An intelligent stove top is just one example of the many IoT applications that arise from mining sensor data. At the start of all of these applications is a need to capture raw sensor data and metadata in a variety of known conditions. With this in mind, we architected our PC application to contain reusable base classes that implement the data flow, and derived classes that implement stove-top-specific behavior. All that changes from one application to the next is the metadata; the methods to communicate with ISF, write raw sensor data to a CSV file, write metadata to an XML file, etc. are all the same. As a result, it is easy to re-spin our stove top logger into a human fall logger, a sump pump logger or ….

Stay tuned for the next entry in our Mining Sensor Data blog series, where Mike Stanley will import our raw sensor data and metadata into Matlab, and use it to train machine learning algorithms.