Have you ever read a text message or email, even from someone you know very well, however you are still unsure whether the content was meant to be interpreted factually, or was sarcastic, or intentionally ironic? Without the extra ‘context’ of hearing the message sender’s voice and/or seeing their facial expressions, you may not be sure of the true meaning of the message.
Sensor Fusion refers to the practice of combining raw and conditioned input data from two or more sensors to produce information you can be more confident is correct, as well as inferring new information not obtainable by a single sensor.
Types of Sensors applicable for Industry 4.0
The list of available, low power sensors for the physical environment is long and constantly growing.
Environmental Sensors give information about the characteristics of the immediate area your asset is located, and include ambient light level sensors, color sensors, barometric pressure sensors , air and object temperature sensors, gas detector sensors (such as CO2, CO, and so on), radiation sensors, magnetometer, Hall Effect sensors for detecting objects that affect Earth’s natural magnetic field such as cars, moisture sensors, biological sensors, and the list goes on.
Position and Movement Sensors give information about where your asset is on Earth, and how (or if) it is moving. Accelerometers, Gyroscopic orientation sensors, GPS , GLONASS and BeiDou satellite-based location sensors, even Wi-Fi and Cellular radio devices can provide location information.
Measurement Sensors are specialized for getting levels of specific assets, such as a float sensor telling you how much gasoline is left in our automobile fuel tank, voltage and current sensors measuring electrical energy flow and charge remaining in batteries, wheel speed sensors in wheeled vehicles, and wind speed and direction sensors. Timers are a type of sensor that can invoke an action once a specified period of time has passed.
Radio Frequency Sensors are specialized radio receivers employed in technologies such as RFID and NFC, which can be used to identify a nearby object or person.
Sensor Fusion Techniques
There are three generally accepted sensor fusion techniques: Competitive Fusion, Complementary Fusion, and Cooperative Fusion
Competitive Fusion refers to gathering inputs of many sensors of the same type in the same environment. Each sensor produces its own measurement of the single readout, and results are combined using “voting” (majority wins) or averaging calculations. For example, say you had a room of temperature sensors, and one of the temperature sensors is placed in the direct path of intense sunlight, say from a skylight, while another 5 are not. The sensor under the skylight will naturally read a much higher temperature than the others, and a voting algorithm throws out this outlying reading. If the sensor under the skylight were the only sensor, the room’s temperature would have been reported as much higher than it actually was.
Complementary Fusion refers to the combining of readings from two or more different types of sensors to provide a more complete understanding of an environment. A great example of this is Microsoft Kinect and similar sensors which use imaging sensors in concert with in infrared depth sensors to create a 3-D ‘point cloud’ model of a real environment.
Cooperative Fusion refers to using multiple instances of the same type of sensor, each with a slightly different position or perspective on the target environment, and employing mathematical and physical rules to generate higher-order information. A great example of this is using two image sensors slightly apart to generate 3-D information about an environment, whereas using only a single image sensor could not.
Applications of Sensor Fusion for Industry 4.0
Securent’s Cyber-Physical Trackers (CPT’s) have onboard location, velocity, and movement sensors. A simple but valuable application of sensor fusion is combining the current velocity reading with an accelerometer-based ‘bump” sensor. If an asset is currently moving at 55 MPH and a short bump is detected, it is most likely a truck has hit a pothole in the road, and the CPT can report this bump event including GPS location to cloud data services and log the likelihood of a pothole at this location, which can be used for other vehicles to avoid this pothole and possible damage to the vehicle and cargo.
Another example of multi-sensor fusion capability in Securent CPT’s is employing additional sensor methods to determine location in the event GPS data is not accessible, such as in an urban environment or indoors. CPT’s have on-board Wi-Fi and Bluetooth radios in addition to cellular radios, all of which can provide somewhat accurate location information individually, but much more ‘confident’ and accurate information when used together.
For example, the Wi-Fi radio can be turned on momentarily to scan for “Station ID’s” (SSID’s) and Radio Signal Strength Indicator readings (RSSI) of nearby Wi-Fi Access Points. This “Wi-Fi Survey” data can be uploaded to the Securent cloud for analysis, where databases mapping Wi-Fi Access Point SSID’s to physical addresses is consulted, and techniques such as triangulation on multiple Wi-Fi Access Points along with their relative signal strengths can provide an even more accurate probable location of the CPT.
The CPT’s Bluetooth Low Energy radio may be momentarily turned on to ‘listen’ for iBeacon radio beacon transmissions nearby such as found in retail stores. Each iBeacon transmission received contains a unique identifier which can be sent to the Securent Cloud to resolve against an Internet database of iBeacon locations to deduce the probable location of the CPT.
Another common component in Sensor Fusion technology application is keeping a memory of recent activity to refer to when attempting to make a determination. For example, the CPT uses GPS when available to determine not only precise location, but velocity as well, from which distance and direction of travel may be deduced. So when the CPT loses its ability to receive GPS signals, such as driving through a long tunnel, it can still refer to its history of location and velocity data to deduce its likely current location – digital “bread crumbs” if you will.
Combining or “Fusing” GPS location and movement history data, along with other data sources such as Wi-Fi and iBeacon surveys, allow Securent CPT’s to continue operating in environments where GPS is temporarily not available where other devices may not.
Another function Securent CPT’s are capable of performing is entering into a “Environment Data Collection Mode”, where fused results of multiple sensors are logged to a ‘timeline’ and uploaded to the Securent Cloud for processing and analysis. For example, a Securent CPT can be programmed to periodically, say every 2 minutes, take a GPS reading and correlate the timestamp and location along with velocity and Wi-Fi Access Points in range at the time of the reading. Securent Data Analysis Services can then use this data to produce ever more accurate correlations of Wi-Fi Access Points to GPS locations.
Securent is actively prototyping sensor fusion approaches and algorithms using the aforementioned sensors and more, including strain gauges, microphones, LIDAR, and Air Particulate Matter detectors for environmental studies. Sensor data streams are being recorded and ‘fed’ into deep neural network models using AI techniques to experiment with forming predictive models and knowledge bases for future expert systems, all to help usher in the fourth industrial revolution.
Contact Securent today to learn more about sensor fusion and AI techniques, and partner with Securent to develop and use these exciting technological advancements for your business.