Mitsubishi Electric Corporation(TOKYO: 6503) announced today it has developed a high-performance sensor database that performs fast storage, searchesand aggregation of massive data collected from sensorsthat are expected to be used widely in the IoT (Internet of Things)era.The database enablessensor data to be used quickly for purposes such as maintaining roads,railways and other infrastructure, monitoring factoriesand managingenergy use in buildings and homes.It has the capacity to process up to 100 trillion dataitems, such as three-dimensional measurementsby laser sensors on/around roadstotaling 200,000 kilometersin length, or sensor data accumulated in a factory over a three-yearperiod using 100,000 sensors that take measurements every 100 milliseconds.
Compared to existing databases, such as commonly used relational databases, Mitsubishi Electric has determined that itshigh-performance sensor database reduces storage space, load time,and search and aggregation time each to a rangeof just one-tenth to one-thousandthof current levels. This was achieved by optimizing:
– data compression,to reduce input/output data traffic by selectingcompression patternsfrom more than 700 combinations;
– data location,by arrangingcompressed data in storage blocks to reduce input/output time; and
– data processing units,by processing data in cache memorywhere possible to enhance parallel processing.
Massive data-processing performance can be improved through hardware enhancement, such as parallel or distributed processing using many servers, in-memory processing using large amountsof memory,or fast storage devicesusing flash memory. All of these methods, however, require very expensive hardware. Remarkably, Mitsubishi Electric’s high-performance sensor databaseachieves fast processing using a single server with only one or two CPUs and 4GB of main memory.
The amount of sensor data in a database generally increaseswith long-term use. Some trial systems start with small amounts of data for verification prior to full-scaleoperation using larger data. Mitsubishi Electric’s high-performance sensor database allows incremental expansion of servers as the amount of data grows. Storage capacity and performance are enhanced in the absence of data migration or changes in application software.