Many companies are drowning in a sea of sensor data collected from research, design, validation, and manufacturing operations. The engineers and scientists often lack the tools they need to efficiently capture, organize, and extract information from these data for general accessibility and usefulness.
 |
Computer-based data acquisition, control, and automated test systems can generate thousands of parameters every second. They stream raw data out to engineering and operations personnel who must interpret the information to ensure their products meet design specifications, along with quality and regulatory requirements. (Ever try to drink water from a fire hose?) This is information overload, but too much is better than too little. Still, how are you supposed to sift through all these data and make sense of them?
The ability to convert raw data into usable and timely information enables engineers and managers to better troubleshoot design and production problems, improve processes, and enhance efficiency and productivity. Real-time access to usable information can answer a variety of questions:
- 1. What's the overall yield and first-pass yield of product A supplied by our contract manufacturer last month?
- 2. How will changing component A affect product specifications and reliability?
- 3. How do variations in reactor temperature affect batch cycle times and product quality?
- 4. What key performance indicators are critical to maximize lot-to-lot yields?
The answers are out there somewhere, and the key to finding them lies in establishing a process methodology and acquiring the tools that can capture, organize, and provide secure access to the source data for analysis, visualization, and reporting (see Figure 1). Let's take a closer look.
 Figure 1. Turning sensor data into usable information requires a process methodology and tools for capturing, organizing, and providing secure access to the data whenever and wherever it's needed by authorized personnel. |
Capturing Data
The first step toward collecting and storing raw data will be a powerful, scalable, flexible repository. Industry-standard databases such as Microsoft SQL Server or Oracle are proven and reliable repositories. Storage requirements are application dependent and are determined by the types of data, sampling frequency, and length of time the data must be stored. Data from sensors and other measurement systems can consist of scalar data from analog measurements such as temperature and discrete signals from ON/OFF switches, ASCII strings from RFID or analyzers, binary data from vision and inspection systems, and data arrays from event-sequence recorders. Some applications need only short-term access to the stored data, but regulated industries among others require archiving for many years.
Securely and reliably capturing large volumes of real-time data requires additional considerations. Does your application require fail-safe data transfers to keep the information safe should a network, server, or application crash? Do you need data buffering to ensure that the data stream does not outpace the receiver? Do you need to manage data compression so that you can reduce network traffic and storage space?
You can develop your own data capture setup or buy one off the shelf that satisfies your particular requirements. One class of repositories, data historians, is commonly used in process, batch, and discrete manufacturing. They can aggregate real-time data from a variety of sensors, industrial networks, distributed control systems, programmable logic controllers, and supervisory control and data acquisition (SCADA) systems using industry standards such as OPC. Data historians are commonly based on proprietary databases optimized for storing and retrieving time-series data. They vary in size from small SCADA systems monitoring 100 points/s (e.g., temperature, flow, pressure, level), to large plantwide versions that are part of distributed control systems monitoring 100,000+ data points in oil refineries and chemical plants.
Test data management (TDM) systems are another type of off-the-shelf solution for safely and securely capturing lab, research, design, validation, and manufacturing test data. TDM systems capture real-time data from sensors and instruments connected to automated test systems through test software such as LabVIEW and store the data in industry-standard databases. TDM systems vary in size from a single test station in a small lab to large-enterprise systems with hundreds of test stations dispersed around the globe.
Organizing Data
Now that you've captured the data, you need to organize and manage them. This means that you will have to define the way the data are stored and organized in the database so you can search, find, combine, correlate, and filter the information. If you use a relational database, take care to create a schema structured and normalized so as to ensure the data are stored efficiently and optimized for highest performance.
Accessing Data
Information is useless if you can't access it whenever and wherever you need it. How are you going to get your data out of the repository? How many people need concurrent access and how much data will they be accessing? Researchers, engineers, and managers typically want real-time access to lab, design, and manufacturing operations and test data from their desks using a Web browser via the company's Intranet. In many cases, other departments such as QC and customer support need concurrent real-time access so they can see key performance indicators about production throughput, yield, and quality.
Web browsers make it easy to access data, but what about security? Does everyone need access to all stored data, or should some personnel be restricted to information related to their departments or projects? Do your engineers need to access data from home or when they are traveling? Do external suppliers and customers need access? Securing data and information should be given a lot of thought up front during system design.
How will other applications access the captured sensor data? SQL, ADO/ODBC, export to CSV or XML files, or an application programming interface are some ways applications can access sensor data stored in a database.
Once engineers find and extract the sensor data they want, they can use a wide variety of interactive mathematical analysis, statistical process control, data mining, and reporting tools to turn raw data into usable and timely information (see Figure 2). Many engineers favor Microsoft Excel for data analysis and reporting. You can increase efficiency and productivity by using macros and scripting to fully automate analysis of raw sensor data and report generation, and then publish the results through a Web server for instant access to anyone who needs the information.
 Figure 2. Engineers and managers use a variety of interactive mathematical analysis, SPC, data mining, and reporting tools to turn sensor data into information they can use to improve their company's products and manufacturing methods. |
The Benefits
Capturing, organizing, and processing sensor data into usable and timely information can help your company analyze and track key performance metrics through your entire system, identify problems early on, take immediate corrective action—and save money. One communication products company reported that capturing and organizing their sensor data, then finding, sorting, filtering, and selecting the raw data for further analysis and generating reports, was a tedious process that took their engineers days to complete. An automated system using an off-the-shelf TDM product performs these tasks in seconds and provides real-time Web access to usable information across the company.
Instant access to company-wide research, design, and manufacturing information gives engineering, operations, and management a competitive edge on improving their products and increasing productivity.
Don Holley, B.S. Engineering Control Systems, can be reached at VI Technology, Inc., Austin, TX; 512-327-4401, x-266, [email protected], www.vi-tech.com.