Prologue

Prologue

The goal of analytical data system software in the industry is always challenging.  Analytical instrument that require new data system software.  Any new model of analytical instrument involved advanced technology and idea, but also often we don't know detail mechanism or performance based on theory so that is difficult to determine validation method.  Unless we have solid validation method described as written material, it is impossible to design software.  Software need to support new instruments that derived from previous model with upgrade/modification.  Software also need to be supported all existing data in the archive and make sure nothing break for existing data review or often re-process.

When I started my carrier at analytical instrument industry, UV detector for LC was about 1mAU that is equivalent to 1% p-p noise on strip chart recorder at 0.08AUFS so that was enough accurate to acquire data by 12bit ADC with automatic gain which directory hooked up to 8bit micro processor in conjunction with timer interrupt.  Almost 30 years from that,  detectors are now micro-AU level (1000 times improvement).  It automatically change the design of data acquisition system in order to acquire accurate data with respect to aperture error.  Processor interrupt latency is unacceptable for 14bit or higher bit resolution even for slow (1Hz) signal.

From software point of view, 30 years old 12bit auto gained data are exist in the database.  According to development cycle for drugs from discovery stage to production, 30 years are not short period of time so that has to be manipulated with new software.  On the other hand, modern 24bit data are acquiring 1440 samples a day an instrument which of course manipulated daily.

How to design software interface and data representation layer is extremely important.   In the industry, software is the key to make a progress of quick product delivery to market from science at the R&D.

Another challenging goal is the 24/7 operation, especially on the Windows platform.  Actually, I did not think this is difficult because I have been done this in the couple of previous project for HPLC instrumentation, however I recently realized it was wrong.  Most of previous projects were based on traditional C style programming rather than modern C++.  Using modern C++ programming using STL, BOOST and templates eliminate memory leaks and lines of code.  It is more than 100 times easy to improve final production quality with minimum number of human resources.  We are now using such technique for the project and carefully tested for memory leaks.  Since therefore I did not doubt 24/7 operation causing a problem.

However, when we tested on a project for sequence acquisition as a part of "continuous uptime test", we got a memory crash every 3.5days, which is corresponding to about 1600 samples under typical FAST LC condition.  I was very disappointed this result and spent couple of month to address this problem.  Finally, I have figured out that problem is not out of memory but address space fragmentation a.k.a. memory fragmentation.  In nature of C++ language, operator new and delete is hocked up to language it self and there is no practical way to control them from application code.  In old days, memory allocation is only available by implicit call of malloc, but modern C++ using hundred times more call of new and delete from STL and application defined contractor and destructor.  Especially on Windows XP, it is rapidly making memory fragment.  Technical details to resolving memory fragmentation to be discussed later.

Reporting engine is another challenge for data system software.  It is always a problem and time consuming part of overall project.  Why this is always a problem.  Well, I think there are two reasons.  Software project attempt to generate report engine as general as possible without spend enough time for users tasks and workflow at the requirement process.  We have to stop and think about what is 'general report engine'.  It is most often implement as "report template editor" and "report generator" and it works as a software feature, but often that doesn't resolve user's problem.  For example, HPLC software can generate report that contains instrument validation report, calibration report, quantitative result and summary report as a set of analysis report corresponding to sequence.  This is works fine with most of user, but GMP regulated QC lab want to add pass/fail result on the report according to own criteria described in SOP.  Then software project attempt to resolve this in the own software project.  However let us step back and see over all user workflow.  User's organization maybe report pass/fail result by single analysis report but mostly require several different analysis.  A simple example, 3 runs of quantitative analysis run and take an average, or combined LC/MS result with MS/MS spectra for selected ions which is very much depending on the samples.  Reporting features on the scientific analytical data system should be implemented separately from the core data system software using commercially available rendering mechanism.