17 How to Handle Repetitive Tasks
Recalculate control. This control allows you to tie the output of the operation to the source data such that any changes to the source data can trigger an update of the output. This allows you to analyze multiple datasets by, for instance, serial import of a new data file to replace existing data. This feature is also the basis for creating Analysis Templates. The Analysis Template concept is explained below.
All Analysis and most data processing dialog boxes in Origin include a
Three options are available for the Recalculate control:
- No lock is displayed in the output.
- Changes to the input data will not result in an update of the output.
- An auto green lock is displayed in the output columns and graphs that contain plots of the output data.
- The output will be automatically updated when input data is changed.
- You can also recall the dialog to make changes to the analysis settings.
- A manual green lock is dispalyed in output columns and graphs that contain plots of the output data.
- The lock will turn yellow in color when input data is changed. You can manually trigger an update for each operation, or for all such pending operations in the project.
- You can also recall the dialog to make changes to the analysis settings.
A left-click on the lock displays a menu that provides multiple options including changing analysis parameters, opening source data sheet, switching to result sheets, and controlling the status of the operation such as switching from manual update to auto update.
Settings in analysis dialogs and most other data processing dialogs can be saved as a Dialog Theme file. Once saved, these Theme files containing your custom settings can be recalled as needed. Multiple theme files can be saved from a dialog, allowing for easy repeat analysis of datasets that may each require different settings.
Last used dialog box settings are automatically saved as a <last used> Theme. Origin also allows you to save your custom settings as the <default> Theme. The default Theme, last used Theme, and any named Theme that you have saved, can all be accessed from the theme fly-out menu that opens from the dialog box. These same Themes are also available from the main menu item associated with the analysis dialog box.
Dialog Themes are managed with the Theme Organizer tool, available from the Tools menu.
Tutorial: Saving and Re-using a Dialog Theme
- Import the file Linear Fit.dat from the Samples\Curve Fitting\ subfolder.
- Highlight column B and select Analysis: Fitting: Linear Fit...
- In the Linear Fit dialog that opens, check the Fix Intercept checkbox (under Fit Options) and set the Fix Intercept at edit box to 0.
- Click the button next to the Dialog Theme control and select Save as .... In the Theme Name box, enter Zero Intercept and press OK. Press OK again to close the Linear Fit dialog box and perform the analysis. FitLinear1 and FitLinearCurve1 result sheets are added to the workbook.
- Return to the source data and highlight column C. Select Analysis: Fitting: Linear Fit from the menu. You will see a fly-out menu with multiple Theme options including the Zero Intercept Theme you saved in the previous step.
- Select your saved Theme. The analysis is automatically performed on Column C using the settings saved in the Theme. Note that the dialog box does not open.
- Hold the SHIFT key while clicking on your Theme in the main menu and the associated dialog box will open with settings from the selected Theme loaded into the dialog box.
- The default theme Origin shipped for an analysis is called System Default. click the fly-out menu in analysis dialog and choose System Default to load it.
- Click the fly-out menu in analysis dialog and choose Delete to delete themes you have created, including customized <default> theme.
- The customized <default> themes for all analysis dialogs are saved in Defaults.xml under User Files folder. Delete it will restore system default settings of all analysis dialogs.
Project and Workbook Templates
The Project as Template
The Origin project file can be used as a template for carrying out repetitive graphing and analysis tasks. Create the desired graphs and/or analysis results from data in your workbook(s) and save the project. You then simply replace the workbook data, to automatically update graphs and results. When saving the project, you have the following options:
- Use the File: Save Project without Data menu item. This will clear all data before saving the project.
- Use the File: Duplicate Project without Data menu item. This will create a copy of the existing project where all the data will be cleared before saving.
You can, of course, save the current project with data, but the above two options may prove useful when working with very large datasets as they will reduce project file size.
The Workbook as Template
You can save a workbook as a template for repetitive graphing and/or analysis tasks. The workbook can contain worksheets with data, floating or embedded graphs and notes windows.
If the workbook contains analysis operations, the File: Save Workbook as Analysis Template... menu command can be used to save the workbook. All data columns that are used in the analysis operations in the workbook will be cleared before saving. Data that are not associated with analysis operations will not be cleared. Workbooks saved in this manner can be accessed quickly from File: Recent Books on the main menu.
Tutorial: Creating an Analysis Template
- Start with a new workbook and import the file Samples\Curve Fitting\Sensor01.dat.
- Select column B and use the Analysis: Fitting: Linear Fit and open the Linear Fit dialog.
- Change the Recalculate drop-down to Auto.
- Check the Fix Intercept check box and enter 0 in the Fix Intercept at edit box.
- Click OK to close the dialog and perform the linear regression.
- Switch to the FitLinear1 report sheet to view results including plots of the best-fit line and residuals.
- Now switch back to the original data sheet and import the file Samples\Curve Fitting\Sensor02.dat. The analysis results will be automatically updated with this new data. You can continue to use the workbook for other data. Or right click the workbook title and choose Duplicate without Data to create a new workbook with linear fitting operation saved in it. Then import new data in new workbook and thus save a project with multiple such workbooks if desired.
- With the workbook active, select the menu File: Save Workbook as Analysis Template..., and in the dialog that opens, give a name such as Linear Fit of Sensor Data and click Save.
- Choose File: Recent Books menu item and select the template that was saved in the previous step. The workbook will open and the data sheet will be empty.
- Import the file Samples\Curve Fitting\Sensor3.dat into the empty data sheet (1st sheet). The analysis results will update upon import of the new data.
The workbook template can even contain custom report sheets and a summary sheet. Thus, you can create an individual report for each file. The advantage of the workbook-as-template method is that the Batch Processing tool can be used to process multiple files or datasets and create a final summary report for all files that are processed as part of the batch analysis. View the Batch Plotting and Batch Analysis sections of this chapter for examples of using the workbook as a template when handling repetitive tasks.
Origin provides several methods for batch plotting of graphs from multiple datasets or files. The following two sections outline how to create multiple graphs from (1) data that is already in worksheets, and (2) multiple data files. In addition to these two procedures, batch plotting can also be performed programmatically using LabTalk script or Origin C.
Duplicating Graphs with Data from Other Books/Sheets/Columns
If you have several workbooks, worksheets or columns with similar data structure as you the data used to plot the graph, you can have Origin clone that graph via Window: Duplicate (Batch Plotting) menu with new data. There are two cases:
- If you have plotted a graph with a single data and customized it, and want to clone the graph with other data in the same worksheet: Choose Window: Duplicate (Batch Plotting): Duplicate with New Columns. Pick other data (columns) that you want to plot. Each column will be plotted as a new graph.
- If you have plotted a graph with data in one worksheet or workbook and customized the graph, and you want to clone the graph with other worksheet or workbook with a similar data structure: Choose Window: Duplicate (Batch Plotting): Duplicate with New Sheets/Books. Origin will list all worksheets or workbooks with a similar data structure. Pick the worksheet or workbook you want to plot from. Each worksheet or workbook will be plotted as a new graph.
The Workbook as a Template for Processing Multiple Files
If you want to plot graphs from many data files but don't want to import all files to workbooks before plotting, you can import one file, create the desired graph(s) based on that data, then add the graph(s) to your workbook and save the workbook as a template. Using this template you can process multiple files, creating a workbook for each file and its corresponding graph.
Tutorial: Creating graphs from multiple data files
- With a new workbook active, choose File: Import Single ASCII and import the file Sensor01.dat from the Samples\Curve Fitting subfolder of the Origin installation folder.
- Highlight column B and create a line+symbol graph of the data.
- Double click on the X axis to open the Axis dialog. Make sure Scale tab is active. Select both Horizontal and Vertical on the left panel and set Rescale to be Auto. This is to make sure graph scale will update automatically based on data.
- In the workbook, right-click on the worksheet tab and select Add Graph as Sheet, then select the graph created above. This will add a new workbook sheet containing an embedded graph.
- Switch to the data sheet, double-click on the tab rename the sheet as Data.
- Select the Worksheet: Clear Worksheet menu item to clear the data in this sheet. Note that this step is optional. Clearing the data will reduce the size of the template saved in the next step. Note that you should clear only the data sheet, and not the other sheets such as those with embedded graphs.
- Select the File: Save Window As... menu item, assign a name such as Sensor Data and Graph and press Save.
- Now we can use this template to process multiple files. Select the File: Batch Processing... menu item.
- In the dialog box that opens set the Batch Processing Mode to Load Analysis Template, then set the Analysis Template control to point to your saved template.
- Set Data Source to Import from Files and select the three files Sensor01.dat, Sensor02.dat, and Sensor03.dat from the Samples\Curve Fitting subfolder.
- Set the Data Sheet to Data and set Result Sheet to <none>.
- Press OK to close the dialog box. You should get three workbooks with the data imported into the first sheet and the graphs updated in the 2nd sheet. To further edit any of the graphs, double-click on the graph to pop up an editable page.
If processing of your data requires some custom import settings, you can save those settings to the data sheet from the Import and Export dialog box (Save to <Sheet>, available from the Dialog Theme menu). Settings thus saved to the sheet will be used for import when batch processing of multiple files using the workbook as a template.
Origin provides several ways to perform batch analysis of multiple files, data columns, or data plots.
Analyzing Multiple Datasets in Dialogs
Some analysis dialog boxes, for instance Linear Fit and Nonlinear Fit, support analysis of multiple datasets. Report sheets created by these dialog boxes include a summary table listing the parameter values for each dataset and other pertinent results such as goodness-of-fit indicators. The summary table can be copied to an external sheet for further processing.
Tutorial: Fitting Multiple Datasets
- Open a new workbook and import the file Samples\Curve Fitting\Multiple Gaussians.dat from the Origin installation folder.
- Set the Long Names of the four Y columns as Peak 1, Peak 2, Peak 3 and Peak 4.
- Select all four Y columns, and use the Analysis: Fitting: Nonlinear Curve Fit... menu item to open the NLFit dialog box.
- Select Gauss from the Function drop-down list, then press the Fit button to perform fitting and close the dialog box.
- Switch to the FitNL1 report sheet. You will see a table named Summary which lists the fit parameters and fit statistics for each dataset.
In the NLFit report sheet, click on the downward-pointing arrow button next to the table named Summary and select Create Copy as New Sheet. This will create a copy of the table in which all cells are linked to the report. Any updates/changes to the fit will automatically update the values in this copied sheet. This sheet can then be used to plot or to perform secondary analysis on the fit parameters.
Using Gadgets for Analyzing Multiple Curves
Origin includes several gadgets for performing interactive analysis on plotted data. Gadgets perform a range of functions such as selecting a data range of interest, switching from one dataset to another, and setting various preferences specific to the analysis being carried out.
Most gadgets offer an option to perform the analysis on all data plots in the current layer, or all data plots in the graph page. This allows for performing repetitive analysis on multiple datasets using the same settings, and generating a table of results across all datasets.
Batch Analysis Using an Analysis Template
Batch Processing tool allows you to process multiple files or datasets using an Analysis Template. Simply perform the analysis on one of the files, include all desired results and report sheets in one workbook, and save that workbook as an Analysis Template. The tool then uses the Template to process multiple files/datasets. You have the option to retain one workbook for each file/dataset, and additionally, to create a summary table with select analysis parameters and other metadata that you have pre-configured in your analysis template.
Tutorial: Batch Analysis of Multiple Files using an Analysis Template
- From the main menu, choose File: Batch Processing.... This opens the Batch Processing dialog box.
- Set Batch Processing Mode to Load Analysis Template.
- Press the browse button to the right of the Analysis Template box and select the file Sensor Analysis.OGW located in the Samples\Batch Processing subfolder under your Origin installation folder. This Analysis Template contains multiple sheets set up for linear regression analysis, reporting, and summary tables.
- Set Data Source to Import From Files, then press the browse button to the right of the File List and select the files Sensor01.dat, Sensor02.dat and Sensor03.dat from the Samples\Curve Fitting subfolder under your Origin installation folder.
- Set Dataset Identifier to File Name, Data Sheet to Data, and Result Sheet to Result, thus selecting the appropriate sheets in the Template.
- Uncheck Delete Intermediate Workbook.
- Click OK to close the dialog box. The three files are processed. You will find three books with the data, analysis results, and report for each file, and a fourth book containing the summary table of analysis results from all files.
Saving the initial workbook as an Analysis Template is optional. You can simply save the Origin project (.opj) and next time replace the data in your workbook to update all results and graphs. The Batch Processing tool also has an option to repeatedly import files into the active window, allowing you to simply re-use an existing book within a project (which contains all desired analysis and graphs) as an on-the-fly template for the batch analysis.
You can batch generate analysis reports using a custom MS Word template, with the option of outputting a PDF file for each report. For more information, see this tutorial.
Repeating Analysis on Other Datasets or Data Plots
For some analysis operations, you can perform the analysis on one dataset or data plot and then repeat the analysis for all other data. This feature is available via a special shortcut menu entry, when you click on the lock associated with the operation. In worksheet columns or reports, clicking the lock will show the menu command Repeat this for All Y columns. Selecting this will repeat the analysis on all other Y columns in the source data sheet. In a graph, clicking the lock will show the menu command Repeat this for All Plots. Selecting this will repeat the analysis for all other data plots in the graph page, even if the plots are in different layers.
This is particularly useful for such analysis dialog boxes as smoothing, or interpolation, etc. that support input of only one dataset. As long as the data are contained in one worksheet or plotted in one graph, the analysis can be repeated on all other datasets.
Tutorial: Normalizing Multiple Columns in a Worksheet
- Import the file Samples\Curve Fitting\Multiple Gaussians.dat into an empty workbook.
- Select column B, then click the menu Analysis: Signal Processing: smooth and open the dialog box.
- Accept the defaults and press OK to perform smoothing. A new column will be added with the smoothed data.
- Click on the lock in the output column and select Repeat this for All Y columns. Three more columns of smoothed data with same settings will be generated from the data in columns C thru E.
Repeating Analysis Using Data Filters
Data Filter feature can also be used in conjunction with the colcopy (column copy) X-Function to produce multiple graphs from the same source data using different filtering conditions. The filtered data can also be analyzed, allowing you to compare graphs and analysis results across multiple filter conditions.
Large multi-column datasets can be quickly reduced by applying filter conditions to one or more columns. This
Selected columns from the source data sheet can be copied to create child sheets where the filter condition stays synchronized with the parent sheet, or is locked to the child sheet. When the source data sheet is updated, all child sheets, associated graphs and analysis results will automatically update using their respective filter conditions. Additionally, the filter condition of a particular child sheet can be pushed back to the parent sheet at any time.
The image below displays the source workbook with data for several makes of automobiles. Two books were created with a subset of columns and a data filter to restrict the data to a particular make of automobile. Linear regression analysis of the filtered data was performed, allowing comparison of the results across the two filters.
Tutorial: Locking a Filter Condition on Copied Columns
- Import the file Samples\Statistics\Automobile.dat
- Click on the Make column, then right-click and select Filter: Add or Remove Filter context menu.
- Click on the filter icon that was added to the column, and uncheck all makes other than Honda (Choose Select All to clear all and then select Honda). Click OK.
- Hold down the CTRL key and click and select the Make, Power and Engine Displacement columns. Next, right-click and select the Copy Columns to... context menu.
- Accept defaults in the dialog that opens, and click Ok to copy the columns. A new worksheet will be added to the workbook.
- Click and hold the tab of the new worksheet and drag it to an empty spot in the Origin workspace to create a separate workbook.
- Click on any of the locks in the columns of this copied sheet, and select Worksheet Filters: Lock. The filter conditions will be locked to this sheet. If you change the filter condition in the original data sheet, this copied sheet will not be affected.
- You can modify the filter condition to Toyota and use Copy Columns to to create another worksheet. Plot Power vs Engine Replacement, and do linear fit from such new worksheet to compare between different automobile makes.
Automating Tasks Using Programming
In addition to the above mentioned methods for automating tasks Automating Origin using the interface, graphing and analysis features can also be accessed programmatically from either the LabTalk scripting language or from Origin C. Access to Graph Themes and templates, and Analysis Templates can be programmed. You can set up some of the procedure manually by first creating the templates using the graphical user interface, and then write your code to call the templates as needed.
You can get a broad look at what programming options are available in Origin by browsing the Programming Chapter of this User Guide. More in-depth programming-related information is linked to from that chapter.
Topics for Further Reading