Quantcast
Channel: IBM Mainframe Computers Forums
Viewing all articles
Browse latest Browse all 9405

All Other Mainframe Topics :: RE: Storing huge volume of data, compare and process

$
0
0
Author: Robert Sample
Posted: Mon Jan 16, 2017 6:20 pm (GMT 5.5)

Quote:
* If I use tape files, I feel the efficiency will be low compared to VSAM.
This makes ABSOLUTELY no sense. To create a VSAM data set from your tape data, you will have to read all 100 tape files, sort to remove duplicates, and then define a VSAM data set and load it from the remaining data. Simply reading the 100 tape files and doing your comparisons means you are NOT performing the latter steps of this process, which -- by definition -- means you are increasing efficiency.

Write a program in the language of your choice to read the smaller data set into memory (a COBOL array, for example), and use that to drive your processing. You can load the array in key sequence. This allows you to use binary SEARCH if the tape files are not sorted by key sequence, or merely make one pass through the array for each tape if they are sorted by key sequence. Either way, even adding the time to create the program, you'll use much less time each month than you would by creating a VSAM data set.
_________________
TANSTAAFL

The first rule of code reuse is that the code needs to be worth re-using.

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." -- Donald Knuth


Viewing all articles
Browse latest Browse all 9405

Trending Articles