Author: SRICOBSAS
Subject: 24x7 online logs using DSN1LOGP (similar to CA Log Analyzer)
Posted: Thu Aug 18, 2016 11:52 am (GMT 5.5)
Hi DB2 Experts,
I am in a kind of difficult situation in my project. I would be really grateful if experts could help me out.
On a daily basis I need to extract all log messages from the DB2 archive logs and all daily trace data from the active SMF datasets, paste these output in a word doc, upload the word doc onto a sharepoint folder
The catch is I need to do be able to query the logs while DB2 is active (.i.e the bootstrap dataset is active and hence difficult to touch it using DSN1LOGP)
There are 3 bad news in this regard
1) My client has discontinued the use of CA Log Analyzer which they were evaluating for the past 3 months. Hence CA support is not available.
2) Our DB2 DBA has gone on a long medical emergency (probably after hearing this CA news he is planning to quit)
3) I have been pulled in for this daily log activity. I come from a Business Intelligence background and did some COBOL programming.
Since nobody was around whom I could ask I began by looking what are the available tools for this log extraction / trace extraction activity. I found:
Informatica PowerExchange
IBM Data Studio
BMC MainView
Compuware DBA-Xpert
With Informatica PowerExchange there is a way to trap the logs (from the active log dataset). But the problem is the tables need to have "DATA CAPTURE CHANGES" on. The client is not in favor of that.
I couldnt find any log display capabilities in Data Studio
BMC Mainview can show the active traces (even if tracing is not turned on using START TRACE) but the problem is it will capture only those traces / those commands which were issued by a z/OS batch job. The requirement is to extract ALL log messages (for all DB2 activity originating within z/OS or outside it. Various external applications connect to the mainframe DB2 via ODBC / JDBC and perform routine selects / inserts / updates. Hence ALL DB2 activities need to be trapped and stored)
Couldnt find any logging capability using Compuware DBA-Xpert
So I am back to DSN1LOGP and DSN1SMFP
Has anyone tried using DSN1LOGP on archive log datasets while DB2 is still active? I can get the RBAStart / RBAEnd / DBID / OBID info from Compuware's DBA-Xpert (or even from the sysibm catalog tables- SYSTABLES and SYSTABLESPACESTATS). I can also map these RBAs to the correct archive log (for e.g if 3 selects, 2 inserts and 5 row deletes happened yesterday on a table XYZ, I can get to know that today the archive log XYZ_ARCHLOG_D001 will contain all these info). The challenge is to extract the SQL DDL / DML activity from these archive logs while DB2 is up and running.
As for DSN1SMFP I googled around and thought of going for the below steps:
1) Insert AUDIT policies on tables which need to be audited in SYSIBM.SYSAUDITPOLICIES
2) Turn on tracing using START TRACE dest(SMF) at the start of the day
3) At the end of the day do a STOP TRACE
4) Use DSN1SMFP to read the relevant IFCID records and publish the SMF traces on sharepoint
I heard there is some overhead for turning on tracing but since this is the first time I am doing this activity I can convince the client for the business need in this regard
In short my client wants me to use the native IBM log / smf utilities DSN1LOGP and DSN1SMFP (or the available tools- Informatica / IBM Data Studio / Mainview / DBA-Xpert) and develop a 24x7 CA-LOg Analyzer kind online log extraction utility (I believe CA Log Analyzer can extract logs even while DB2 is online)
Apologies for the lengthy post, I also wanted to post the tool research I have done so far.
Would be grateful for any pointers / directions in this regard.
Thanks & Regards
visionplus.trams@outlook.com
infra.app.supp.se@gmail.com
_________________
SriCobSAS
Subject: 24x7 online logs using DSN1LOGP (similar to CA Log Analyzer)
Posted: Thu Aug 18, 2016 11:52 am (GMT 5.5)
Hi DB2 Experts,
I am in a kind of difficult situation in my project. I would be really grateful if experts could help me out.
On a daily basis I need to extract all log messages from the DB2 archive logs and all daily trace data from the active SMF datasets, paste these output in a word doc, upload the word doc onto a sharepoint folder
The catch is I need to do be able to query the logs while DB2 is active (.i.e the bootstrap dataset is active and hence difficult to touch it using DSN1LOGP)
There are 3 bad news in this regard
1) My client has discontinued the use of CA Log Analyzer which they were evaluating for the past 3 months. Hence CA support is not available.
2) Our DB2 DBA has gone on a long medical emergency (probably after hearing this CA news he is planning to quit)
3) I have been pulled in for this daily log activity. I come from a Business Intelligence background and did some COBOL programming.
Since nobody was around whom I could ask I began by looking what are the available tools for this log extraction / trace extraction activity. I found:
Informatica PowerExchange
IBM Data Studio
BMC MainView
Compuware DBA-Xpert
With Informatica PowerExchange there is a way to trap the logs (from the active log dataset). But the problem is the tables need to have "DATA CAPTURE CHANGES" on. The client is not in favor of that.
I couldnt find any log display capabilities in Data Studio
BMC Mainview can show the active traces (even if tracing is not turned on using START TRACE) but the problem is it will capture only those traces / those commands which were issued by a z/OS batch job. The requirement is to extract ALL log messages (for all DB2 activity originating within z/OS or outside it. Various external applications connect to the mainframe DB2 via ODBC / JDBC and perform routine selects / inserts / updates. Hence ALL DB2 activities need to be trapped and stored)
Couldnt find any logging capability using Compuware DBA-Xpert
So I am back to DSN1LOGP and DSN1SMFP
Has anyone tried using DSN1LOGP on archive log datasets while DB2 is still active? I can get the RBAStart / RBAEnd / DBID / OBID info from Compuware's DBA-Xpert (or even from the sysibm catalog tables- SYSTABLES and SYSTABLESPACESTATS). I can also map these RBAs to the correct archive log (for e.g if 3 selects, 2 inserts and 5 row deletes happened yesterday on a table XYZ, I can get to know that today the archive log XYZ_ARCHLOG_D001 will contain all these info). The challenge is to extract the SQL DDL / DML activity from these archive logs while DB2 is up and running.
As for DSN1SMFP I googled around and thought of going for the below steps:
1) Insert AUDIT policies on tables which need to be audited in SYSIBM.SYSAUDITPOLICIES
2) Turn on tracing using START TRACE dest(SMF) at the start of the day
3) At the end of the day do a STOP TRACE
4) Use DSN1SMFP to read the relevant IFCID records and publish the SMF traces on sharepoint
I heard there is some overhead for turning on tracing but since this is the first time I am doing this activity I can convince the client for the business need in this regard
In short my client wants me to use the native IBM log / smf utilities DSN1LOGP and DSN1SMFP (or the available tools- Informatica / IBM Data Studio / Mainview / DBA-Xpert) and develop a 24x7 CA-LOg Analyzer kind online log extraction utility (I believe CA Log Analyzer can extract logs even while DB2 is online)
Apologies for the lengthy post, I also wanted to post the tool research I have done so far.
Would be grateful for any pointers / directions in this regard.
Thanks & Regards
visionplus.trams@outlook.com
infra.app.supp.se@gmail.com
_________________
SriCobSAS