Site Network: Home | About

Showing posts with label powercenter. Show all posts
Showing posts with label powercenter. Show all posts

It is same as on Vista! I did this yesterday on my friend's machine shipped with Windows 7.

http://karteekblog.blogspot.com/2009/12/powercenter-860-on-vista.html

Karteek

I was recently going through my.informatica.com site to find some interesting tool set called “Debugging Tools” from Informatica. There is a tool called “MetaQuery 2.0”, a java window application, that connects to PowerCenter repository database and fetches some key properties of various repository related objects. As my-informatica is for customers & partners, I can not give the site contents directly or MetaQuery tool for downloading here (it may be classified).  However, I am just pasting the description provided there for this tool below.
Metaquery 2.0
“Metaquery is an Informatica tool to query against PowerCenter repository for some of the key properties in session, workflow, and mapping objects. This tool is helpful to gather information regarding properties set in any of these objects (like session or workflow) and get the output report exported into a readable excel sheet or text file. The latest version supports repositories in DB2 and Sybase in addition to Oracle repositories. More properties in session and workflow are queries upon as well.”
I think this certainly is a useful tool to have for repository analysis and debugging. This tool is not exposing the queries it is using to fetch the data (in any readable format), but you are allowed only to choose the options on GUI (for example, sessions with target table enabled option) and submit. I usually write some queries on repository with all sort of reverse engineering, but as the underlying architecture/database design is no where documented it is difficult at times to get the requirement done properly and accurately.
Some screen shots of tool:
image
image
image
When I saw this tool first time on my-informatica site, I felt happy that it would reveal some facts about how some repository tables are related and logic behind using them. But it was not the case as the tool revealed only usage options from GUI, and all the underlying queries are still hidden in java .class files.  I had to use an alternative approach for revealing the queries. I used JD Java Decompiler and decompiled the class file (.\XML\SQLQueriesOra.class) and that is it – all the queries that the tool used for 46 features are available. I have compiled all of them at once place in a spread sheet (link provided below). Same query details can also be retrieved from java command line console, where in java echos the query each time you submit the request from front end. Hope you enjoy it!
A sample query used for getting the “The name of the source table used in the mapping” is below:
SELECT A.SUBJ_NAME,C.MAPPING_NAME,D.INSTANCE_NAME SOURCE_NAME_IN_MAPPING FROM OPB_SUBJECT A,OPB_MAPPING C,OPB_WIDGET_INST D WHERE A.SUBJ_ID=C.SUBJECT_ID AND C.MAPPING_ID=D.MAPPING_ID AND D.WIDGET_TYPE=1 AND D.WIDGET_ID IN (SELECT SRC_ID FROM OPB_SRC )
Welcome you all to add some of your experiences on your repository usage.
Repository Queries for PowerCenter 8x (from MetaQuery).xls
Karteek

I still have to figure out the internals of Informatica BULK target load type. How exactly it works, how it is different from its Normal load, and more importantly how its implementation is different from database to database. Today I have encountered with some issue with bulk load type enabled and ROLLBACKing the transaction (in Informatica) using Transaction Control transformation.
My idea is to issue ROLLBACK after inserting few records into the target. So, I have picked SCOTT.EMP table (in Oracle) as source with ORDER BY EMPNO.
As I wanted to issue ROLLBACK in the half i.e. after loading about 7 records out of 14, I created a Transaction Control Transformation with “IIF(EMPNO = 7782, TC_ROLLBACK_BEFORE)”
This failed with below ORA- error.

Rollback statistics:
WRT_8162 ===================================================
WRT_8330 Rolled back [6] inserted, [0] deleted, [0] updated rows for the target [EMP]

CMN_1022 [
ORA-24795: Illegal ROLLBACK attempt made

Database driver error...
Function Name : Commit
Database driver error...
Function Name : Rollback]
However, this worked well with Normal target load type. If Informatica’s bulk load type is like Oracle’s INSERT with  APPEND hint, direct transaction in Oracle should also fail, but that would not be the case. So, this makes me feel that Informatica has some special handling of BULK load type without completely throwing it to database side.
karteek@orcl10gr2>insert into /*+ APPEND */ emp select * from scott.emp;
14 rows created.
karteek@orcl10gr2>rollback;
Rollback complete.
karteek@orcl10gr2>
  • APPEND hint makes the INSERT statement to write the data directly into the data blocks above the high water mark, and more over this supports only INSERT with SELECT clause, but not with VALUES clause where in we pass values for each record.
  • Chances are there that Informatica is using an approach similar to BULK COLLECT INTO in Oracle. But, this might not be exactly the same approach, because BULK COLLECT is possible only in PL/SQL.
I think there are few questions that are left unanswered yet.

Update (03/08/2010): (per Srinivas comment below)

I am trying to load the below comma seperated file into a table using direct path enabled in sqlldr.
C:\Users\Karteek\test>type num.dat

1,3
2,4
3,7

SQL>desc tnum;
Name Null? Type
----------------------------------------- -------- ----------------------------
A NOT NULL NUMBER
B NUMBER

SQL> select index_name,status from user_indexes where table_name = 'TNUM';
INDEX_NAME STATUS
------------------------------ --------
TNUM_PK VALID
SQL> select * from tnum;

no rows selected

SQL> $sqlldr karteek control=num.ctl direct=true
Password:
SQL*Loader: Release 10.2.0.3.0 - Production on Mon Mar 8 11:03:47 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.

Load completed - logical record count 3.

SQL> select * from tnum;

A B
---------- ----------
1 3
2 4
3 7

SQL> select index_name,status from user_indexes where table_name = 'TNUM';
INDEX_NAME STATUS
------------------------------ --------
TNUM_PK VALID

SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production
With the Partitioning, OLAP and Data Mining options

C:\Users\Karteek\test>

Happy Holidays!!
Karteek

I just now installed and configured it, and also did a simple "Hello World” kind of test run which went fine.
Quite a simple installtion it was (especially on XP).

There are 2 differences between installation on Windows XP (32bit) and Vista:

  1. appropriate .Net runtime Vista (it needs 1.1) and
  2. picking up the right setup files
Windows Vista comes with .Net comes .Net 2.0. Use the below link to install .Net 1.1 on Vista. This is required for PowerCenter client applications to properly work.

http://www.mydigitallife.info/2007/12/27/install-microsoft-net-framework-11-on-windows-vista-fix-regsvcsexe-fails-error/

On Windows XP run the batch file “Informatica 8.6\Server\install.bat” for server, and “Informatica 8.6\Client\install.bat” for client (This is a standard approach).

On Vista, this is slightly different. For some reason running the same batch files on Vista in XP compatibility mode is not working (failing while creating Informatica Services, with Error Code – S. Symptoms explained here) – perhaps batch script is not initiating the original installer/.exe files in compatibility mode, may be !
So, as a workaround, run the actual installation files in XP Compatibility Mode as Administrator.
Installation files are as below (in PowerCenter 8.6.0 setup CD):
  • Informatica 8.6\Server\Server\Windows\Disk1\InstData\VM\install.exe  - for installing server
  • Informatica 8.6\Client\Client\Disk1\InstData\VM\install.exe  - for installing client
Installation there after is quite easy – just follow the instructions. All this worked for me. If you find any difficulty, let me know.

Hope it helps!
Karteek

In the last post I described the approach for installing Informatica Powercenter 7.1 on WIndows Vista . There was something I did on configuring Windows services which I would like to mention here.

When Informtica is installed on Windows with repository on Oracle database, it will show the following services (as display names) in “Services” window.

  1. OracleServiceORCL10G (there are other Oracle services as well like, but I am not considering them for the current scenario, as this service alone is just enough to describe what I want to say)
  2. Informatica Repository Server (PmRepServer)
  3. Informatica (Powermart)

Previously I used to keep Oracle in “Automatic” startup option so that Oracle database is available as my machine boots up (means I do not have to manually bring it up). I had wished the same effect for both Informatica services as well, so that I did not have to start them every time I reboot my machine. Problem was that there is a dependency among these 3 services. Rep service can not start unless Oracle database is available (since Rep service requires rep database), and Informatica service can not start without Rep service in place running. Due to these dependencies among services and I did not set up any dependency rule between them, they all used to start their services at one time (after system reboots), but only Oracle used to succeed as Oracle does not have dependency on others (at least not on other 2 Informatica services).

I always overlooked the “Dependencies” tab that is there when you open properties of a service (Right click on service and choose Properties). But the last time, while I was installing Infa 7.1 on my Vista laptop I gave a close look at, and realized that that was something useful for me. You can very easily define the dependencies between services so that they start or stop in order.

The following steps need to be performed in order to create a dependency (Consider taking backup of current registry settings)

  1. Run 'regedit' to open your registry.
  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services and locate the service that you need to set a dependency for. (here you need to search for service name, not display name i.e. locate PmRepServer not “Informatica Repository Server”)
  3. Open the 'DependOnService' key on the right side. If the selected service does not have a 'DependOnService' key, then create one by right-clicking and selecting New > Multi-String Value.
  4. In the value field, enter the names of all services that the current service will depend on. Each service name must be entered properly (again, it should be service name, not disaplay name) and on a separate line.
  5. Click OK

Perform the step 2 and 3 for both “Informatica Repository Server” (PmRepServer) with “OracleServiceORCL10G” as DependOnService, and “Informatica” (Powermart) as DependOnService on “Informatica Repository Server”.

This setup makes sure that the service start happens only in “OracleServiceORCL10G > Informatica Repository Server > Informatica” order, and the service stop in reverse order.

There is a new Startup Type option introduced in Windows Vista called “Automatic (Delayed Start)”. I feel this is another good feature. This option delays the Automatic starting up of services for some time (may be few seconds) until the machine boot-up is completely done (you can avoid/reduce the painful slow/no responses on restarts). I enabled this option on “OracleServiceORCL10G” service. Anyway, other 2 services are now dependent on Oracle service, so enabling delayed startup type explicitly would have no additional effect, so they are just “Automatic” startup type.

Hope it helps!

Karteek

6 months ago I got my new Dell 1555 laptop with Windows Vista. I never wished I would use Vista after reading its reviews about bad resource consumption and performance complaints in many places. But my experience using Vista was not that bad, I enjoyed its look and feel, with some simple & cool features (like address bar, search just to name a few). And when it comes to performance it is not worried some on my machine with Intel Core Duo T6400 3 GHz and 4 G RAM.

However the major drawback I personally felt with Vista is its compatibility with many software especially in the midst of its reviews there is even no hope that software vendors put any efforts to release Vista compatible packages. Most of the software in BI space are not compatible on Vista. So, I dared to use virtual machine with XP on it so that I can do my works. But the performance of VM was really bad, and I could not do any work on it - it spoiled both host and guests. It has been more than 1 month that I did not run VM at least once :(.

Yesterday, I was browsing Oracle.com and noticed Oracle 10g for Vista is available for download - don't know since when it was there, but my bad I did not notice it until yesterday. I downloaded and installed it - cool, it's working. And then I started trying with Informatica 7.1. I know it is not compatible on Vista, but kept trying resolving entire issues one after other, and finally Informatica 7.1 on Vista - it worked!

I got some strange errors like a few below while I was trying installation. Most of the part is just same as it was on XP when I did last time. Entire credit should go to Vista! Yes, truly. Though Informatica 7.1 does not have any compatibility, Vista has opened its windows to host this software with its amazing "Compatibility" feature.

Here are the steps...
Create a database user (repo/repo@orcl) with necessary privileges like Create Session, Create table etc... – This user is required for holding the Informatica repository objects (will be used in the steps explained below).

Steps below are for configuring Informatica 7.1 on Windows Vista:

1. Run Launch.exe in compatibility mode (Windows XP)
2. After installation (Product License Key is required)
3. Setup Repository and Informatica servers (steps below)
         a. Informatica Repository Server Setup - with defaults unchanged, give password for Administrator
         b. Informatica Server Setup –
                  i. Give Informatica server name (INFA_SERVER) and hostname (localhost) in Server tab
                 ii. Give Repository name (REP_SERVER), repository user (Administrator - pwd for this user comes from step 3a), repository hostname (localhost) in Repository tab
                iii. Add licences in Licenses tab (not required to add Product License, but others like Connectiviy, Partitioning etc).
4. Go to Windows Services and start "Informatica Repository Server" service (PmRepServer) – this uses details entered in 3a.
5. Open “Repository Server Administrator Console”
        a. Right click on "Informatica Repository Servers" and click "New Server Registration"
        b. Enter the hostname given in step 3b in Repository tab (localhost).
        c. Above entered hostname will appear in the server list. Right click on the entered hostname and click "Connect"
        d. Enter the administrator password given in step 3a
        e. Right click on the folder called "Repositories" and click "New Repository"
                   i. Enter the repository server name that is entered in step 3b in General tab (not required to choose Global Repository check box)
                  ii. Enter database details (DBUser, DBPassword, ConnetString) in Database Connection tab.
                 iii. Add the licences, including product license in Licenses tab
                 iv. Click Ok - this will start creating the repository tables in the database location specified above.
6. Go to Windows Services and start “Informatica” service (Powermart) – this uses details entered in 3b.
7. Open "Repository Manager" and connect to repository (REP_SERVER) using Administrator and its password
8. To create additional user (recommended), go to "Security" menu and click "Manage Users and Privileges" and add a user.
9. Create a folder - go to "Folder" menu item and create. Change owner of folder to the user created in above step
10. Go to Workflow Manager (Run as Administrator) and login with Administrator and click "Server Configuration" under "Server" menu
         a. Add New server - give server name and hostname as given in step 3b (INFA_SERVER, hostname), and set "$PMRootDir" in server variables list (C:\Program Files\Informatica PowerCenter 7.1.1\Server). This server will be assigned to a workflow or session when before it has to run.

As I said, very few steps are different in installation on Vista compared to XP.
        • For Vista, it's required to install Informatica (Launch.exe) in XP compatibility mode (as Informatica and Vista are not compatible) and
        • Run the client applications (Designer, Workflow Manager, probably Monitor as well) as "Run as Administrator".

Below picture lists the files that will be created under "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Informatica PowerCenter 7.1.1" post installation of Informatica 7.1 (after Step 2)


Hope it helps!

I have been working on 8.6 for quite a long time, but never tried setting it up. I am planning that soon, if possible in the next week, with a post collecting my experiences.

Karteek

I was just going through Powercenter Monitor's help guide and noticed a page describing about the different possible statuses that a workflow or a task can be in at a given point in time. I found a status  called "Unknown Status" as one possbile status, and per guide here is when it can happen.


Unknown Status

Tasks The Integration Service cannot determine the status of the workflow or task, or the Integration Service does not respond to a ping from the Workflow Monitor. The status may be changing. For example, the status may have been Running but is changing to Stopping.


I felt this status is somewhat interesting and thought I should pay some attention. First possible cause (explained in help), failing to ping IS, is quite convincing. However, I am wondering why a workflow would take "Unknown Status" when it changes from  Running to Stopping like in the example explianed in second possible case. I guess if this has to fit in second case, that status would only for a very brief period - brief enough to change the  status from Running to Stopping for example. Please comment on this second point if someone has better idea.


I am posting this now because I have recently experience some other interesting scenario that lead a workflow to "Unknown Status". So, I thought it is worth sharing this experience...


Workflow changed to "Unknown Status" even before it started running. So, no workflow log file generated. Only next obvious source of information is IS log, and I found below entry (slightly modified as it is classified) in there...


From IS Log:


ERROR Wed Oct 7 00:55:09 2009 1409415520 LM_36317 Error in starting execution of workflow [id = 2651] [w_sales_fact].
ERROR Wed Oct 7 00:55:09 2009 1409415520 LM_36363 Workflow [w_sales_fact]: Cannot rename workflow log file [/opt/app/Informatica/pc/server/infa_shared/WorkflowLogs/w_orders_LOAD.log.bin] to [/opt/app/Informatica/pc/server/infa_shared/WorkflowLogs/w_orders_LOAD.log.1.bin].
INFO Wed Oct 7 00:55:01 2009 1409415520 LM_36814 Workflow [w_sales_fact]: Got scheduled request to start workflow.
INFO Wed Oct 7 00:55:01 2009 1398925664 LM_36814 Workflow [w_orders]: Got scheduled request to start workflow.
INFO Wed Oct 7 00:55:00 2009 183009705024 LM_36813 Workflow [w_sales_fact], run instance name [] is scheduled to run now, schedule last saved by user [karteek].
INFO Wed Oct 7 00:55:00 2009 1388435808 VAR_27029 Use persisted repository value [0] for user-defined workflow/worklet variable:[$$Load_Failed_ctr].
INFO Wed Oct 7 00:55:00 2009 1388435808 VAR_27029 Use persisted repository value [0] for user-defined workflow/worklet variable:[$$Is_LoadFailed].
INFO Wed Oct 7 00:55:00 2009 1388435808 VAR_27027 Use default value [] for user-defined workflow/worklet variable:[$$Delay].
INFO Wed Oct 7 00:55:00 2009 183009705024 LM_36813 Workflow [w_orders], run instance name [] is scheduled to run now, schedule last saved by user [karteek].



The reason it went into unknown status was that there were 2 workflows that were wrongly configured with a same log file name (w_orders_LOAD.log), and also scheduled for some time. At one point of time on thier schedule they were both initiated (acquired  the execution lock) and attempted to create their workflow log files. Unfortunately they were doing the write attempt on same file exactly at same time, and only one was through, but other was left in "Unknown Status".


This is very common thing that we as developers miss coding/naming standanrds, and here in Informaica chances would increase when we copy the same object and customize it for a different purpose. We overlook many things as  the original piece worked quite well. Here it exactly happened the same..."w_orders" was copied from "w_sales_fact" , but missed to change its "w_orders" workflow specific changes, and one such  change being the log file name. We corrected the log file name and fixed that problem forever for that workflow. But how many such other workflows with wrong log filenames was a question. So, I then did some research on Powercenter repository database, and wrote the below query. This basically searches wherever conflicts are there between workflow name and its log file name. You can change the log file name pattern below as per your naming standards.

select subject_area, task_name workflow_name, attr_value cur_log_name, task_name || '.log' new_log_name 
from infav8.rep_task_attr a, infav8.rep_all_tasks b
where attr_name = 'Workflow Log File Name'
and a.task_id = b.task_id
and a.version_number = b.version_number
and task_name || '.log' <> attr_value  -- change pattern in this line
order by 1, 2



Hope it helps!

Update 11/28/2009:
I have collected few more situations that can possibly turn a workflow's status into "Unknown Status".
  • When a PowerCenter repository have versioning enabled and a dependent activity is still checked out when scheduling the workflow - solve this by checking in all the dependent objects (from Informatica KB - ID: 100943)
  • There is a known issue (CR 182478) that has been fixed in PowerCenter 8.6, causing workflow to be in "Unknown Status". To resolve the issue, upgrade to PowerCenter 8.6. However, as a workaround in versions earlier than PowerCenter 8.6, set Save workflow log for these runs to zero (0) in workflow properties (ID: 106278).
Update 12/21/2009
When workflow parameter files is defined (in workflow properties), but if either parameter file is not present or no permissions  to read it, then also workflow goes into Unknown status (verified in 8.6). Error message would be like one below.

(my_is) Start task: ERROR: Workflow [wf_test]: Parameter file [/opt/app/Informatica/pc/server/infa_shared/SrcFiles/test.prm] not found. Please check the Integration Service log for more information.



Karteek