Monday, August 31, 2009

Single Source of Truth - BI Publisher in Siebel

I have written about Siebel reporting with BI Publisher and the reports conversion from Actuate to BI Publisher last week here. However, since the development group in Oracle has been working on some enhancements to improve the reporting and migration process better so at some point in future the information on those posts might become to be staled. So I’m listing the source of information where you can keep yourself updated with the latest information below. Please bookmark and visit them periodically or when you need.

I’ll talk about more about the Siebel reporting with BI Publisher with topics like ‘how to administrate the reports’, ‘how to setup the BI Publisher environment’, etc, from tomorrow, so stay tuned!

Thursday, August 27, 2009

6 Keys to Deliver Successful Reports Conversion Projects

Why Are We Doing This?

I hear often something like ‘we need to convert 100 numbers of reports within next two months, can you do it?’ Well, the answer might be Yes or No, it depends on budget, timeline, and other constraints. But more importantly, do we really need to convert all the 100 reports? We need to look at all the reports by asking, 'How many reports are really used today?’,  ‘How many of the reports are really meeting today’s business requirements?’ and ‘How are those reports serving or impacting their business?’. Without asking such questions and having clear answers, all the money and effort which will be spent for the conversion related work including project preparation, project delivery, production support and maintenance will not get much return, and the worst the project might not get enough sponsor support so that the it might be forced to shut down at the middle of the project.

Energize Your Users and Get Them Excited!

As I have discussed in the previous reports conversion related posts the reports conversion itself involves a lot of manual tasks and always requires a series of testing before going to a production. Also don’t forget that the reports conversion means you are introducing a new paradigm of the reporting system to your users who might have some resistance to move to a new system just because it’s new! and they’re not psychologically comfortable with it. So we need to think about how to support both end users and developers in a form of training, documentation, user group, discussion forum, etc so that not only they will be able to transform as easy as possible and minimize a business continuity risk impact, but also we can get them energized and excited and obtain a positive support and constructive feedback from them for even a bigger and wider project.

Bring the Values to Business!

The reports conversion initiative can be a great opportunity to review the existing reports and reporting framework including its management, administration, and maintenance process. You might be able to reduce huge numbers of the reports, which of course will have big impact on the maintenance and administration cost. Also, you might be able to optimize the reporting framework to server your business better. The new BI Publisher’s RTF Template base reporting design framework allows users to use their most familiar desktop tool, MS-Word, to design the reports layout, which means you can involve the end users to design the reports layout and formatting in more collaborative form and reduce a turn around time to meet their requirements. With the BI Publisher’s RTF Template’s flexible layout/formatting and calculation support, there might be some of the requirements that couldn’t have been done before but now can be done easily. BI Publisher’s separation of data and layout layers development approach allows developers or designers to work off-line mode so that they can work anywhere and anytime.

So here is, ‘6 Keys to Deliver Successful Reports Conversion Project’!

1. Clear Objectives and Value Proposition

It’s very important to set the objectives for the reports conversion project. Why are we doing the conversion ? Is that only because Actuate support will be expired soon ? Oracle has made the decision of converting the old reporting technology to the new BI Publisher base reporting system because there are significant benefits that would server our customers better. And such benefits should be able to be translated to ‘How can those benefits serve our business better?’

Such questions lead to a set of answers which should drive the reports requirements definition and reports and reporting system design. And they can be a base guideline for all the activities during the project hence it helps leading the project team together and deliver a consistent result.

2. Requirements, Requirements, Requirements!

Even though it might take an extra time at the beginning before you start the conversion itself it would be really beneficial for both the business and IT sides to collect and review the reporting requirements based on today’s business environment, and define the requirements in a form that both the business and IT can agree upon. And this will be a baseline for the whole reports conversion exercise including the design, development, system testing, UAT(User Acceptance Testing).

Also before you start the reports conversion you want to analyze the requirements to see how to implement them with this new BI Publisher base reporting technology.

3. Gap Analysis and Solutions Development based on Best Practice

You have two different technologies, old reporting system and new reporting system. This means two different methodology to implement a same requirement.  When you find the gaps between the old and new technology and develop solutions for them then they should be shared with other developers and projects so that each developer won’t need to invent the wheel every time. Also sometimes you might have possibly many ways to implement a same requirement with BI Publisher, so it’s important to analyze which options would be the best and why. These solutions can be used for many requirements and have huge impact on both maintenance and performance.

This analysis requires BI Publisher’s expert and best practice but once you have established that then the conversion process itself would be less stressful and most importantly more predictable hence easier to plan and less surprise!

3. Incremental Release, Not Big Ban

When you need to convert many reports, for example, 100 reports, it would be better to separate into small chunk of the reports sets and have a different releases schedule, maybe by department, by audience type, by reporting type, or by priority. In this way the business can recognize the value of the conversion initiative sooner and can provide valuable feedback that might have impact on the conversion strategy and reports design methodology. Again, you are going to bring a great value to your business by delivering this conversion initiative so close and continuous collaborative working relationship with the business is critical and it’s important to check if the work is aligning with the objectives and serving your business better. By focusing on a smaller user audience not just you can deliver the converted reports sooner and support them better but also you can have them on your side to promote the initiative to bigger audience for your next phases of the project. I would say three months would be typically the maximum to deliver. If it goes more than that then the business requirements would be changed and harder to manage them hence more chance to deliver something that can not serve your users.

4. Right Resource at Right Place

By this time I’m sure you’re fully aware of it, but again, the reports conversion is not just a magic that is supported by some kind of technology or methodology. It involves many phases and different types of works in order to deliver it successfully as the same way you deliver a reports development or any other projects. Therefore it requires different types of resources and you want to allocate them at a right timing and a right place and with right combination. Here is a list of the resources that would contribute the success.

  • Project Manager
  • Business Users/Analyst
  • Functional Developers who are familiar with the backend application and the source tables such as EBS, Siebel, PSFT, JD Edward, etc.
  • BI Publisher - BI Publisher has many technology but you want this BI Publisher resources who have extensive experience especially in RTF Template development

And of course, usual resource requirements for any project delivery applies such as project sponsors, system administrators/DBA, etc.

5. Well Thought and Active User Support

Again, the reports conversion will introduce a new reporting system to your users who have already accustomed to the older reporting system. The way they look at the reporting has already been established on the older system. So it will be high possibility that the users will get confused or start building a negative impact on the initiative. It’s important to understand how the users are running their reports and managing them today from their perspective. And starting from the perspective we need to identify what would be the gaps in terms of the usability and reporting procedure and build a custom training plan to address such gaps. Also, the benefits of what the new reporting system can offer often not been recognized fully by the end users or even by the developers! So it is important to translate the benefits into their daily business environment and educate the values with their language. And if you start with a smaller release with a limited number business users then they can help you to develop such training and to translate and sell the values to the business.

Also, it would help if you start hosting a user support group session at weekly or monthly where the users can throw questions or discuss requirements and solutions. What we have internally is a wiki site where we post any tips/techniques, best practice, FAQ, troubleshooting guide, which we’ve found through the development/conversion exercise to share with others. This is helpful especially when you think about scaling the development/conversion works so the more development work goes the more knowledge and experience you have as a whole team which can translate to reduce the delivery time and cost. So it’s something like ‘Center of Excellence’ as some people call it. But note that hosting such wiki site or ‘Center of Excellence’ will take a leadership effort and commitment, otherwise it will be forgotten after a few weeks, I’m sure you have seen this before! But when it works it really brings an amazing impact.

6. Project Management

Lastly it requires a sounds project management, but that is something that I don’t think I need to emphasize especially at here. Nothing is different, all the methodologies of project management will help to deliver the reports conversion project successfully as it does for any kind of projects.

Wednesday, August 26, 2009

Actuate to BI Publisher Reports Conversion for Siebel – Part 2

Yesterday, we were looking at how to convert Actuate reports to BI Publisher by using the Conversion Assistant tool. Now you got the Actuate reports converted to BI Publisher reports. The next step is how to enable the BI Publisher reports in Siebel application. I’m going to talk about the steps to upload and register the BI Publisher RTF Template files and associate them with relevant Siebel Views today.

Registering BI Publisher Report for Siebel

Once the RTF Template files are ready then you want to upload them to Siebel application and register them with appropriate Integration Objects.

Here is a list of the steps required to register the RTF Template files.

  1. Login to Siebel application
  2. In the Siebel application, navigate to the ‘Administration - BIP Reports’ screen, then ‘Report Template Registration’ view.

    Report_Template_Registration3
  3. In the ‘Report Template Registration’ view, click New.
  4. In the ‘Report Template Registration’ list, type a report name and optionally a description.
  5. Select the RTF Template file, and then fill in the rest of the fields.

    Filed

    Description

    Template

    The RTF report template

    Primary Integration Object

    The primary integration object for the report. The Primary Integration Object field is automatically populated when you select an integration object in the Report Template Integration Object list. This field is read-only in the Report Template Registration list.

    Output Type

    Format of the generated report. Choices are: PDF, HTML, RTF, EXCEL, PPT, or All.

    XLIFF

    XLIFF files contain the strings of the report column, field labels, and captions that are used for translation purposes.

    Start Date

    A start date for the report.

    End Date

    An end date for the report. After the end date passes, the report no longer appears in the Reports Menu.

    Once you fill in the above information now you need to associate a primary integration object for the report

  6. Select the Report Template Integration Objects view tab, and then click New.
  7. Fill in the fields in the new record.

    The fields are described in the following table.

    Field Description
    Integration Object Name Name of an integration object.
    Search Specification This is required only when you have multiple integration objects.
    Primary

    Indicates this integration object is the primary integration
    object.


  8. Perform one of the following:
    A: In connected mode, click Upload Files. (The report template (RTF) and XLIFF files are uploaded.)
    B: In disconnected mode, click Generate XLIFF. (The report template (RTF), XLIFF, and XSL files are uploaded.)

Associate Registered Report with Siebel Application View

After uploading and registering the RTF Template file, you need to associate the report with a Siebel application view so that the users can select the report from the context-sensitive Reports Menu on the associated view page.

Here is a list of the steps required to associate a registered report with a Siebel application view

  1. Navigate to the ‘Administration - BIP Reports’ screen, then ‘View Association’ view

    Report_Association2
  2. In the Views list, select a view for which you want to associate a report
  3. In the Reports List view, click ‘Add’

    Report_List
  4. In the Views dialog box, choose a report name, and then click OK

The report is now available in the Reports Menu drop-down list when you are in the view you have just associated with. Now you want to test this newly added report to see if it works in the Siebel application.

 

Conclusion

Now you have seen how to convert Actuate reports to BI Publisher reports and deliver them in the Siebel application. You can use the Reports Conversion Assistant Utility to convert the Actuate reports to BI Publisher RTF Template, but there are many manual processes getting involved to complete the whole conversion as pretty much the case as any other type of conversion/migration exercise. I have already stated this at the ‘Oracle Reports Migration to BI Publisher in EBS’ post, but I can’t emphasize enough that it’s very critical to set clear objectives on ‘why we are converting all the reports?’ and ‘what values we are bringing into our business?’.  And based on the objectives we need to set a migration strategy and an appropriate project plan.

So I’d like to take a moment to talk about what are the keys to deliver the reports conversion projects successfully, regardless if it’s from Oracle Reports, Actuate, Crystal Reports, on tomorrow. So, stay tuned!

Monday, August 24, 2009

Actuate to BI Publisher Reports Conversion for Siebel – Part 1

Actuate to BI Publisher for Siebel

As you have seen in the previous posts the Siebel Reporting with BI Publisher architecture uses BI Publisher only for the reports layout generation, not for the data generation. It uses the Integration Objects, which are metadata objects in Siebel Business Applications that provide the ability to obtain Siebel data in XML format, within EAI Siebel Adapter to retrieve the data from the Siebel database. And this two layers of the report generation architecture is pretty much the same way as before with Actuate base reporting. Actuate has been used to handle the report layout with Actuate Design file (ROD) and generate report outputs based on data returned by the Integration Objects.

So the reports conversion will focus only for the layout part of the reports and that means you will take care of only the BI Publisher RTF Template side of business.

By converting the layout part of the report you can take advantage of BI Publisher’s easy and flexible development framework and better performance and scalability for the reports generation.

Conversion Process

Here is a list of steps required for the Actuate to BI Publisher reports conversion for Siebel.

  1. Obtain Actuate Report File
  2. Convert Actuate Report File to BI Publisher RTF Template File
  3. Review the converted file and Adjust/Update
  4. Upload the RTF Template and Register
  5. Associate the RTF Template with View

Reports Conversion Assistant Utility

When you start thinking about the conversion process, the first question would be if there is any conversion tool available or do we need to convert one report by one manually ? The answer is, Yes, there is a conversion tool available, which is called BI Publisher Actuate Reports Conversion Assistant. It converts the Actuate report files (BAS) to the BI Publisher RTF Template files.

How to Get the Conversion Assistant Utility?

You can get this tool from the metalink.

  1. Go to the metalink.
  2. Go to Patches & Downloads tab
  3. Click on the Oracle, Siebel and Hyperion Products link
  4. Click on Simple Search
  5. Enter patch number 8968224 in the Search by Patch Number/Name box
  6. Pick generic platform and click 'Go' (You should see files for download)

Before you start the conversion process make sure you have downloaded the above tool and locate to an appropriate folder.

How Much Can be Automatically Converted

The Conversion Assistant does the conversion for the basic objects but not for all. And often that you need to manually add or implement some of the functionality in the RTF Template files based on the original Actuate reports. This is due to different ways to achieve a same functionality between the two products (Actuate vs. BI Publisher).

You can find such objects that are not converted after the conversion by looking into the log file, which is also generated by the tool. The log file indicates the functions in the Actuate reports that are not converted with label names and you can find such label names in the generated RTF file with a tag like ‘<!--->’ as place holders.

Also, you want to review the report layout to see if the layout in the RTF template is matching the original report layout or newly defined reports layout requirements. In order to review the report layout you can preview the report with actual reporting data in a desired format such as PDF quite easily, instead of just looking at the RTF template, by using the BI Publisher MS Word Add-in tool called Template Builder.

How to Use the Conversion Assistant Tool?

The Actuate Reports Conversion Assistant is a Java command line tool and it requires JDK 1.5 or later. The conversion tool takes BAS file as an input therefore you need to generate the BAS file from the original ROD file first before you start the conversion. The below image illustrates the conversion workflow by the Conversion Assistant.

Conversison_Utility

The Conversion Assistant takes the following input parameters.

Parameter

Description

-i

Input file name – BAS file name

-o

Output file name – RTF Template fille name

-l

Log file name

And the syntax to convert a single report is:

run –i <bas filename> -o <output filename> -l <log filename>

You can also convert multiple files together and the syntax to convert multiple reports together is:

run –i <Folder_path_for_bas_files> -o <Folder_path_for_RTF_Files>

The above command will read all the files in the <Folder_path_for_bas_files> location and put the converted files with corresponding names in the <Folder_path_for_RTF_Files> location. By default a log file is created at location <Folder_path_for_RTF_Files>.

Review and Adjust Converted RTF Template

As mentioned above, the conversion assistant does the conversion from Actuate to BI Publisher RTF Template but in most cases it still requires manual adjustments. You can review the log file which indicates all the functions from the original Actuate report that are not converted. Also you can open the RTF Template file with the Template Builder (BI Publisher’s MS-Word Add-in tool), load the sample XML data file, and preview the report output to see if the final report output is meeting the business requirements. When you review the RTF Template file the followings are the typical items that requires such manual adjustments.

  • Layout
  • Formatting (Color, Number formatting, Date formatting, etc)
  • Calculation/Aggregation
  • Condition
  • Image (including company logo)

When you review the template files it is very important to compare with the newly defined requirements definition, which is agreed with business users who will consume the reports. There might be some logics or formatting in the report which are not relevant today’s business process. There might be some new requirements that are important today but not in the original report. Often that reports developers are given the original reports and they try to mimic the original reports for their adjustments and end up spending a lot of time for something that is not even required anymore. That’s why it is critical to have the requirement collection, analysis, and definition first even for this type of reports conversion project and make sure the all the developers who will work on the conversion process would crystal clearly understand the requirements and share how to address the gaps between the Actuate and BI Publisher.

How to Get XML Data File

One more thing for reviewing the report outputs to perform the manual adjustments. As mentioned above, you can use the Template Builder to preview the report outputs with sample data within MS-Word without connecting to the database or Siebel system. In order to preview the report outputs, you need to get a sample XML data file for each report first. You can generate the XML data file from the integration object for which you want to prepare a BI Publisher report. Here are the steps to obtain the XML data file.

  1. Login to Siebel application
  2. Navigate to the ‘Administration - BIP Reports’ screen, then ‘Sample Data File Generation’ view.
  3. In the Sample Data File Generation list, select the integration object for which you want to generate a data file, and then click Generate Data File.

An XML file will be generated with the name of the integration object followed by .xml. For example, if the integration object name is BIP Literature Fulfillment, the file name for the XML file is BIP Literature Fulfillment.xml.

Now you know how to convert the Actuate report files to BI Publisher RTF Template files. In order to generate Siebel reports with the converted BI Publisher RTF Template files you need to upload the RTF Template files and register them, and associate them with related Siebel Views so that the users can generate the reports from the relevant UI. I will talk about these process tomorrow, so stay tuned!

Friday, August 21, 2009

Running BI Publisher Reports in Siebel

After the previous post, now you know how BI Publisher is integrated within the Siebel architecture as the reporting platform. The next question would be, ‘how to run the BI Publisher reports in Siebel?’ and that is the question I’m going to address today. But it’s really a simple process to run the reports. Siebel application is all Web UI so you can login to Siebel application and navigate to a view (UI) where you want to generate a report for. Within a view you can click a ‘Reports’ icon which will show you a list of the reports accessible from the view.

Siebel Reports Menu
Report_Menu

Under ‘BIP’ section you can view a list of the reports that are associated with the current view. You can click one of the listed reports to run the report. Or you can select ‘My BI Publisher Reports’ to see a list of previously run reports, which are saved on the Siebel server.

My Report List View
My_Report_List

Once you select one of the reports a dialog box will pop up where you can select the report output such as PDF, HTML, etc.

Report_Output_Selection

You can select one of the report output format type and it will generate a report in the specified format and display.

Report_Output

And here is a reporting work flow diagram.

image

Well, it’s that simple. But the key is that you can access to only the reports that have been associated with the view you are current at. So somebody has to set that up for you before you start running the reports. And that is something that I will talk about as part of the ‘Actuate to BI Publisher Reports Conversion’ topic, which will follow tomorrow, so stay tuned!

Thursday, August 20, 2009

Siebel Reporting Architecture with BI Publisher

BI Publisher is the Reporting Platform for Siebel

BI Publisher has been introduced to Siebel users since its Release 8.1.1 as a new reporting platform replacing the older Actuate base reporting platform. It is tightly integrated with the Siebel architecture and its applications. The users can run and manage their reports with BI Publisher from Siebel applications UI.

The BI Publisher integration patch for older releases like 7.7, 7.8, 8.0 are currently planned. The detail can be found from a metalink note.

So now you can use BI Publisher to start developing new reports and taking advantage of the a lot of benefits that BI Publisher offers. But how should we do with the existing reports that have been developed with the older Actuate base platform?

That is the question I’d like to address by discussing how to convert the existing Actuate base reports to BI Publisher base reports in Siebel applications. But, before I start talking about the reports conversion, let’s take a look at how the new Siebel reporting works with BI Publisher first.

Two Different Modes for BI Publisher Integration

There are two different ways of integrating BI Publisher as Siebel reporting platform. One is for Connected mode (Siebel Web UI) and another is for Disconnected mode (Siebel Mobile client and Developer web client). This is only a difference in terms of the way BI Publisher is integrated technically and how the reports are generated inside the Siebel application. So from the end user point of view there is no difference when they request to run the reports.

The Connected mode utilizes BI Publisher Server while the Disconnected mode uses internally embedded version of BI Publisher reporting engine, which is a set of Java libraries. I will describe the two different Siebel Reports Architectures and Reporting flow below.

Siebel Reports Architecture and Report Flow Generation for Connected Mode

The Siebel Web Client allows you to run reports in connected mode. When you try to run the reports, the related Integration Objects, which provide the ability to obtain Siebel data in XML format, extracts the relevant data from the Siebel database. A Siebel reporting server (XMLP Report Server Component) issues an outbound Web service call and pass the XML data to a standalone BI Publisher Server. The BI Publisher Server accepts the XML data file, convert a registered RTF Template to XSL-FO template, marry the XML data with the template, and generate a final report output such as PDF. The BI Publisher Server then sends the report output file back to the XMLP Report Server Component using an outbound Web service call. The report is stored in the Siebel File System and then displayed in the Siebel UI.

Siebel Reports Architecture Diagram for Connected Mode

Here is the Siebel Reports Architecture diagram.

image

 

  1. A report generation event is triggered from a view in the Siebel user interface
  2. The Siebel Application Object Manager (AOM) routes the event to Siebel database
  3. The Server Request Broker (SRBroker) and the Server Request Processor (SRProc) monitor the request and pass it to the XMLP Report Server Component through the Report Driver Service
  4. The XMLP Driver Service makes a call to the XMLP Data Service
  5. The data service requests data to the EAI Siebel Adapter
  6. The EAI Siebel Adapter fetches data from the database
  7. The EAI Siebel Adapter returns the data
  8. The XMLP Data Service returns control back to XMLP Driver Service through the data service
  9. The XMLP Driver Service again makes a call to the XMLP Adapter Service
  10. The XMLP Adapter makes a proxy business service
  11. The Proxy Business service makes a Web service call to the BI Publisher Server to execute the report
  12. The BI Publisher Server executes report generation, returns back generated report binary data
  13. The control comes back to the XMLP Adapter Service
  14. The XMLP Adapter Service then downloads the report and creates the file in the Siebel File System
  15. And it displays the report in the UI

 

Siebel Reports Architecture and Report Flow Generation for Disconnected Mode

The mobile clients—Siebel Mobile Web Client and Siebel Developer Web Client, allow you to run Siebel Reports interactively in disconnected mode.
In disconnected mode, the BI Publisher Server is a logical component that uses the BI Publisher XDO Engine to manage report generation. The XMLP Report Business Service is the interface to this logical component that instantiates the EAI Java Business Service to load the JAR files to the JVM (Java Virtual Machine). The BI Publisher XDO Engine then processes the XML data, XSL template, and XLIFF files to output the report.

So again, even with this mode BI Publisher is used only to generate the final report output, not for the data generation part. The reporting data is generated by the Integration Object and the XML data and the relevant RTF Template file are passed to the BI Publisher XDO engine, which will generate a final report output.

Siebel Reports Architecture Diagram for Disconnected Mode

Here is the architecture diagram for Disconnected mode

image

  1. A report generation event is triggered from a view in the Siebel application.
  2. Data is fetched from the database by the EAI Siebel Adapter and stored as XML in the XMLP\DATA folder in the Siebel installation directory.
  3. A call is routed to the XMLP Report Java Business Service.
  4. The XMLP Report Business Service instantiates the EAI Java Business Service, and loads the JAR files to the JVM (Java Virtual Machine).
  5. The BI Publisher XDO Engine loads the XDO classes from the JAR files, and then the XML, XLIFF, and XSL template are given as input to the XDO classes for generating the report. The report is
    temporarily stored in XMLP\REPORTS, and then sent to the Siebel File System.
  6. The control then goes back to the Siebel user interface where the generated report appears.

So this is how Siebel has integrated BI Publisher for each mode (Connected vs. Disconnected) and how the reports are generated within the integration from a technical perspective. Next, I will talk about how to run the BI Publisher reports within Siebel. Stay tuned!

Wednesday, August 19, 2009

Oracle Consulting Service - Oracle Reports to BI Publisher Migration (Conversion)

Before going to the ‘Actuate to BI Publisher Conversion’ topic, I would like to introduce our consulting services prepared specifically to the ‘Oracle Reports to BI Publisher Migration (Conversion)’.

There are two types of the services. One is, ‘Complete Migration Service’, which is to provide a service to convert the entire existing Oracle Reports base reports to BI Publisher base reports. With this service we will provide a full conversion service to enable organizations to reduce administrative, maintenance, and development cost, and to empower business users with the benefit of the new reporting architecture by delivering a complete reports migration available in the EBS instance with our best practice approach and reports migration methodology we have developed in Oracle consulting. This service will provide a full reports migration/development life cycle starting from developing project objectives/project plan and ending with end users training/documentation/production support.

Another is, ‘Reports Migration Accelerator for EBS’, which is to assist organizations to understand the migration process and set up an optimized plan to execute with enough skill sets by reviewing the existing reports and the business/technical requirements and providing hands-on base training with a selected number of actual reports within the organization’s instance.

And of course these are packaged services but can be customizable to suite to each organization’s requirements.

Here are the data sheets for the both services, please download if you would be interested in, and feel free to contact me at Kanichiro.Nishida@oracle.com

Ok, that’s it for the advertisement! And I will start Actuate to BI Publisher reports conversion from tomorrow, so stay tuned!

Tuesday, August 18, 2009

Oracle Reports Migration to BI Publisher in EBS – Part 3

Register BI Publisher Report Template to EBS

After you have completed converting the Oracle Reports RDF file to BI Publisher Data Template and RTF Template with the conversion tool, the next step is to register such template files. You can do so by logging into the EBS web UI with BI Publisher Administrator responsibility and register them. After the registration you need to update the existing concurrent program to use the new Data Template file.

Register Data Template as Data Definition in Template Manager

First, you need to register your Data Template.

  1. Login to EBS
  2. Choose XML Publisher Administrator responsibility
  3. Click Data Definition in the left box
    clip_image002
  4. Click ‘Create Data Definition’ button to create a new Data Definition
    clip_image004
  5. Input Data Definition Name
  6. Input Code
  7. Choose Application. (e.g. Receivables)
  8. Input Start Date
  9. Specify the XML data file as Preview Data. (Optional)
  10. Specify your Data Template file as Data Template
  11. Click Apply to save
    clip_image006

Register RTF Template as Template in Template Manager

Once your Data Template has been registered now you can register your RTF Template.

  1. Click Templates tab
  2. Click ‘Create Template’ button to create a new Template
    clip_image008
  3. Input Template Name
  4. Input Template Code name
  5. Choose Application (e.g. Receivables)
  6. Choose Data Definition
    You need to choose the Data Definition that you have created at the previous process step.
  7. Choose Template Type, in this case it’s ‘RTF’
  8. Specify Start Date
  9. Specify RTF Template file that you have developed
  10. Choose English as Language
  11. Check ‘Translatable’ checkbox
  12. Click ‘Apply’ button to save
    clip_image010

Create a new Concurrent Program with Data Engine

Once you have completed the registration of both Data Template and RTF Template then you want to create a new concurrent program based on the original concurrent program, which is associated with the Oracle Reports base report, and associate it with the templates.

  1. Choose System Administrator responsibility
  2. Go to Concurrent/Program/Define
  3. Select the original Reports Concurrent program
  4. Click ‘Copy to’ to copy the original concurrent program

In a popup window,

  1. Type Program name
  2. Type Short Name (This must be identical to the Data Definition Code that you have registered in Template Manager.)
  3. Choose Application, e.g. Receivable (AR)
  4. Check both ‘include Incompatible Program’ and ‘include Parameters’
  5. Choose ‘XDODTEXE’ as Executable
    This is XML Publisher Data Engine Java Concurrent program that takes Data Template and extract data based on the template information and generate XML file
  6. Choose XML as Output Format
  7. Save
    clip_image012

Assign the new Concurrent Program to a Request Group

Now you need to assign the newly created concurrent program to a request group that your users have access to, otherwise the users can not find the new concurrent program to run.

  1. Go to Security/Responsibility/Request
  2. Select a Request Group which is assigned to your responsibility that you want to run the request with. (e.g. Receivables All for Receivables Manager)
  3. Add the new Concurrent Request program and Save
  4. Make sure the users’ responsibility has the Request Group
    clip_image014

Run the Report Request

Now you want to test this report to see all the registration and configuration at the previous steps have been done appropriately.

  1. Go to Submit Request Window or select View/Request from the tool bar menu
  2. Choose the new Concurrent Request program
  3. Select parameter values if any.
  4. You should see the RTF Template you have registered in the Template Manager.
    clip_image016
  5. Click Submit

Once the request is run with complete status successfully now you should be able to open the PDF report by clicking ‘View Output’. If the request has completed with an error click ‘View Log’ to find the detail problem.

Conclusion

After the ‘Oracle Reports Migration to BI Publisher in EBS Part 1, 2, and 3’, now you have 1) converted the original Oracle Reports base report to BI Publisher base report which consists of Data Template and RTF Template, 2) registered the converted template files with EBS, 3) created a new concurrent program by copying the original and associate it with the templates 4) validated the converted report.

This is a normal process of the reports conversion from Oracle Reports to BI Publisher for EBS. Of course, as discussed at the beginning, you can convert only the layout part of the report and keep using Oracle Reports as a data generation engine. In this case you need to register only the RTF Template and associate it with the original concurrent program through Data Definition.

Again, the most critical thing is not just the technical side of the conversion process, it’s more the business side. While administration and maintenance cost in IT departments will be reduced by converting to BI Publisher reporting architecture it’s important to note that there can be a huge value at the business side who will be the users and could see a lot of potential values with this new reporting architecture and this reports conversion initiative itself.

For example, by moving to BI Publisher reporting architecture now the users can use MS-Word to design the reports by themselves. With RTF Template the users can have images, charts, table of contents, URL hyper link, etc in their reports easily. Separation of the layout design piece from the data definition enables the users to make changes on the layout easier with less risk. BI Publisher’s reporting engine has better performance and scalability than Oracle Reports.

So not only delivering the x number of the reports being converted on time, but also it’s important to discuss how to meet the business requirements in today’s business condition and how the new reporting architecture can improve their business and set clear objectives to be shared through the conversion project phases. And such objectives should drive on what conversion option to be chosen and how to set the conversion plan and execute.

So that’s it for the reports migration (conversion) for EBS. I will start another reports conversion topic, ‘Actuate to BI Publisher conversion for Siebel’ from tomorrow, so stay tuned!

Monday, August 17, 2009

Oracle Reports Migration to BI Publisher in EBS – Part 2

Run the Conversion Command Line Tool

The conversion utility is a Java command line tool and requires a set of BI Publisher related Java library (JAR) files, which you can find from your BI Publisher Server or Desktop installation. In order to run the conversion utility you need to set CLASSPATH with the library files first. If you have BI Publisher server installed then the below is an example location where you can find such JAR files.

e.g. D:\OracleBI\oc4j_bi\j2ee\home\applications\xmlpserver\xmlpserver\WEB-INF\lib

Here is a list of the library files that are required to be set in the CLASSPATH.

  • Collections.jar
  • xmlparserv2-904.jar
  • xdocore.jar
  • aolj.jar

The conversion command line tool takes the following parameters:

Parameter Name

Required

Description

source

Must

Source directory for Oracle Reports files. All reports must be in the same format - either RDF or XML.

target

Must

Target directory to create Oracle BI Publisher report objects. This includes the Oracle BI Publisher Report file (.xdo), the layout template file (.rtf), the PL/SQL package, and log file. This folder must exist before the command is run.

oraclehome

Optional

If your reports are in Oracle Reports XML format do not specify this parameter. If your reports are not in Oracle Reports XML format, specify the Oracle home path where Oracle Report Designer (9i or later version) is installed.

debug

Optional

To run the utility in debug mode and write debug statements to the log file.

Here is an example of how you can run the command line tool. First make sure you have set the CLASSPATH, then run the tool.

>set CLASSPATH= D:\lib\collections.jar; D:\lib\xmlparserv2-904.jar; D:\lib\xdocore.jar;D:\lib\aolj.jar

>java.exe oracle.apps.xdo.rdfparser.BIPBatchConversion -source D:\Reports\ARCUST.rdf –target C:\Reports\output -oraclehome D:\DevSuiteHome_1012 –debug

After you run the tool successfully you should get a set of files including Data Template, RTF Template, PLSQL package files, log file, etc. Now you need to review the log file, validate each file and perform the post conversion tasks.

Validate and Compile PLSQL Package

PLSQL Package specification and body files are generated from the conversion tool. This PLSQL package can be called from BI Publisher Data Template as a report trigger. It can be called before the report run (Before Trigger) or after (After Trigger). This PLSQL package needs to be reviewed and you need to manually run the files against a database to create. This is required only when you use Data Template to generate the reporting data. If you will keep using Oracle Reports to generate the data then you don’t need the PLSQL package to be created in the database.

 

Validate Data Template

Now you need to validate the Data Template file. Make sure all the SQL queries are right and the SQL columns and XML elements are mapped appropriately. When you have multiple queries and they have parent-child relationship the XML structure definition might get complex so you want to make sure the grouping structure in the XML structure definition is defined correctly. Also, the Before/After triggers are called in this Data Template so you need to review the PLSQL package and function names. Lastly the parameter definition needs to be reviewed carefully. Any parameters which are used in the PLSQL package are also need to be defined here correctly.

Validate RTF Template

The layout and formatting are very hard to be precise especially when the reports are pixel-perfect and have complicated logics. So you want to review the converted RTF Template and start updating it to mimic the original report layout or follow the newly defined report layout requirements.

When you validate the RTF Template you would need a XML data file to preview the report final output in a certain format such as PDF. You can download the XML data file by running the original concurrent program after changing its output format to ‘XML’.

Post Migration Tasks - Check Log File

When you look at the log file, which is generated by the conversion tool, you will find the issues encountered during the conversion and manual changes required to complete the report. The below is a part of an example log file content, which shows one object called ‘F_COMM’ was not converted by the tool. 

The following format triggers which are the part of RDF,

has not been supported in this Template:

--------------------------------------------------------------------------

Object Name : F_COMM

=========================================================

PL/Sql code

function F_COMMFormatTrigger return boolean is

This is because the ‘F_COMM’ uses ‘F_COMMFormatTrigger’ formatting trigger, in Oracle Reports and that is not supported by the conversion tool to be converted. So you need to manually convert this functionality into the RTF Template file by using ‘Conditional Formatting’ functionality.

So make sure that you review the log file and see what is not converted and manually add such functionality to the RTF Template file if necessary. Not being converted by the tool doesn’t mean BI Publisher can not support such. Any functionality available in Oracle Reports can be supported by BI Publisher, it’s just a different way to implement the same functionality. One thing to note, since BI Publisher report consists of two parts, one is a data generation and another is layout and formatting part. So something to do with data such as calculation, aggregation, condition, etc can be done by either Data Template or RTF Template. It really depends on the requirements. In general it’s better to do such in Data Template because 1) you can use the database power to do such calculation hence the performance tend to be better 2) once you have implemented the logic in the Data Template then you can reuse with multiple RTF Template files instead of implementing it in each RTF Template files.

Post Migration Tasks – Others

There might be some other items that you want to add to the converted RTF Template file than the ones you find in the log file. For example you might want to add images or do an appropriate number/date formatting. When you find some differences after the validation of the RTF Template file this is the time to fill the gap by implementing such functionality with BI Publisher methodology.

Next Step

After you have completed all the tasks above now you are ready to start registering the converted BI Publisher report template files to Oracle EBS environment to complete the overall conversion process. I will cover how to register the files to EBS tomorrow. So stay tuned!

Friday, August 14, 2009

Oracle Reports Migration to BI Publisher for EBS – Part 1

BI Publisher (XML Publisher) in EBS

Oracle EBS is the first application which had integrated with BI Publisher more than 5 years ago and converted its main reporting platform to BI publisher from Oracle Reports and now it's providing over 2000 pre-seeded BI Publisher base reports. So the integration is pretty mature and robust. Because of this long history BI Publisher is still called as ‘XML Publisher’ in EBS, which is the original product name before changing it to ‘BI Publisher’.

 

Two Parts of BI Publisher Reports Definition

As you know BI Publisher has separated the report query and layout to different layers and uses Data Template for the query portion and RTF Template for the layout so users can focus on just the data query or just the report layout without bothering the other. If you want to change the report layout you can make such modification in the RTF Template without touching the Data Template. This has brought a much flexible reports development approach and reduced the maintenance cost.

This two parts of the reports definition means you can use just one of them with another technology to generate final reports output. For example, if you already have an application or framework to produce a data in a XML format then you can just use BI Publisher RTF Template to generate reports on top of the XML data. This also applies to the Oracle Reports base EBS reporting, where you can use Oracle Reports to generate the XML data and use BI Publisher to generate report outputs based on the data.

Complete Conversion vs. Halfway Conversion?

In EBS you have two options to convert your Oracle Reports base reports. One is a complete conversion, which converts Oracle Reports RDF to BI Publisher RTF Template and Data Template. After the conversion you will have just BI Publisher to generate the data and the report outputs. Another is a halfway conversion, which converts only for the layout portion to RTF Template but keeps the data query portion so Oracle Reports still keep taking care of querying and generating the data and BI Publisher will take the XML data and marry with RTF template to generate final report outputs.

Both options have pros and cons. The complete migration option would more make sense if you think about a long term. Most importantly you don't want to have two different tools to develop one report, which would cause you to keep Oracle Reports resources and BI Publisher resources including machine resources, human resources. Also BI Publisher's query generation engine in general produce better performance, typically 40% or more so why not just convert everything to BI Publisher ?

On the other hand, BI Publisher doesn't offer UI base Data Template builder so you need to develop necessary SQL queries by using your SQL tools while Oracle Reports offers a UI base Query Builder where you can define queries, parameters, grouping, etc graphically. So some users who might have been used to the graphical tool to develop such data model might have a hesitation to give it up. Also, converting the data query portion means more work hence higher cost for the entire conversion project itself in a short term. If the main objective of the migration is to take advantage of the BI Publisher's flexible and easy pixel perfect production ready reports layout development capability then you might want to convert only the layout portion first then convert the data model part later.

I’m going to focus on how to convert the Oracle Reports base reports to BI Publisher base reports including both the query part (Data Template) and layout part (RTF Template) first. The detail of how to register the migrated Data Template and RTF Template will be discussed separately in another post.

Reports Conversion Process

The migration process involves the following steps

  1. Obtain the original Oracle Reports report definition file (either as an .RDF file or an XML output of the report) that needs to be migrated to BI Publisher.
  2. Run the BI Publisher Reports conversion utility
  3. Deploy the PLSQL package to the database. (Optional)
  4. Validate and Update Data/RTF Template
  5. Add additional layout/formatting requirements to RTF Template (Optional)
  6. Register the Data/RTF Template to XML Publisher Repository in EBS

Conversion Tool Overview

The conversion process uses a reports conversion utility, which is a Java command line tool and automatically converts Oracle Reports definition files to BI Publisher related files. Here is a list of the files generated by the tool after the conversion.

  1. BI Publisher Report definition file that includes the Data Model (.xdo)

    This is an xml formatted file that contains BI Publisher report definition. This is required only if you want to use BI Publisher Enterprise Server (Standalone version) instead of EBS embedded version to manage and run the reports.
  2. BI Publisher Data Template file (.xml).

    This is a xml file containing the report data model definition. This is required when you want to convert to a full BI Publisher. If you want to keep your Oracle Reports to generate XML data then you don’t need this file.
  3. PL/SQL Package specification and body creation file (.pls)

    This is a PLSQL package file, which contains the plsql code used in Report triggers. This PLSQL package should be deployed to the database. This package will be called in the BI Publisher report from the Data Template as Before/After triggers to provide the same functionality as the Before/After triggers of Reports file. Again, you need this only for the complete migration.
  4. BI Publisher RTF Layout Template (.rtf)

    This is the RTF Template file, which has the report layout definition. Users can open this file with MS Word and modify the layout and formatting if required.
  5. Log file (.log)

    This is a log file that contains issues/warnings during the conversion and a log of unconverted objects from the Reports definition file (RDF)
  6. List of values file (.xml)

    This file is generated when the parameters in the Oracle Reports file use List of values.

Manual Tasks after the Conversion

Although the conversion utility does the reports conversion and allows the overall conversion experience much easier, it doesn’t complete an end to end conversion process. After the automatic reports conversion with the tool you will still need to perform the post conversion tasks, which include deployment of the PLSQL package, manual layout adjustment, additional conditional formatting and calculations for the RTF Template, etc. Depends on how complex the original reports are and the result of the conversion by the tool the time required for the post conversion tasks would be different and it would go longer. Therefore, before you start the conversion it is very critical to review and define the original reports requirements so that you can validate the converted report outputs based on the defined requirements and meet your business requirements efficiently with the conversion process without spending too much time on something that is not required anymore in today’s business environment.

Here is a list of typical manual tasks required for RTF Template. :

  1. PLSQL Format Triggers – In Oracle Reports you can use PLSQL format triggers to format data in the reports. The utility creates a placeholder for the all PLSQL format triggers but without adding any functionality. You need to write code in the BI Publisher RTF Template to enable the formatting trigger functionality.
  2. Color Coding of the Layout - You need to add all color coding manually.
  3. Number/Date Formatting - The BI Publisher report uses its default data formats and not the formats in the original report. Items need to be assigned formats manually.
  4. Images – Images are not converted. You need to manually add the images.

Converts RDF or XML?

With the BI Publisher reports conversion utility you can start the conversion with either Oracle Reports definition files (RDF) or XML format report definition files which can be generated with Oracle Reports 9i or later version. If you want to convert from the RDF files then you need Oracle Reports Developer to be installed on your machine where you will run the BI Publisher reports conversion utility. The conversion utility internally calls the Oracle Reports Developer and convert the RDF files to XML files first then it will convert the XML files to a set of BI Publisher report related files. If you don’t have the Oracle Reports Developer installed yet you can download it from here.

In this post I will use the RDF file. You can copy the RDF file under the EBS Apple Top on the file system. Once you obtain the RDF file you’re ready to start the conversion process. I’ll cover the step-by-step process tomorrow, so stay tuned!

Thursday, August 13, 2009

BI Publisher Reports Conversion

So, we are back after almost 2 months long break! :) We were working on many other BIEE/BI Publisher projects and realized it’s already August! hence haven’t updated this blog for that long…

Anyway, back to the subject, today I’m going to talk about the BI Publisher reports conversion. As most of you have already known, BI Publisher is a Oracle’s strategic Fusion Apps reporting platform. We had many different reporting technology such as Oracle Reports for Oracle E-Business Suite, Crystal Reports for PeopleSoft, Actuate for Siebel, etc, but Oracle had decided to use BI Publisher as a standard reporting platform and have converted each application’s reporting architecture to BI Publisher based architecture.

BI Publisher is tightly integrated in each application and offer UI pages to create, manage, and generate reports sourcing data out from the application transaction system. If you’re interested further here is a link for more information.

Also, Oracle has already converted the pre-seeded reports from the old technology base to new BI Publisher base reports so you can see many of the reports being provided with BI Publisher reports templates (RTF Template) and can generate them with the new reporting architecture.

Now, you can start developing new reports with BI Publisher’s RTF Template and Data Template (Data Template is only available for EBS) upload them to the application. But how about the existing reports that you have already developed with the old reporting platform ? Depends on how long you have your application and your specific situation but most likely there should be numbers of existing reports already and obviously you don’t want to keep two types of reports with two different reporting platforms.

And here comes the BI Publisher reports conversion. I will talk about this topic over the next couple of days to cover the following types.

  • Oracle Reports to BI Publisher for E-Business Suite
  • Actuate Reports to BI Publisher for Siebel
  • Crystal Reports to BI Publisher for PeopleSoft

So stay tuned!