Why Hadoop? The power of Hadoop and Who uses it?

Why Hadoop? The power of Hadoop and Who uses it?

Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce, where the application is divided into many small fragments of work, each of which may be executed or reexecuted on any node in the cluster. In addition, it provides a distributed file system (HDFS) that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both Map/Reduce and the distributed file system are designed so that node failures are automatically handled by the framework.

Map/Reduce is a programming paradigm that was made popular by Google where in a task is divided in to small portions and distributed to a large number of nodes for processing (map), and the results are then summarized in to the final answer (reduce). Google and Yahoo use this for their search engine technology, among other things.
Hadoop is a generic framework for implementing this kind of processing scheme. As for why it kicks ass, mostly because it provides neat features such as fault tolerance and lets you bring together pretty much any kind of hardware to do the processing. It also scales extremely well, provided your problem fits the paradigm.
You can read all about it on the website.
As for some examples, Paul gave a few, but here's a few more you could do that are not so web-centric:

• Rendering a 3D film. The "map" step distributes the geometry for every frame to a different node, the nodes render it, and the rendered frames are recombined in the "reduce" step.
• Computing the energy in a system in a molecular model. Each frame of a system trajectory is distributed to a node in the "map" step. The nodes compute the the energy for each frame,
  and then the results are summarized in the "reduce" step.

Essentially the model works very well for a problem that can be broken down in to similar discrete computations that are completely independent, and can be recombined to produce a final result.

References:

When Should You Use Hadoop?
http://www.readwriteweb.com/cloud/2011/01/when-should-you-use-hadoop.php

Employees From Yahoo, Google, And Facebook Are Flocking To These Start-Ups:
http://www.businessinsider.com/employees-from-yahoo-google-and-facebook-are-flocking-to-these-start-ups-2011-12?utm_source=twitterfeed&utm_medium=twitter

Apache Hadoop:
http://hadoop.apache.org/
http://wiki.apache.org/hadoop/

Quote of the Day!

A great quote from Dr. Kalamji-

"The Dream Is Not What You See In Sleep, Dream Is Which Does Not Let You Sleep!!!"

JTA Timeout Errors - BPEL 10g

JTA Timeout Errors - BPEL 10g

As a BPEL Developer, you may have come across many times the below error -

"JTA transaction is not present or the transaction is not in active state. The current JTA transaction is not present or it is not in active state when processing activity or instance "1255921-BpInv2-BpSeq2.5-5......"

Basically, you will get this JTA timeout error if some of the partner links in your BPEL are taking so much of time than the default specified transaction time value on the OC4J app server for BPEL execution. Then, you need follow the below steps to tune your BPEL engine to take care of the transaction processing.

Set the transaction-timeout and syncMaxWaitTime properties as follows:

Property	File Location	This Value Must Be...	Example
transaction-timeout	SOA_Oracle_Home\j2ee\home\config\transaction-manager.xml	Larger than the transaction-timeout value in orion-ejb-jar.xml and the syncMaxWaitTime value.	7200
transaction-timeout	SOA_Oracle_Home\j2ee\home\application-deployments\orabpel\ejb_ob_engine\orion-ejb-jar.xml	Less than the transaction-timeout value in transaction-manager.xml. Note: You must set all transaction-timeout properties that display in this file.	3600
syncMaxWaitTime	SOA_Oracle_Home\bpel\domains\domain_name\config\domain.xml where domain_name is the name of the domain to which you are deploying.	Less than the transaction-timeout value in orion-ejb-jar.xml.	240

Reference: http://docs.oracle.com/cd/B31017_01/integrate.1013/b28981/app_trblshoot.htm

11g OSB Installation separately with Derby Server

11g OSB Installation separately with Derby Server:-

OSB 11g Installationn with Derby Server:-

Required Softwares:

wls1033_oepe111150_win32.exe

ofm_osb_generic_11.1.1.3.0_disk1_1of1.zip

--------

First, Install WLS with OEPE

wls1033_oepe111150_win32.exe

then Install OSB 11g

C:\Siva\Softwares\11g OSB with Derby server\ofm_osb_generic_11.1.1.3.0_disk1_1of1\osb\Disk1>setup.exe -jreLoc C:\Siva\Installations\11gOSB\Oracle\Middleware\jdk160_18

-----------

Then configure the OSB domain

C:\Siva\Installations\11gOSB\Oracle\Middleware\oracle_common\common\bin> config.cmd

Even if it fails with 1527 connection to the Derby Server, please move ahead

------------

Starting and Stopping the OSB Server

C:\Siva\Installations\11gOSB\Oracle\Middleware\user_projects\domains\base_domain>startWebLogic.cmd

URLs:

http://localhost:7001/sbconsole

http://localhost:7001/console
----------------

 

Retrieving or Selecting key/values from Array list without using while loop in BPEL

Retrieving or Selecting key/values from Array list without using while loop

We usually make use of while loop to select indexed key/value pair in the middle of execution of BPEL and this is too cumbersome and performance not good. Hence, use the below xpath syntax to get value for the respective key from the array list of key/value pairs.

/client:ArrayTestProcessRequest/client:cust[contains(client:Name,'Siva')]/client:Value

/client:ArrayTestProcessRequest/client:cust[client:Name='Siva']/client:Value

Respective schema would be like:

 

 

  

   

    

     

      

      

     

    

   

  

 

  

   

    

     

      

      

     

    

   

  

 

Step by Step Installation of SOA Suite 11g with OSB - 11gPS3

I had installed SOA Suite 11g many times on various environments and jotted down the steps I had followed and felt like this would be useful to all of us and posting the same here.

If you are going to install the SOA Suite 11g, kindly have a look at the steps below for clear understanding:


SOA PS3 11.1.1.4 Installationn with OSB:-

Installing the Oracle XE:

OracleXEUniv.exe

sys/manager

and then tune the DB by using the below SQLs

connect sys/manager@XE as sysdba
show parameter session
show parameter processes
alter system reset sessions scope=spfile sid='*';
alter system set processes=300 scope=spfile;
shutdown immediate
startup
show parameter session
show parameter processes

--
Installing the WLS:

the installable should contais the WLS + Coherence + OEPE

wls1034_oepe111161_win32.exe

--
Creating the schemas using RCU:

ofm_rcu_win_11.1.1.4.0_disk1_1of1.zip

pre-check before running RCU

set RCU_JDBC_TRIM_BLOCKS=TRUE
echo %RCU_JDBC_TRIM_BLOCKS%

D:\Softwares\PS311114\ofm_rcu_win_11.1.1.4.0_disk1_1of1\rcuHome\BIN>rcu.bat

Password for the schema users: welcome1

--
Installing the SOA Suite:

ofm_soa_generic_11.1.1.4.0_disk1_1of2.zip
ofm_soa_generic_11.1.1.4.0_disk1_2of2.zip

D:\Softwares\PS311114\ofm_soa_generic_11.1.1.4.0_disk1_1of2\Disk1>setup.exe -jreLoc D:\ICE\Oracle\Middleware\jdk160_21

--
Installing the OSB:

V24316-01.zip

D:\Softwares\PS311114\osb\V24316-01\Disk1>setup.exe -jreLoc D:\ICE\Oracle\Middleware\jdk160_21

--

Configuring the OSB and SOA:

D:\ICE\Oracle\Middleware\oracle_common\common\bin>config.cmd
--

Installing the JDeveloper:

jdevstudio11114install.exe

and then SOA Compositor extension.

soa-jdev-extension.zip

Finally, Happy SOA Development!!!
----------------------------------------------------------------------------