org.griphyn.cPlanner.transfer.refiner
Class Chain

java.lang.Object
  org.griphyn.cPlanner.transfer.AbstractRefiner
      org.griphyn.cPlanner.transfer.MultipleFTPerXFERJobRefiner
          org.griphyn.cPlanner.transfer.refiner.Default
              org.griphyn.cPlanner.transfer.refiner.Chain

All Implemented Interfaces:

Refiner

public class Chain

extends Default

This transfer refiner builds upon the Default Refiner. The defaul Refiner allows the transfer of multiple files in a single condor job. However, it adds the stage in transfer nodes in parallel leading to multiple invocation of the globus-url-copy at remote execution pools, while running huge workflows. This refiner, tries to circumvent this problem by chaining up the stagein jobs instead of scheduling in parallel. This works best only when the top level of the workflow requires stage in jobs. The correct way is that the traversal needs to be done breath first in the TransferEngine.java.

Version:

$Revision: 1.2 $

Author:

Karan Vahi, Gaurang Mehta

Nested Class Summary

private class	Chain.SiteTransfer A container to manage the transfer jobs that are needed to be done on a single site.
private class	Chain.TransferChain A shallow container class, that contains the list of the names of the transfer jobs and can return the last job in the list.

Field Summary

static String	DEFAULT_BUNDLE_FACTOR The default bundling factor that identifies the number of transfer jobs that are being created per execution pool for the workflow.
static String	DESCRIPTION A short description of the transfer refinement.
protected PoolInfoProvider	mSCHandle The handle to the Site Catalog.
private Map	mSIBundleMap The map containing the stage in bundle values indexed by the name of the site.
private Map	mStageInMap A map indexed by execution sites.

Fields inherited from class org.griphyn.cPlanner.transfer.refiner.Default

mFileTable, mLogMsg, mReplicaMechanism

Fields inherited from class org.griphyn.cPlanner.transfer.AbstractRefiner

mDAG, mLogger, mPOptions, mProps, mTPT, mTXInterImplementation, mTXStageInImplementation, mTXStageOutImplementation

Fields inherited from interface org.griphyn.cPlanner.transfer.Refiner

INTER_POOL_PREFIX, REGISTER_PREFIX, STAGE_IN_PREFIX, STAGE_OUT_PREFIX

Constructor Summary

Chain(ADag dag, PegasusProperties properties, PlannerOptions options)
The overloaded constructor.

Method Summary

void	addRelation(String parent, String child, String site, boolean parentNew) Adds a new relation to the workflow.
void	done() Prints out the bundles and chains that have been constructed.
String	getDescription() Returns a textual description of the transfer mode.
int	getSiteBundleValue(String site, String key) Determines the bundle factor for a particular site on the basis of the key associcated with the underlying transfer transformation in the transformation catalog.

Methods inherited from class org.griphyn.cPlanner.transfer.refiner.Default

addInterSiteTXNodes, addJob, addRelation, addStageInXFERNodes, addStageOutXFERNodes, addStageOutXFERNodes, constructFileKey, createRegistrationJob

Methods inherited from class org.griphyn.cPlanner.transfer.MultipleFTPerXFERJobRefiner

loadImplementations

Methods inherited from class org.griphyn.cPlanner.transfer.AbstractRefiner

isSiteThirdParty, logConfigMessages, runTPTOnRemoteSite

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_BUNDLE_FACTOR

public static final String DEFAULT_BUNDLE_FACTOR

The default bundling factor that identifies the number of transfer jobs that are being created per execution pool for the workflow.

See Also:

Constant Field Values

mSCHandle

protected PoolInfoProvider mSCHandle

The handle to the Site Catalog. It is instantiated in this class.

mSIBundleMap

private Map mSIBundleMap

The map containing the stage in bundle values indexed by the name of the site. If the bundle value is not specified, then null is stored.

mStageInMap

private Map mStageInMap

A map indexed by execution sites. Each value is a SiteTransfer object, that contains the Bundles of stagin transfer jobs.

See Also:

Chain.TransferChain

DESCRIPTION

public static final String DESCRIPTION

A short description of the transfer refinement.

See Also:

Constant Field Values

Constructor Detail

Chain

public Chain(ADag dag,
             PegasusProperties properties,
             PlannerOptions options)

The overloaded constructor.

Parameters:

dag - the workflow to which transfer nodes need to be added.

properties - the PegasusProperties object containing all the properties required by Pegasus.

options - the options passed to the planner.

Method Detail

addRelation

public void addRelation(String parent,
                        String child,
                        String site,
                        boolean parentNew)

Adds a new relation to the workflow. In the case when the parent is a transfer job that is added, the parentNew should be set only the first time a relation is added. For subsequent compute jobs that maybe dependant on this, it needs to be set to false.

Specified by:

addRelation in interface Refiner

Overrides:

addRelation in class Default

Parameters:

parent - the jobname of the parent node of the edge.

child - the jobname of the child node of the edge.

site - the execution site where the transfer node is to be run.

parentNew - the parent node being added, is the new transfer job and is being called for the first time.

getSiteBundleValue

public int getSiteBundleValue(String site,
                              String key)

Determines the bundle factor for a particular site on the basis of the key associcated with the underlying transfer transformation in the transformation catalog. If none specified in transformation catalog then one is picked up from the site catalog. If the key is not found in the site catalog too , then the global default is returned.

Parameters:

site - the site at which the transfer job is being run.

key - the bundle key whose value needs to be searched.

Returns:

the bundle factor.

See Also:

DEFAULT_BUNDLE_FACTOR

done

public void done()

Prints out the bundles and chains that have been constructed.

Specified by:

done in interface Refiner

Overrides:

done in class Default

getDescription

public String getDescription()

Returns a textual description of the transfer mode.

Specified by:

getDescription in interface Refiner

Overrides:

getDescription in class Default

Returns:

a short textual description