Documentation

MapReducer

Define parallel execution environment for mapreduce and tall arrays

描述

example

MapReducerdefines the execution environment forMapReduce或高阵列。使用MapReducerfunction to change the execution environment to use a different cluster or to switch between serial and parallel development.

The default execution environment uses either the local MATLAB®session, or a parallel pool if you have Parallel Computing Toolbox™. If you have Parallel Computing Toolbox installed, when you use thetall或者MapReducefunctions, MATLAB automatically starts a parallel pool of workers, unless you have changed the default preferences. By default, a parallel pool uses local workers, typically one worker for each core in your machine. If you turn off the自动创建平行池选项,如果要使用并行资源,则必须明确启动池。看指定您的平行首选项.

When working with tall arrays, useMapReducerto set the execution environment prior to creating the tall array. Tall arrays are bound to the current global execution environment when they are constructed. If you subsequently change the global execution environment, then the tall array is invalid, and you must recreate it.

Note

在MATLAB中,您无需使用MapReducerbecauseMapReduce算法和高阵列计算仅在本地MATLAB会话中自动运行。如果您还具有并行计算工具箱,则可以使用其他MapReducer在此页面上列出的配置选项并行运行。如果您的Matlab Compiler™,则可以使用单独MapReducerconfiguration options for running in deployed environments.

看:MapReducer在MATLAB文档中,或MapReducer在MATLAB编译器文档中。

MapReducer没有输入参数会创建一个新的MapReducerexecution environment with all the defaults and sets this to be the currentMapReduce或高阵列执行环境。您可以使用GCMR获取当前的MAPREDUCER配置。

  • If you have default preferences (自动创建平行池已启用),您尚未打开平行池,然后MapReduceropens a pool using the default cluster profile, setsGCMR基于此池并返回此MAPREDUCER。

  • 如果您打开了平行池,那么MapReducersetsGCMR基于当前池并返回此MAPREDUCER。

  • If you have disabled自动创建平行池, and you have not opened a parallel pool, thenMapReducersetsGCMR基于本地MATLAB会话到MAPREDUCER,并MapReducer返回此MAPREDUCER。

example

MapReducer(0)specifies thatMapReduce或者tallarray calculations run in the MATLAB client session without using any parallel resources.

MapReducer(poolobj)指定并行执行的并行池MapReduce或高阵列。poolobjis aParallel.Pool目的。The default pool is the current pool that is returned or opened byGCP.

MapReducer(hadoopCluster)指定hadoop®群集以并行执行MapReduce或高阵列。hadoopClusteris aParallel.Cluster.hadoop目的。

MapReducer(先生)sets the global execution environment forMapReduce或使用先前创建的MapReducer对象的高阵列,先生, 如果它是ObjectVisibility属性是'On'.

先生= mapReducer(___)returns a MapReducer object to specify the execution environment. You can define several MapReducer objects, which enables you to swap execution environments by passing one as an input argument toMapReduce或者MapReducer.

先生= mapReducer(___,'ObjectVisibility','Off')隐藏MAPREDUCER对象的可见性,先生, using any of the previous syntaxes. Use this syntax to create new MapReducer objects without affecting the global execution environment ofMapReduce.

例子

Develop in Serial and Then Use Local Workers or Cluster

如果您想在串行中开发,不使用本地工人或指定的集群,请输入:

MapReducer(0);
如果您使用MapReducerto change the execution environment after creating a tall array, then the tall array is invalid and you must recreate it. To use local workers or your specified cluster again, enter:
MapReducer(gcp);

MapReducerwith自动创建平行池关闭

如果您关闭了自动创建平行池选项,如果要使用并行资源,则必须明确启动池。看指定您的平行首选项for details.

The following code shows how you can useMapReducerwithout input arguments to set the execution environment to your local MATLAB session and then specify a local parallel pool:

>> MapReducer >> Parpool('local',1);
使用“本地”配置文件...连接到1名工人。
>>收集(最小(rand(rand(1000,1))))
Evaluating tall expression using the Local MATLAB Session: Evaluation completed in 0 sec ans = 5.2238e-04

输入参数

全部收缩

并行执行的池,指定为Parallel.Pool目的。

例子:poolobj = GCP

hadoop群集用于并行执行, specified as aParallel.Cluster.hadoop目的。

例子:hadoopCluster = parallel.cluster.Hadoop

Output Arguments

全部收缩

执行环境MapReduce和高阵列, returned as a MapReducer object.

如果是ObjectVisibilityproperty of先生设定为'On', then先生定义所有的默认执行环境MapReduce算法和高阵列计算。如果是ObjectVisibility属性是'离开',你可以通过先生as an input argument toMapReduceto explicitly specify the execution environment for that particular call.

You can define several MapReducer objects, which enables you to swap execution environments by passing one as an input argument toMapReduce或者MapReducer.

尖端

One of the benefits of developing your algorithms with tall arrays is that you only need to write the code once. You can develop your code locally, then useMapReducerto scale up and take advantage of the capabilities offered by Parallel Computing Toolbox,MATLAB Parallel Server™, or MATLAB Compiler, without needing to rewrite your algorithm.

在R2014b中引入