MapReducer
Define parallel execution environment for mapreduce and tall arrays
Syntax
描述
MapReducer
defines the execution environment forMapReduce
或高阵列。使用MapReducer
function to change the execution environment to use a different cluster or to switch between serial and parallel development.
The default execution environment uses either the local MATLAB®session, or a parallel pool if you have Parallel Computing Toolbox™. If you have Parallel Computing Toolbox installed, when you use thetall
或者MapReduce
functions, MATLAB automatically starts a parallel pool of workers, unless you have changed the default preferences. By default, a parallel pool uses local workers, typically one worker for each core in your machine. If you turn off the自动创建平行池选项,如果要使用并行资源,则必须明确启动池。看指定您的平行首选项.
When working with tall arrays, useMapReducer
to set the execution environment prior to creating the tall array. Tall arrays are bound to the current global execution environment when they are constructed. If you subsequently change the global execution environment, then the tall array is invalid, and you must recreate it.
Note
在MATLAB中,您无需使用MapReducer
becauseMapReduce
算法和高阵列计算仅在本地MATLAB会话中自动运行。如果您还具有并行计算工具箱,则可以使用其他MapReducer
在此页面上列出的配置选项并行运行。如果您的Matlab Compiler™,则可以使用单独MapReducer
configuration options for running in deployed environments.
看:MapReducer
在MATLAB文档中,或MapReducer
在MATLAB编译器文档中。
MapReducer
没有输入参数会创建一个新的MapReducer
execution environment with all the defaults and sets this to be the currentMapReduce
或高阵列执行环境。您可以使用GCMR
获取当前的MAPREDUCER配置。
If you have default preferences (自动创建平行池已启用),您尚未打开平行池,然后
MapReducer
opens a pool using the default cluster profile, setsGCMR
基于此池并返回此MAPREDUCER。如果您打开了平行池,那么
MapReducer
setsGCMR
基于当前池并返回此MAPREDUCER。If you have disabled自动创建平行池, and you have not opened a parallel pool, then
MapReducer
setsGCMR
基于本地MATLAB会话到MAPREDUCER,并MapReducer
返回此MAPREDUCER。
MapReducer(0)
specifies thatMapReduce
或者tall
array calculations run in the MATLAB client session without using any parallel resources.
MapReducer(
指定并行执行的并行池poolobj
)MapReduce
或高阵列。poolobj
is aParallel.Pool目的。The default pool is the current pool that is returned or opened byGCP
.
MapReducer(
指定hadoop®群集以并行执行hadoopCluster
)MapReduce
或高阵列。hadoopCluster
is aParallel.Cluster.hadoop目的。
MapReducer(
sets the global execution environment for先生
)MapReduce
或使用先前创建的MapReducer对象的高阵列,先生
, 如果它是ObjectVisibility
属性是'On'
.
returns a MapReducer object to specify the execution environment. You can define several MapReducer objects, which enables you to swap execution environments by passing one as an input argument to先生
= mapReducer(___)MapReduce
或者MapReducer
.
隐藏MAPREDUCER对象的可见性,先生
= mapReducer(___,'ObjectVisibility','Off')先生
, using any of the previous syntaxes. Use this syntax to create new MapReducer objects without affecting the global execution environment ofMapReduce
.
例子
Develop in Serial and Then Use Local Workers or Cluster
如果您想在串行中开发,不使用本地工人或指定的集群,请输入:
MapReducer(0);
MapReducer
to change the execution environment after creating a tall array, then the tall array is invalid and you must recreate it. To use local workers or your specified cluster again, enter:MapReducer(gcp);
MapReducer
with自动创建平行池关闭
如果您关闭了自动创建平行池选项,如果要使用并行资源,则必须明确启动池。看指定您的平行首选项for details.
The following code shows how you can useMapReducer
without input arguments to set the execution environment to your local MATLAB session and then specify a local parallel pool:
>> MapReducer >> Parpool('local',1);
使用“本地”配置文件...连接到1名工人。
>>收集(最小(rand(rand(1000,1))))
Evaluating tall expression using the Local MATLAB Session: Evaluation completed in 0 sec ans = 5.2238e-04
输入参数
Output Arguments
尖端
One of the benefits of developing your algorithms with tall arrays is that you only need to write the code once. You can develop your code locally, then useMapReducer
to scale up and take advantage of the capabilities offered by Parallel Computing Toolbox,MATLAB Parallel Server™, or MATLAB Compiler, without needing to rewrite your algorithm.