ALS

java.lang.Object
- org.apache.spark.ml.PipelineStage
- - org.apache.spark.ml.Estimator<ALSModel>
  - - org.apache.spark.ml.recommendation.ALS

All Implemented Interfaces:

java.io.Serializable, Params, Identifiable
```
public class ALS
extends Estimator<ALSModel>
```
:: Experimental :: Alternating Least Squares (ALS) matrix factorization.
ALS attempts to estimate the ratings matrix R as the product of two lower-rank matrices, X and Y, i.e. X * Yt = R. Typically these approximations are called 'factor' matrices. The general approach is iterative. During each iteration, one of the factor matrices is held constant, while the other is solved for using least squares. The newly-solved factor matrix is then held constant while solving for the other factor matrix.
This is a blocked implementation of the ALS factorization algorithm that groups the two sets of factors (referred to as "users" and "products") into blocks and reduces communication by only sending one copy of each user vector to each product block on each iteration, and only for the product blocks that need that user's feature vector. This is achieved by pre-computing some information about the ratings matrix to determine the "out-links" of each user (which blocks of products it will contribute to) and "in-link" information for each product (which of the feature vectors it receives from each user block it will depend on). This allows us to send only an array of feature vectors between each user block and product block, and have the product block find the users' ratings and update the products based on these messages.
For implicit preference data, the algorithm used is based on "Collaborative Filtering for Implicit Feedback Datasets", available at http://dx.doi.org/10.1109/ICDM.2008.22, adapted for the blocked approach used here.
Essentially instead of finding the low-rank approximations to the rating matrix R, this finds the approximations for a preference matrix P where the elements of P are 1 if r > 0 and 0 if r <= 0. The ratings then act as 'confidence' values related to strength of indicated user preferences rather than explicit ratings given to items.

See Also:
Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ALS.InBlock$`
`static class`	`ALS.Rating<ID>` :: DeveloperApi :: Rating class for better code readability.
`static class`	`ALS.Rating$`
`static class`	`ALS.RatingBlock$`

Constructor Summary

Constructors
Constructor and Description

ALS()

ALS(java.lang.String uid)

Constructors
Constructor and Description
`ALS()`
`ALS(java.lang.String uid)`

Method Summary

Methods
Modifier and Type	Method and Description
`protected static <T> T`	`$(Param<T> param)`
`static DoubleParam`	`alpha()`
`DoubleParam`	`alpha()` Param for the alpha parameter in the implicit preference formulation (>= 0).
`static IntParam`	`checkpointInterval()`
`static Params`	`clear(Param<?> param)`
`ALS`	`copy(ParamMap extra)` Creates a copy of this instance with the same UID and some extra params.
`protected static <T extends Params> T`	`copyValues(T to, ParamMap extra)`
`protected static <T extends Params> ParamMap`	`copyValues$default$2()`
`protected static <T extends Params> T`	`defaultCopy(ParamMap extra)`
`static java.lang.String`	`explainParam(Param<?> param)`
`static java.lang.String`	`explainParams()`
`static ParamMap`	`extractParamMap()`
`static ParamMap`	`extractParamMap(ParamMap extra)`
`static Param<java.lang.String>`	`finalStorageLevel()`
`Param<java.lang.String>`	`finalStorageLevel()` Param for StorageLevel for ALS model factors.
`ALSModel`	`fit(Dataset<?> dataset)` Fits a model to the input data.
`static <T> scala.Option<T>`	`get(Param<T> param)`
`static double`	`getAlpha()`
`double`	`getAlpha()`
`static int`	`getCheckpointInterval()`
`static <T> scala.Option<T>`	`getDefault(Param<T> param)`
`static java.lang.String`	`getFinalStorageLevel()`
`java.lang.String`	`getFinalStorageLevel()`
`static boolean`	`getImplicitPrefs()`
`boolean`	`getImplicitPrefs()`
`static java.lang.String`	`getIntermediateStorageLevel()`
`java.lang.String`	`getIntermediateStorageLevel()`
`static java.lang.String`	`getItemCol()`
`java.lang.String`	`getItemCol()`
`static int`	`getMaxIter()`
`static boolean`	`getNonnegative()`
`boolean`	`getNonnegative()`
`static int`	`getNumItemBlocks()`
`int`	`getNumItemBlocks()`
`static int`	`getNumUserBlocks()`
`int`	`getNumUserBlocks()`
`static <T> T`	`getOrDefault(Param<T> param)`
`static Param<java.lang.Object>`	`getParam(java.lang.String paramName)`
`static java.lang.String`	`getPredictionCol()`
`static int`	`getRank()`
`int`	`getRank()`
`static java.lang.String`	`getRatingCol()`
`java.lang.String`	`getRatingCol()`
`static double`	`getRegParam()`
`static long`	`getSeed()`
`static java.lang.String`	`getUserCol()`
`java.lang.String`	`getUserCol()`
`static <T> boolean`	`hasDefault(Param<T> param)`
`static boolean`	`hasParam(java.lang.String paramName)`
`static BooleanParam`	`implicitPrefs()`
`BooleanParam`	`implicitPrefs()` Param to decide whether to use implicit preference.
`protected static void`	`initializeLogIfNecessary(boolean isInterpreter)`
`static Param<java.lang.String>`	`intermediateStorageLevel()`
`Param<java.lang.String>`	`intermediateStorageLevel()` Param for StorageLevel for intermediate datasets.
`static boolean`	`isDefined(Param<?> param)`
`static boolean`	`isSet(Param<?> param)`
`protected static boolean`	`isTraceEnabled()`
`static Param<java.lang.String>`	`itemCol()`
`Param<java.lang.String>`	`itemCol()` Param for the column name for item ids.
`static ALS`	`load(java.lang.String path)`
`protected static org.slf4j.Logger`	`log()`
`protected static void`	`logDebug(scala.Function0<java.lang.String> msg)`
`protected static void`	`logDebug(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)`
`protected static void`	`logError(scala.Function0<java.lang.String> msg)`
`protected static void`	`logError(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)`
`protected static void`	`logInfo(scala.Function0<java.lang.String> msg)`
`protected static void`	`logInfo(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)`
`protected static java.lang.String`	`logName()`
`protected static void`	`logTrace(scala.Function0<java.lang.String> msg)`
`protected static void`	`logTrace(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)`
`protected static void`	`logWarning(scala.Function0<java.lang.String> msg)`
`protected static void`	`logWarning(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)`
`static IntParam`	`maxIter()`
`static BooleanParam`	`nonnegative()`
`BooleanParam`	`nonnegative()` Param for whether to apply nonnegativity constraints.
`static IntParam`	`numItemBlocks()`
`IntParam`	`numItemBlocks()` Param for number of item blocks (>= 1).
`static IntParam`	`numUserBlocks()`
`IntParam`	`numUserBlocks()` Param for number of user blocks (>= 1).
`static Param<?>[]`	`params()`
`static Param<java.lang.String>`	`predictionCol()`
`static IntParam`	`rank()`
`IntParam`	`rank()` Param for rank of the matrix factorization (>= 1).
`static Param<java.lang.String>`	`ratingCol()`
`Param<java.lang.String>`	`ratingCol()` Param for the column name for ratings.
`static DoubleParam`	`regParam()`
`static void`	`save(java.lang.String path)`
`static LongParam`	`seed()`
`static <T> Params`	`set(Param<T> param, T value)`
`protected static Params`	`set(ParamPair<?> paramPair)`
`protected static Params`	`set(java.lang.String param, java.lang.Object value)`
`ALS`	`setAlpha(double value)`
`ALS`	`setCheckpointInterval(int value)`
`protected static <T> Params`	`setDefault(Param<T> param, T value)`
`protected static Params`	`setDefault(scala.collection.Seq<ParamPair<?>> paramPairs)`
`ALS`	`setFinalStorageLevel(java.lang.String value)`
`ALS`	`setImplicitPrefs(boolean value)`
`ALS`	`setIntermediateStorageLevel(java.lang.String value)`
`ALS`	`setItemCol(java.lang.String value)`
`ALS`	`setMaxIter(int value)`
`ALS`	`setNonnegative(boolean value)`
`ALS`	`setNumBlocks(int value)` Sets both numUserBlocks and numItemBlocks to the specific value.
`ALS`	`setNumItemBlocks(int value)`
`ALS`	`setNumUserBlocks(int value)`
`ALS`	`setPredictionCol(java.lang.String value)`
`ALS`	`setRank(int value)`
`ALS`	`setRatingCol(java.lang.String value)`
`ALS`	`setRegParam(double value)`
`ALS`	`setSeed(long value)`
`ALS`	`setUserCol(java.lang.String value)`
`static java.lang.String`	`toString()`
`static <ID> scala.Tuple2<RDD<scala.Tuple2<ID,float[]>>,RDD<scala.Tuple2<ID,float[]>>>`	`train(RDD<ALS.Rating<ID>> ratings, int rank, int numUserBlocks, int numItemBlocks, int maxIter, double regParam, boolean implicitPrefs, double alpha, boolean nonnegative, StorageLevel intermediateRDDStorageLevel, StorageLevel finalRDDStorageLevel, int checkpointInterval, long seed, scala.reflect.ClassTag<ID> evidence$1, scala.math.Ordering<ID> ord)` :: DeveloperApi :: Implementation of the ALS algorithm.
`StructType`	`transformSchema(StructType schema)` :: DeveloperApi ::
`java.lang.String`	`uid()` An immutable unique ID for the object and its derivatives.
`static Param<java.lang.String>`	`userCol()`
`Param<java.lang.String>`	`userCol()` Param for the column name for user ids.
`protected static StructType`	`validateAndTransformSchema(StructType schema)`
`StructType`	`validateAndTransformSchema(StructType schema)` Validates and transforms the input schema.
`static void`	`validateParams()`
`static MLWriter`	`write()`

Methods inherited from class org.apache.spark.ml.Estimator
fit, fit, fit, fit

Methods inherited from class org.apache.spark.ml.PipelineStage
transformSchema

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.Params
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn, validateParams

Methods inherited from interface org.apache.spark.ml.util.Identifiable
toString

Constructor Detail

ALS
```
public ALS(java.lang.String uid)
```

ALS
```
public ALS()
```

Method Detail

load

public static ALS load(java.lang.String path)

train

public static <ID> scala.Tuple2<RDD<scala.Tuple2<ID,float[]>>,RDD<scala.Tuple2<ID,float[]>>> train(RDD<ALS.Rating<ID>> ratings,
                                                                                   int rank,
                                                                                   int numUserBlocks,
                                                                                   int numItemBlocks,
                                                                                   int maxIter,
                                                                                   double regParam,
                                                                                   boolean implicitPrefs,
                                                                                   double alpha,
                                                                                   boolean nonnegative,
                                                                                   StorageLevel intermediateRDDStorageLevel,
                                                                                   StorageLevel finalRDDStorageLevel,
                                                                                   int checkpointInterval,
                                                                                   long seed,
                                                                                   scala.reflect.ClassTag<ID> evidence$1,
                                                                                   scala.math.Ordering<ID> ord)

:: DeveloperApi :: Implementation of the ALS algorithm.

Parameters:: ratings - (undocumented); rank - (undocumented); numUserBlocks - (undocumented); numItemBlocks - (undocumented); maxIter - (undocumented); regParam - (undocumented); implicitPrefs - (undocumented); alpha - (undocumented); nonnegative - (undocumented); intermediateRDDStorageLevel - (undocumented); finalRDDStorageLevel - (undocumented); checkpointInterval - (undocumented); seed - (undocumented); evidence$1 - (undocumented); ord - (undocumented)
Returns:: (undocumented)

toString

public static java.lang.String toString()

params
```
public static Param<?>[] params()
```

validateParams
```
public static void validateParams()
```

explainParam

public static java.lang.String explainParam(Param<?> param)

explainParams

public static java.lang.String explainParams()

isSet

public static final boolean isSet(Param<?> param)

isDefined

public static final boolean isDefined(Param<?> param)

hasParam

public static boolean hasParam(java.lang.String paramName)

getParam

public static Param<java.lang.Object> getParam(java.lang.String paramName)

set

public static final <T> Params set(Param<T> param,
             T value)

set

protected static final Params set(java.lang.String param,
         java.lang.Object value)

set

protected static final Params set(ParamPair<?> paramPair)

get

public static final <T> scala.Option<T> get(Param<T> param)

clear

public static final Params clear(Param<?> param)

getOrDefault

public static final <T> T getOrDefault(Param<T> param)

$

protected static final <T> T $(Param<T> param)

setDefault

protected static final <T> Params setDefault(Param<T> param,
                    T value)

setDefault

protected static final Params setDefault(scala.collection.Seq<ParamPair<?>> paramPairs)

getDefault

public static final <T> scala.Option<T> getDefault(Param<T> param)

hasDefault

public static final <T> boolean hasDefault(Param<T> param)

defaultCopy

protected static final <T extends Params> T defaultCopy(ParamMap extra)

extractParamMap

public static final ParamMap extractParamMap(ParamMap extra)

extractParamMap

public static final ParamMap extractParamMap()

copyValues

protected static <T extends Params> T copyValues(T to,
                              ParamMap extra)

copyValues$default$2

protected static <T extends Params> ParamMap copyValues$default$2()

logName

protected static java.lang.String logName()

log

protected static org.slf4j.Logger log()

logInfo

protected static void logInfo(scala.Function0<java.lang.String> msg)

logDebug

protected static void logDebug(scala.Function0<java.lang.String> msg)

logTrace

protected static void logTrace(scala.Function0<java.lang.String> msg)

logWarning

protected static void logWarning(scala.Function0<java.lang.String> msg)

logError

protected static void logError(scala.Function0<java.lang.String> msg)

logInfo

protected static void logInfo(scala.Function0<java.lang.String> msg,
           java.lang.Throwable throwable)

logDebug

protected static void logDebug(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

logTrace

protected static void logTrace(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

logWarning

protected static void logWarning(scala.Function0<java.lang.String> msg,
              java.lang.Throwable throwable)

logError

protected static void logError(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

isTraceEnabled

protected static boolean isTraceEnabled()

initializeLogIfNecessary

protected static void initializeLogIfNecessary(boolean isInterpreter)

predictionCol

public static final Param<java.lang.String> predictionCol()

getPredictionCol

public static final java.lang.String getPredictionCol()

userCol

public static Param<java.lang.String> userCol()

getUserCol

public static java.lang.String getUserCol()

itemCol

public static Param<java.lang.String> itemCol()

getItemCol

public static java.lang.String getItemCol()

maxIter

public static final IntParam maxIter()

getMaxIter
```
public static final int getMaxIter()
```

regParam

public static final DoubleParam regParam()

getRegParam

public static final double getRegParam()

checkpointInterval

public static final IntParam checkpointInterval()

getCheckpointInterval

public static final int getCheckpointInterval()

seed
```
public static final LongParam seed()
```

getSeed
```
public static final long getSeed()
```

rank
```
public static IntParam rank()
```

getRank
```
public static int getRank()
```

numUserBlocks

public static IntParam numUserBlocks()

getNumUserBlocks
```
public static int getNumUserBlocks()
```

numItemBlocks

public static IntParam numItemBlocks()

getNumItemBlocks
```
public static int getNumItemBlocks()
```

implicitPrefs

public static BooleanParam implicitPrefs()

getImplicitPrefs

public static boolean getImplicitPrefs()

alpha
```
public static DoubleParam alpha()
```

getAlpha
```
public static double getAlpha()
```

ratingCol

public static Param<java.lang.String> ratingCol()

getRatingCol

public static java.lang.String getRatingCol()

nonnegative

public static BooleanParam nonnegative()

getNonnegative

public static boolean getNonnegative()

intermediateStorageLevel

public static Param<java.lang.String> intermediateStorageLevel()

getIntermediateStorageLevel

public static java.lang.String getIntermediateStorageLevel()

finalStorageLevel

public static Param<java.lang.String> finalStorageLevel()

getFinalStorageLevel

public static java.lang.String getFinalStorageLevel()

validateAndTransformSchema

protected static StructType validateAndTransformSchema(StructType schema)

save

public static void save(java.lang.String path)
                 throws java.io.IOException

Throws:: java.io.IOException

write
```
public static MLWriter write()
```

uid
```
public java.lang.String uid()
```
Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:
(undocumented)

setRank
```
public ALS setRank(int value)
```

setNumUserBlocks

public ALS setNumUserBlocks(int value)

setNumItemBlocks

public ALS setNumItemBlocks(int value)

setImplicitPrefs

public ALS setImplicitPrefs(boolean value)

setAlpha
```
public ALS setAlpha(double value)
```

setUserCol

public ALS setUserCol(java.lang.String value)

setItemCol

public ALS setItemCol(java.lang.String value)

setRatingCol

public ALS setRatingCol(java.lang.String value)

setPredictionCol

public ALS setPredictionCol(java.lang.String value)

setMaxIter
```
public ALS setMaxIter(int value)
```

setRegParam
```
public ALS setRegParam(double value)
```

setNonnegative

public ALS setNonnegative(boolean value)

setCheckpointInterval

public ALS setCheckpointInterval(int value)

setSeed
```
public ALS setSeed(long value)
```

setIntermediateStorageLevel

public ALS setIntermediateStorageLevel(java.lang.String value)

setFinalStorageLevel

public ALS setFinalStorageLevel(java.lang.String value)

setNumBlocks
```
public ALS setNumBlocks(int value)
```
Sets both numUserBlocks and numItemBlocks to the specific value.

Parameters:
value - (undocumented)

Returns:
(undocumented)

fit
```
public ALSModel fit(Dataset<?> dataset)
```
Description copied from class: Estimator

Fits a model to the input data.

Specified by:

fit in class Estimator<ALSModel>

Parameters:
dataset - (undocumented)

Returns:
(undocumented)

transformSchema
```
public StructType transformSchema(StructType schema)
```
Description copied from class: PipelineStage

:: DeveloperApi ::
Derives the output schema from the input schema.

Specified by:

transformSchema in class PipelineStage

Parameters:
schema - (undocumented)

Returns:
(undocumented)

copy
```
public ALS copy(ParamMap extra)
```
Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly.

Specified by:

copy in interface Params

Specified by:

copy in class Estimator<ALSModel>

Parameters:
extra - (undocumented)

Returns:
(undocumented)
See Also:
defaultCopy()

rank
```
public IntParam rank()
```
Param for rank of the matrix factorization (>= 1). Default: 10

Returns:
(undocumented)

getRank
```
public int getRank()
```

numUserBlocks
```
public IntParam numUserBlocks()
```
Param for number of user blocks (>= 1). Default: 10

Returns:
(undocumented)

getNumUserBlocks
```
public int getNumUserBlocks()
```

numItemBlocks
```
public IntParam numItemBlocks()
```
Param for number of item blocks (>= 1). Default: 10

Returns:
(undocumented)

getNumItemBlocks
```
public int getNumItemBlocks()
```

implicitPrefs
```
public BooleanParam implicitPrefs()
```
Param to decide whether to use implicit preference. Default: false

Returns:
(undocumented)

getImplicitPrefs
```
public boolean getImplicitPrefs()
```

alpha
```
public DoubleParam alpha()
```
Param for the alpha parameter in the implicit preference formulation (>= 0). Default: 1.0

Returns:
(undocumented)

getAlpha
```
public double getAlpha()
```

ratingCol
```
public Param<java.lang.String> ratingCol()
```
Param for the column name for ratings. Default: "rating"

Returns:
(undocumented)

getRatingCol

public java.lang.String getRatingCol()

nonnegative
```
public BooleanParam nonnegative()
```
Param for whether to apply nonnegativity constraints. Default: false

Returns:
(undocumented)

getNonnegative
```
public boolean getNonnegative()
```

intermediateStorageLevel
```
public Param<java.lang.String> intermediateStorageLevel()
```
Param for StorageLevel for intermediate datasets. Pass in a string representation of StorageLevel. Cannot be "NONE". Default: "MEMORY_AND_DISK".

Returns:
(undocumented)

getIntermediateStorageLevel

public java.lang.String getIntermediateStorageLevel()

finalStorageLevel
```
public Param<java.lang.String> finalStorageLevel()
```
Param for StorageLevel for ALS model factors. Pass in a string representation of StorageLevel. Default: "MEMORY_AND_DISK".

Returns:
(undocumented)

getFinalStorageLevel

public java.lang.String getFinalStorageLevel()

validateAndTransformSchema
```
public StructType validateAndTransformSchema(StructType schema)
```
Validates and transforms the input schema.

Parameters:
schema - input schema

Returns:
output schema

userCol
```
public Param<java.lang.String> userCol()
```
Param for the column name for user ids. Default: "user"

Returns:
(undocumented)

getUserCol
```
public java.lang.String getUserCol()
```

itemCol
```
public Param<java.lang.String> itemCol()
```
Param for the column name for item ids. Default: "item"

Returns:
(undocumented)

getItemCol
```
public java.lang.String getItemCol()
```

Class ALS

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.spark.ml.Estimator

Methods inherited from class org.apache.spark.ml.PipelineStage

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.spark.ml.param.Params

Methods inherited from interface org.apache.spark.ml.util.Identifiable

Constructor Detail

ALS

ALS

Method Detail

load

train

toString

params

validateParams

explainParam

explainParams

isSet

isDefined

hasParam

getParam

set

set

set

get

clear

getOrDefault

$

setDefault

setDefault

getDefault

hasDefault

defaultCopy

extractParamMap

extractParamMap

copyValues

copyValues$default$2

logName

log

logInfo

logDebug

logTrace

logWarning

logError

logInfo

logDebug

logTrace

logWarning

logError

isTraceEnabled

initializeLogIfNecessary

predictionCol

getPredictionCol

userCol

getUserCol

itemCol

getItemCol

maxIter

getMaxIter

regParam

getRegParam

checkpointInterval

getCheckpointInterval

seed

getSeed

rank

getRank

numUserBlocks

getNumUserBlocks

numItemBlocks

getNumItemBlocks

implicitPrefs

getImplicitPrefs

alpha

getAlpha

ratingCol

getRatingCol