public class Connection extends Object implements Closeable
Connection
object. The JMLC API is designed after JDBC. A DML script is precompiled by calling
the prepareScript(String, String[], String[])
method or the prepareScript(String, Map, String[], String[])
method on the Connection
object, which returns a
PreparedScript
object. Note that this is similar to calling
a prepareStatement
method on a JDBC Connection
object.
Following this, input variable data is passed to the script by calling the
setFrame
, setMatrix
, and setScalar
methods of the PreparedScript
object. The script is executed via PreparedScript
's
executeScript
method,
which returns a ResultVariables
object, which is similar to a JDBC
ResultSet
. Data can be read from a ResultVariables
object by calling
its getFrame
and
getMatrix
methods.
Constructor and Description |
---|
Connection()
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(DMLConfig dmlconfig)
Connection constructor, the starting point for any other JMLC API calls.
|
Connection(DMLConfig dmlconfig,
CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Close connection to SystemDS, which clears the
thread-local DML and compiler configurations.
|
double[][] |
convertToDoubleMatrix(InputStream input,
int rows,
int cols)
Converts an input stream of a string matrix in textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string matrix in csv or textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(String input,
int rows,
int cols)
Converts an input string representation of a matrix in textcell format
into a dense double array.
|
double[][] |
convertToDoubleMatrix(String input,
String meta)
Converts an input string representation of a matrix in csv or textcell format
into a dense double array.
|
FrameBlock |
convertToFrame(InputStream input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a frame block.
|
FrameBlock |
convertToFrame(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a frame in csv or textcell format
into a frame block.
|
FrameBlock |
convertToFrame(InputStream input,
String meta)
Converts an input stream of a string frame in csv or textcell format
into a frame block.
|
FrameBlock |
convertToFrame(String input,
int rows,
int cols)
Converts an input string representation of a frame in textcell format
into a frame block.
|
FrameBlock |
convertToFrame(String input,
String meta)
Converts an input string representation of a frame in csv or textcell format
into a frame block.
|
MatrixBlock |
convertToMatrix(InputStream input,
int rows,
int cols)
Converts an input stream of a string matrix in text format
into a matrix block.
|
MatrixBlock |
convertToMatrix(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string matrix in csv or text format
into a matrix block.
|
MatrixBlock |
convertToMatrix(InputStream input,
String meta)
Converts an input stream of a string matrix in csv or textcell format
into a matrix block.
|
MatrixBlock |
convertToMatrix(String input,
int rows,
int cols)
Converts an input string representation of a matrix in textcell format
into a matrix block.
|
MatrixBlock |
convertToMatrix(String input,
String meta)
Converts an input string representation of a matrix in csv or textcell format
into a matrix block.
|
String[][] |
convertToStringFrame(InputStream input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a dense string array.
|
String[][] |
convertToStringFrame(InputStream input,
int rows,
int cols,
String format)
Converts an input stream of a string frame in csv or text format
into a dense string array.
|
String[][] |
convertToStringFrame(String input,
int rows,
int cols)
Converts an input stream of a string frame in textcell format
into a dense string array.
|
String[][] |
convertToStringFrame(String input,
String meta)
Converts an input string representation of a frame in csv or textcell format
into a dense string array.
|
void |
gatherMemStats(boolean stats)
Sets a boolean flag indicating if memory profiling statistics should be
gathered.
|
PreparedScript |
prepareScript(String script,
Map<String,String> nsscripts,
Map<String,String> args,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
|
PreparedScript |
prepareScript(String script,
Map<String,String> args,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
|
PreparedScript |
prepareScript(String script,
String[] inputs,
String[] outputs)
Prepares (precompiles) a script and registers input and output variables.
|
double[][] |
readDoubleMatrix(String fname)
Reads an input matrix in arbitrary format from HDFS into a dense double array.
|
double[][] |
readDoubleMatrix(String fname,
Types.FileFormat fmt,
long rows,
long cols,
int blen,
long nnz)
Reads an input matrix in arbitrary format from HDFS into a dense double array.
|
String |
readScript(String fname)
Read a DML or PyDML file as a string.
|
String[][] |
readStringFrame(String fname)
Reads an input frame in arbitrary format from HDFS into a dense string array.
|
String[][] |
readStringFrame(String fname,
Types.FileFormat fmt,
long rows,
long cols)
Reads an input frame in arbitrary format from HDFS into a dense string array.
|
FrameBlock |
readTransformMetaDataFromFile(String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromFile(String spec,
String metapath)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromFile(String spec,
String metapath,
String colDelim)
Reads transform meta data from an HDFS file path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String spec,
String metapath)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
FrameBlock |
readTransformMetaDataFromPath(String spec,
String metapath,
String colDelim)
Reads transform meta data from the class path and converts it into an in-memory
FrameBlock object.
|
void |
setLineage(boolean lt)
Sets a boolean flag indicating if lineage trace should be captured
|
void |
setStatistics(boolean stats)
Sets a boolean flag indicating if runtime statistics should be gathered
Same behavior as in "MLContext.setStatistics()"
|
public Connection()
public Connection(CompilerConfig.ConfigType... cconfigs)
cconfigs
- one or many boolean compiler configurations to enable.public Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)
dmlconfig
- a dml configuration.cconfigs
- one or many boolean compiler configurations to enable.public Connection(DMLConfig dmlconfig)
dmlconfig
- a dml configuration.public void setStatistics(boolean stats)
stats
- boolean value with true indicating statistics should be gatheredpublic void setLineage(boolean lt)
lt
- boolean value with true indicating lineage should be capturedpublic void gatherMemStats(boolean stats)
stats
- boolean value with true indicating memory statistics should be gatheredpublic PreparedScript prepareScript(String script, String[] inputs, String[] outputs)
script
- string representing the DML or PyDML scriptinputs
- string array of input variables to registeroutputs
- string array of output variables to registerpublic PreparedScript prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)
script
- string representing the DML or PyDML scriptargs
- map of input parameters ($) and their valuesinputs
- string array of input variables to registeroutputs
- string array of output variables to registerpublic PreparedScript prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)
script
- string representing of the DML or PyDML scriptnsscripts
- map (name, script) of the DML or PyDML namespace scriptsargs
- map of input parameters ($) and their valuesinputs
- string array of input variables to registeroutputs
- string array of output variables to registerpublic void close()
close
in interface Closeable
close
in interface AutoCloseable
public String readScript(String fname) throws IOException
fname
- the filename of the scriptIOException
- if IOException occurspublic double[][] readDoubleMatrix(String fname) throws IOException
fname
- the filename of the input matrixIOException
- if IOException occurspublic double[][] readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz) throws IOException
fname
- the filename of the input matrixfmt
- file formatrows
- number of rows in the matrixcols
- number of columns in the matrixblen
- block lengthnnz
- number of non-zero values, -1 indicates unknownIOException
- if IOException occurspublic double[][] convertToDoubleMatrix(String input, String meta) throws IOException
input
- string matrix in csv or textcell formatmeta
- string representing SystemDS matrix metadata in JSON formatIOException
- if IOException occurspublic double[][] convertToDoubleMatrix(String input, int rows, int cols) throws IOException
input
- string matrix in textcell formatrows
- number of rows in the matrixcols
- number of columns in the matrixIOException
- if IOException occurspublic double[][] convertToDoubleMatrix(InputStream input, int rows, int cols) throws IOException
input
- InputStream to a string matrix in textcell formatrows
- number of rows in the matrixcols
- number of columns in the matrixIOException
- if IOException occurspublic double[][] convertToDoubleMatrix(InputStream input, int rows, int cols, String format) throws IOException
input
- InputStream to a string matrix in csv or textcell formatrows
- number of rows in the matrixcols
- number of columns in the matrixformat
- input format of the given streamIOException
- if IOException occurspublic MatrixBlock convertToMatrix(String input, String meta) throws IOException
input
- string matrix in csv or textcell formatmeta
- string representing SystemDS matrix metadata in JSON formatIOException
- if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, String meta) throws IOException
input
- InputStream to a string matrix in csv or textcell formatmeta
- string representing SystemDS matrix metadata in JSON formatIOException
- if IOException occurspublic MatrixBlock convertToMatrix(String input, int rows, int cols) throws IOException
input
- string matrix in textcell formatrows
- number of rows in the matrixcols
- number of columns in the matrixIOException
- if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, int rows, int cols) throws IOException
input
- InputStream to a string matrix in text formatrows
- number of rows in the matrixcols
- number of columns in the matrixIOException
- if IOException occurspublic MatrixBlock convertToMatrix(InputStream input, int rows, int cols, String format) throws IOException
input
- InputStream to a string matrix in csv or text formatrows
- number of rows in the matrixcols
- number of columns in the matrixformat
- input format of the given streamIOException
- if IOException occurspublic String[][] readStringFrame(String fname) throws IOException
fname
- the filename of the input frameIOException
- if IOException occurspublic String[][] readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols) throws IOException
fname
- the filename of the input framefmt
- file format typerows
- number of rows in the framecols
- number of columns in the frameIOException
- if IOException occurspublic String[][] convertToStringFrame(String input, String meta) throws IOException
input
- string frame in csv or textcell formatmeta
- string representing SystemDS frame metadata in JSON formatIOException
- if IOException occurspublic String[][] convertToStringFrame(String input, int rows, int cols) throws IOException
input
- string frame in textcell formatrows
- number of rows in the framecols
- number of columns in the frameIOException
- if IOException occurspublic String[][] convertToStringFrame(InputStream input, int rows, int cols) throws IOException
input
- InputStream to a string frame in text formatrows
- number of rows in the framecols
- number of columns in the frameIOException
- if IOException occurspublic String[][] convertToStringFrame(InputStream input, int rows, int cols, String format) throws IOException
input
- InputStream to a string frame in csv or text formatrows
- number of rows in the framecols
- number of columns in the frameformat
- input format of the given streamIOException
- if IOException occurspublic FrameBlock convertToFrame(String input, String meta) throws IOException
input
- string frame in csv or textcell formatmeta
- string representing SystemDS frame metadata in JSON formatIOException
- if IOException occurspublic FrameBlock convertToFrame(InputStream input, String meta) throws IOException
input
- InputStream to a string frame in csv or textcell formatmeta
- string representing SystemDS frame metadata in JSON formatIOException
- if IOException occurspublic FrameBlock convertToFrame(String input, int rows, int cols) throws IOException
input
- string frame in textcell formatrows
- number of rows in the framecols
- number of columns in the frameIOException
- if IOException occurspublic FrameBlock convertToFrame(InputStream input, int rows, int cols) throws IOException
input
- InputStream to a string frame in textcell formatrows
- number of rows in the framecols
- number of columns in the frameIOException
- if IOException occurspublic FrameBlock convertToFrame(InputStream input, int rows, int cols, String format) throws IOException
input
- InputStream to a string frame in csv or textcell formatrows
- number of rows in the framecols
- number of columns in the frameformat
- input format of the given streamIOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String metapath) throws IOException
metapath
- hdfs file path to meta data directoryIOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String spec, String metapath) throws IOException
spec
- transform specification as json stringmetapath
- hdfs file path to meta data directoryIOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromFile(String spec, String metapath, String colDelim) throws IOException
spec
- transform specification as json stringmetapath
- hdfs file path to meta data directorycolDelim
- separator for processing column names in the meta data file 'column.names'IOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String metapath) throws IOException
metapath
- resource path to meta data directoryIOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String spec, String metapath) throws IOException
spec
- transform specification as json stringmetapath
- resource path to meta data directoryIOException
- if IOException occurspublic FrameBlock readTransformMetaDataFromPath(String spec, String metapath, String colDelim) throws IOException
spec
- transform specification as json stringmetapath
- resource path to meta data directorycolDelim
- separator for processing column names in the meta data file 'column.names'IOException
- if IOException occursCopyright © 2022 The Apache Software Foundation. All rights reserved.