evaluations

Creates, updates, deletes, gets or lists an evaluations resource.

Overview

Name	`evaluations`
Type	Resource
Id	`aws.machinelearning.evaluations`

Fields

The following fields are returned by SELECT queries:

get_evaluation
describe_evaluations

Name	Datatype	Description
`compute_time`	`integer (int64)`	Long integer type that is a 64-bit signed number.
`created_at`	`string (date-time)`	A timestamp represented in epoch time.
`created_by_iam_user`	`string`	The AWS user account that invoked the evaluation. The account type can be either an AWS root account or an AWS Identity and Access Management (IAM) user account. (pattern: <code>arn:aws:iam::[0-9]+:((user/.+)\|(root))</code>)
`evaluation_data_source_id`	`string`	The DataSource used for this evaluation. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`evaluation_id`	`string`	The evaluation ID which is same as the EvaluationId in the request. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`finished_at`	`string (date-time)`	A timestamp represented in epoch time.
`input_data_location_s3`	`string`	The location of the data file or directory in Amazon Simple Storage Service (Amazon S3). (pattern: <code>s3://([^/]+)(/.*)?</code>)
`last_updated_at`	`string (date-time)`	A timestamp represented in epoch time.
`log_uri`	`string`	A link to the file that contains logs of the CreateEvaluation operation.
`ml_model_id`	`string`	The ID of the MLModel that was the focus of the evaluation. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`message`	`string`	A description of the most recent details about evaluating the MLModel.
`name`	`string`	A user-supplied name or description of the Evaluation. (pattern: <code>.\S.\|^$</code>)
`performance_metrics`	`object`	Measurements of how well the MLModel performed using observations referenced by the DataSource. One of the following metric is returned based on the type of the MLModel: BinaryAUC: A binary MLModel uses the Area Under the Curve (AUC) technique to measure performance. RegressionRMSE: A regression MLModel uses the Root Mean Square Error (RMSE) technique to measure performance. RMSE measures the difference between predicted and actual values for a single variable. MulticlassAvgFScore: A multiclass MLModel uses the F1 score technique to measure performance. For more information about performance metrics, please see the Amazon Machine Learning Developer Guide.
`started_at`	`string (date-time)`	A timestamp represented in epoch time.
`status`	`string`	The status of the evaluation. This element can have one of the following values: PENDING - Amazon Machine Language (Amazon ML) submitted a request to evaluate an MLModel. INPROGRESS - The evaluation is underway. FAILED - The request to evaluate an MLModel did not run to completion. It is not usable. COMPLETED - The evaluation process completed successfully. DELETED - The Evaluation is marked as deleted. It is not usable. (PENDING, INPROGRESS, FAILED, COMPLETED, DELETED)

Name	Datatype	Description
`compute_time`	`integer (int64)`	Long integer type that is a 64-bit signed number.
`created_at`	`string (date-time)`	A timestamp represented in epoch time.
`created_by_iam_user`	`string`	The AWS user account that invoked the evaluation. The account type can be either an AWS root account or an AWS Identity and Access Management (IAM) user account. (pattern: <code>arn:aws:iam::[0-9]+:((user/.+)\|(root))</code>)
`evaluation_data_source_id`	`string`	The ID of the DataSource that is used to evaluate the MLModel. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`evaluation_id`	`string`	The ID that is assigned to the Evaluation at creation. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`finished_at`	`string (date-time)`	A timestamp represented in epoch time.
`input_data_location_s3`	`string`	The location and name of the data in Amazon Simple Storage Server (Amazon S3) that is used in the evaluation. (pattern: <code>s3://([^/]+)(/.*)?</code>)
`last_updated_at`	`string (date-time)`	A timestamp represented in epoch time.
`ml_model_id`	`string`	The ID of the MLModel that is the focus of the evaluation. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
`message`	`string`	A description of the most recent details about evaluating the MLModel.
`name`	`string`	A user-supplied name or description of the Evaluation. (pattern: <code>.\S.\|^$</code>)
`performance_metrics`	`object`	Measurements of how well the MLModel performed, using observations referenced by the DataSource. One of the following metrics is returned, based on the type of the MLModel: BinaryAUC: A binary MLModel uses the Area Under the Curve (AUC) technique to measure performance. RegressionRMSE: A regression MLModel uses the Root Mean Square Error (RMSE) technique to measure performance. RMSE measures the difference between predicted and actual values for a single variable. MulticlassAvgFScore: A multiclass MLModel uses the F1 score technique to measure performance. For more information about performance metrics, please see the Amazon Machine Learning Developer Guide.
`started_at`	`string (date-time)`	A timestamp represented in epoch time.
`status`	`string`	The status of the evaluation. This element can have one of the following values: PENDING - Amazon Machine Learning (Amazon ML) submitted a request to evaluate an MLModel. INPROGRESS - The evaluation is underway. FAILED - The request to evaluate an MLModel did not run to completion. It is not usable. COMPLETED - The evaluation process completed successfully. DELETED - The Evaluation is marked as deleted. It is not usable. (PENDING, INPROGRESS, FAILED, COMPLETED, DELETED)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Description
`get_evaluation`	`select`	`region`	Returns an Evaluation that includes metadata as well as the current status of the Evaluation.
`describe_evaluations`	`select`	`region`	Returns a list of DescribeEvaluations that match the search criteria in the request.
`create_evaluation`	`insert`	`region`, `EvaluationId`, `MLModelId`, `EvaluationDataSourceId`	Creates a new Evaluation of an MLModel. An MLModel is evaluated on a set of observations associated to a DataSource. Like a DataSource for an MLModel, the DataSource for an Evaluation contains values for the Target Variable. The Evaluation compares the predicted result for each observation to the actual outcome and provides a summary so that you know how effective the MLModel functions on the test data. Evaluation generates a relevant performance metric, such as BinaryAUC, RegressionRMSE or MulticlassAvgFScore based on the corresponding MLModelType: BINARY, REGRESSION or MULTICLASS. CreateEvaluation is an asynchronous operation. In response to CreateEvaluation, Amazon Machine Learning (Amazon ML) immediately returns and sets the evaluation status to PENDING. After the Evaluation is created and ready for use, Amazon ML sets the status to COMPLETED. You can use the GetEvaluation operation to check progress of the evaluation during the creation operation.
`update_evaluation`	`update`	`region`, `EvaluationId`, `EvaluationName`	Updates the EvaluationName of an Evaluation. You can use the GetEvaluation operation to view the contents of the updated data element.
`delete_evaluation`	`delete`	`region`	Assigns the DELETED status to an Evaluation, rendering it unusable. After invoking the DeleteEvaluation operation, you can use the GetEvaluation operation to verify that the status of the Evaluation changed to DELETED. Caution: The results of the DeleteEvaluation operation are irreversible.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`region`	`string`	AWS region (default: us-east-1)

`SELECT` examples

get_evaluation
describe_evaluations

Returns an Evaluation that includes metadata as well as the current status of the Evaluation.

SELECT
compute_time,
created_at,
created_by_iam_user,
evaluation_data_source_id,
evaluation_id,
finished_at,
input_data_location_s3,
last_updated_at,
log_uri,
ml_model_id,
message,
name,
performance_metrics,
started_at,
status
FROM aws.machinelearning.evaluations
WHERE region = '{{ region }}' -- required
;

Returns a list of DescribeEvaluations that match the search criteria in the request.

SELECT
compute_time,
created_at,
created_by_iam_user,
evaluation_data_source_id,
evaluation_id,
finished_at,
input_data_location_s3,
last_updated_at,
ml_model_id,
message,
name,
performance_metrics,
started_at,
status
FROM aws.machinelearning.evaluations
WHERE region = '{{ region }}' -- required
;

`INSERT` examples

create_evaluation
Manifest

Creates a new Evaluation of an MLModel. An MLModel is evaluated on a set of observations associated to a DataSource. Like a DataSource for an MLModel, the DataSource for an Evaluation contains values for the Target Variable. The Evaluation compares the predicted result for each observation to the actual outcome and provides a summary so that you know how effective the MLModel functions on the test data. Evaluation generates a relevant performance metric, such as BinaryAUC, RegressionRMSE or MulticlassAvgFScore based on the corresponding MLModelType: BINARY, REGRESSION or MULTICLASS. CreateEvaluation is an asynchronous operation. In response to CreateEvaluation, Amazon Machine Learning (Amazon ML) immediately returns and sets the evaluation status to PENDING. After the Evaluation is created and ready for use, Amazon ML sets the status to COMPLETED. You can use the GetEvaluation operation to check progress of the evaluation during the creation operation.

INSERT INTO aws.machinelearning.evaluations (
EvaluationId,
EvaluationName,
MLModelId,
EvaluationDataSourceId,
region
)
SELECT 
'{{ EvaluationId }}' /* required */,
'{{ EvaluationName }}',
'{{ MLModelId }}' /* required */,
'{{ EvaluationDataSourceId }}' /* required */,
'{{ region }}'
RETURNING
evaluation_id
;

# Description fields are for documentation purposes
- name: evaluations
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the evaluations resource.
  - name: EvaluationId
    value: "{{ EvaluationId }}"
    description: |
      A user-supplied ID that uniquely identifies the Evaluation.
  - name: EvaluationName
    value: "{{ EvaluationName }}"
    description: |
      A user-supplied name or description of the Evaluation.
  - name: MLModelId
    value: "{{ MLModelId }}"
    description: |
      The ID of the MLModel to evaluate. The schema used in creating the MLModel must match the schema of the DataSource used in the Evaluation.
  - name: EvaluationDataSourceId
    value: "{{ EvaluationDataSourceId }}"
    description: |
      The ID of the DataSource for the evaluation. The schema of the DataSource must match the schema used to create the MLModel.

`UPDATE` examples

update_evaluation

Updates the EvaluationName of an Evaluation. You can use the GetEvaluation operation to view the contents of the updated data element.

UPDATE aws.machinelearning.evaluations
SET 
EvaluationId = '{{ EvaluationId }}',
EvaluationName = '{{ EvaluationName }}'
WHERE 
region = '{{ region }}' --required
AND EvaluationId = '{{ EvaluationId }}' --required
AND EvaluationName = '{{ EvaluationName }}' --required
RETURNING
evaluation_id;

`DELETE` examples

delete_evaluation

Assigns the DELETED status to an Evaluation, rendering it unusable. After invoking the DeleteEvaluation operation, you can use the GetEvaluation operation to verify that the status of the Evaluation changed to DELETED. Caution: The results of the DeleteEvaluation operation are irreversible.

DELETE FROM aws.machinelearning.evaluations
WHERE region = '{{ region }}' --required
;

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​

UPDATE examples​

DELETE examples​