Tracking-Entitäten manuell erstellen

Importieren Sie die Entitäten zur Herkunftsverfolgung.

import sys
!{sys.executable} -m pip install -q sagemaker

from sagemaker import get_execution_role
from sagemaker.session import Session
from sagemaker.lineage import context, artifact, association, action

import boto3
boto_session = boto3.Session(region_name=region)
sagemaker_client = boto_session.client("sagemaker")

Erstellen Sie die Eingabe- und Ausgabe-Artefakte.

code_location_arn = artifact.Artifact.create(
    artifact_name='source-code-location',
    source_uri='s3://...',
    artifact_type='code-location'
).artifact_arn

# Similar constructs for train_data_location_arn and test_data_location_arn

model_location_arn = artifact.Artifact.create(
    artifact_name='model-location',
    source_uri='s3://...',
    artifact_type='model-location'
).artifact_arn

Schjulen Sie das Modell und holen Sie sich trial_component_arn, der den Trainingsjob represäntiert.

Ordnen Sie die Eingabeartefakte und Ausgabeartefakte dem Trainingsjob zu (Testkomponente).

input_artifacts = [code_location_arn, train_data_location_arn, test_data_location_arn]
for artifact_arn in input_artifacts:
    try:
        association.Association.create(
            source_arn=artifact_arn,
            destination_arn=trial_component_arn,
            association_type='ContributedTo'
        )
    except:
        logging.info('association between {} and {} already exists', artifact_arn, trial_component_arn)

output_artifacts = [model_location_arn]
for artifact_arn in output_artifacts:
    try:
         association.Association.create(
            source_arn=trial_component_arn,
            destination_arn=artifact_arn,
            association_type='Produced'
        )
    except:
        logging.info('association between {} and {} already exists', artifact_arn, trial_component_arn)

Erstellen Sie den Inferenzendpunkt.

predictor = mnist_estimator.deploy(initial_instance_count=1,
                                     instance_type='ml.m4.xlarge')

Erstellen Sie den Endpunktkontext.

from sagemaker.lineage import context

endpoint = sagemaker_client.describe_endpoint(EndpointName=predictor.endpoint_name)
endpoint_arn = endpoint['EndpointArn']

endpoint_context_arn = context.Context.create(
    context_name=predictor.endpoint_name,
    context_type='Endpoint',
    source_uri=endpoint_arn
).context_arn

Ordnen Sie den Trainingsjob (Testkomponente) und den Endpunktkontext zu.

association.Association.create(
    source_arn=trial_component_arn,
    destination_arn=endpoint_context_arn
)

Importieren Sie die Tracking-Entitäten.

import sys
!{sys.executable} -m pip install -q sagemaker

from sagemaker import get_execution_role
from sagemaker.session import Session
from sagemaker.lineage import context, artifact, association, action

import boto3
boto_session = boto3.Session(region_name=region)
sagemaker_client = boto_session.client("sagemaker")

Ruft den Endpunktkontext vom Endpunkt-ARN ab.

endpoint_context_arn = sagemaker_client.list_contexts(
    SourceUri=endpoint_arn)['ContextSummaries'][0]['ContextArn']

Ruft die Testkomponente aus der Zuordnung zwischen der Testkomponente und dem Endpunktkontext ab.

trial_component_arn = sagemaker_client.list_associations(
    DestinationArn=endpoint_context_arn)['AssociationSummaries'][0]['SourceArn']

Ruft das Artefakt zum Standort der Trainingsdaten aus der Assoziation zwischen der Testkomponente und dem Endpunktkontext ab.

train_data_location_artifact_arn = sagemaker_client.list_associations(
    DestinationArn=trial_component_arn, SourceType='Model')['AssociationSummaries'][0]['SourceArn']

Ruft den Standort der Trainingsdaten aus dem Artefakt für den Standort der Trainingsdaten ab.

train_data_location = sagemaker_client.describe_artifact(
    ArtifactArn=train_data_location_artifact_arn)['Source']['SourceUri']
    print(train_data_location)

Antwort:

s3://sagemaker-sample-data-us-east-2/mxnet/mnist/train