# API Reference ## Constructs ### AccessLogsBucket Amazon S3 Bucket configured with best-practices and smart defaults for storing S3 access logs. Default bucket name is `accesslogs---` > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/access-logs-bucket](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/access-logs-bucket) *Example* ```typescript const bucket = new dsf.storage.AccessLogsBucket(this, 'AccessLogsBucket') ``` #### Initializers ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' new storage.AccessLogsBucket(scope: Construct, id: string, props?: BucketProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | aws-cdk-lib.aws_s3.BucketProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Optional - *Type:* aws-cdk-lib.aws_s3.BucketProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | applyRemovalPolicy | Apply the given removal policy to this resource. | | addEventNotification | Adds a bucket notification event destination. | | addObjectCreatedNotification | Subscribes a destination to receive notifications when an object is created in the bucket. | | addObjectRemovedNotification | Subscribes a destination to receive notifications when an object is removed from the bucket. | | addReplicationPolicy | Function to add required permissions to the destination bucket for cross account replication. | | addToResourcePolicy | Adds a statement to the resource policy for a principal (i.e. account/role/service) to perform actions on this bucket and/or its contents. Use `bucketArn` and `arnForObjects(keys)` to obtain ARNs for this bucket or objects. | | arnForObjects | Returns an ARN that represents all objects within the bucket that match the key pattern specified. | | enableEventBridgeNotification | Enables event bridge notification, causing all events below to be sent to EventBridge:. | | grantDelete | Grants s3:DeleteObject* permission to an IAM principal for objects in this bucket. | | grantPublicAccess | Allows unrestricted access to objects from this bucket. | | grantPut | Grants s3:PutObject* and s3:Abort* permissions for this bucket to an IAM principal. | | grantPutAcl | Grant the given IAM identity permissions to modify the ACLs of objects in the given Bucket. | | grantRead | Grant read permissions for this bucket and it's contents to an IAM principal (Role/Group/User). | | grantReadWrite | Grants read/write permissions for this bucket and it's contents to an IAM principal (Role/Group/User). | | grantWrite | Grant write permissions to this bucket to an IAM principal. | | onCloudTrailEvent | Define a CloudWatch event that triggers when something happens to this repository. | | onCloudTrailPutObject | Defines an AWS CloudWatch event that triggers when an object is uploaded to the specified paths (keys) in this bucket using the PutObject API call. | | onCloudTrailWriteObject | Defines an AWS CloudWatch event that triggers when an object at the specified paths (keys) in this bucket are written to. | | s3UrlForObject | The S3 URL of an S3 object. For example:. | | transferAccelerationUrlForObject | The https Transfer Acceleration URL of an S3 object. | | urlForObject | The https URL of an S3 object. Specify `regional: false` at the options for non-regional URLs. For example:. | | virtualHostedUrlForObject | The virtual hosted-style URL of an S3 object. Specify `regional: false` at the options for non-regional URL. For example:. | | addCorsRule | Adds a cross-origin access configuration for objects in an Amazon S3 bucket. | | addInventory | Add an inventory configuration. | | addLifecycleRule | Add a lifecycle rule to the bucket. | | addMetric | Adds a metrics configuration for the CloudWatch request metrics from the bucket. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `applyRemovalPolicy` ```typescript public applyRemovalPolicy(policy: RemovalPolicy): void ``` Apply the given removal policy to this resource. The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced. The resource can be deleted (`RemovalPolicy.DESTROY`), or left in your AWS account for data recovery and cleanup later (`RemovalPolicy.RETAIN`). ###### `policy`^Required - *Type:* aws-cdk-lib.RemovalPolicy --- ##### `addEventNotification` ```typescript public addEventNotification(event: EventType, dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Adds a bucket notification event destination. > [https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html](https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html) *Example* ```typescript declare const myLambda: lambda.Function; const bucket = new s3.Bucket(this, 'MyBucket'); bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(myLambda), {prefix: 'home/myusername/*'}); ``` ###### `event`^Required - *Type:* aws-cdk-lib.aws_s3.EventType The event to trigger the notification. --- ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (Lambda, SNS Topic or SQS Queue). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] S3 object key filter rules to determine which objects trigger this event. Each filter must include a `prefix` and/or `suffix` that will be matched against the s3 object key. Refer to the S3 Developer Guide for details about allowed filter rules. --- ##### `addObjectCreatedNotification` ```typescript public addObjectCreatedNotification(dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Subscribes a destination to receive notifications when an object is created in the bucket. This is identical to calling `onEvent(EventType.OBJECT_CREATED)`. ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (see onEvent). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] Filters (see onEvent). --- ##### `addObjectRemovedNotification` ```typescript public addObjectRemovedNotification(dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Subscribes a destination to receive notifications when an object is removed from the bucket. This is identical to calling `onEvent(EventType.OBJECT_REMOVED)`. ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (see onEvent). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] Filters (see onEvent). --- ##### `addReplicationPolicy` ```typescript public addReplicationPolicy(roleArn: string, accessControlTransition?: boolean, account?: string): void ``` Function to add required permissions to the destination bucket for cross account replication. These permissions will be added as a resource based policy on the bucket > [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-accesscontroltranslation.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-accesscontroltranslation.html) ###### `roleArn`^Required - *Type:* string --- ###### `accessControlTransition`^Optional - *Type:* boolean --- ###### `account`^Optional - *Type:* string --- ##### `addToResourcePolicy` ```typescript public addToResourcePolicy(permission: PolicyStatement): AddToResourcePolicyResult ``` Adds a statement to the resource policy for a principal (i.e. account/role/service) to perform actions on this bucket and/or its contents. Use `bucketArn` and `arnForObjects(keys)` to obtain ARNs for this bucket or objects. Note that the policy statement may or may not be added to the policy. For example, when an `IBucket` is created from an existing bucket, it's not possible to tell whether the bucket already has a policy attached, let alone to re-use that policy to add more statements to it. So it's safest to do nothing in these cases. ###### `permission`^Required - *Type:* aws-cdk-lib.aws_iam.PolicyStatement the policy statement to be added to the bucket's policy. --- ##### `arnForObjects` ```typescript public arnForObjects(keyPattern: string): string ``` Returns an ARN that represents all objects within the bucket that match the key pattern specified. To represent all keys, specify ``"*"``. If you need to specify a keyPattern with multiple components, concatenate them into a single string, e.g.: arnForObjects(`home/${team}/${user}/*`) ###### `keyPattern`^Required - *Type:* string --- ##### `enableEventBridgeNotification` ```typescript public enableEventBridgeNotification(): void ``` Enables event bridge notification, causing all events below to be sent to EventBridge:. Object Deleted (DeleteObject) - Object Deleted (Lifecycle expiration) - Object Restore Initiated - Object Restore Completed - Object Restore Expired - Object Storage Class Changed - Object Access Tier Changed - Object ACL Updated - Object Tags Added - Object Tags Deleted ##### `grantDelete` ```typescript public grantDelete(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants s3:DeleteObject* permission to an IAM principal for objects in this bucket. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantPublicAccess` ```typescript public grantPublicAccess(allowedActions: ...string[], keyPrefix?: string): Grant ``` Allows unrestricted access to objects from this bucket. IMPORTANT: This permission allows anyone to perform actions on S3 objects in this bucket, which is useful for when you configure your bucket as a website and want everyone to be able to read objects in the bucket without needing to authenticate. Without arguments, this method will grant read ("s3:GetObject") access to all objects ("*") in the bucket. The method returns the `iam.Grant` object, which can then be modified as needed. For example, you can add a condition that will restrict access only to an IPv4 range like this: const grant = bucket.grantPublicAccess(); grant.resourceStatement!.addCondition(‘IpAddress’, { “aws:SourceIp”: “54.240.143.0/24” }); Note that if this `IBucket` refers to an existing bucket, possibly not managed by CloudFormation, this method will have no effect, since it's impossible to modify the policy of an existing bucket. ###### `allowedActions`^Required - *Type:* ...string[] the set of S3 actions to allow. Default is "s3:GetObject". --- ###### `keyPrefix`^Optional - *Type:* string the prefix of S3 object keys (e.g. `home/*`). Default is "*". --- ##### `grantPut` ```typescript public grantPut(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants s3:PutObject* and s3:Abort* permissions for this bucket to an IAM principal. If encryption is used, permission to use the key to encrypt the contents of written files will also be granted to the same principal. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantPutAcl` ```typescript public grantPutAcl(identity: IGrantable, objectsKeyPattern?: string): Grant ``` Grant the given IAM identity permissions to modify the ACLs of objects in the given Bucket. If your application has the '@aws-cdk/aws-s3:grantWriteWithoutAcl' feature flag set, calling `grantWrite` or `grantReadWrite` no longer grants permissions to modify the ACLs of the objects; in this case, if you need to modify object ACLs, call this method explicitly. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* string --- ##### `grantRead` ```typescript public grantRead(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grant read permissions for this bucket and it's contents to an IAM principal (Role/Group/User). If encryption is used, permission to use the key to decrypt the contents of the bucket will also be granted to the same principal. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantReadWrite` ```typescript public grantReadWrite(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants read/write permissions for this bucket and it's contents to an IAM principal (Role/Group/User). If an encryption key is used, permission to use the key for encrypt/decrypt will also be granted. Before CDK version 1.85.0, this method granted the `s3:PutObject*` permission that included `s3:PutObjectAcl`, which could be used to grant read/write object access to IAM principals in other accounts. If you want to get rid of that behavior, update your CDK version to 1.85.0 or later, and make sure the `@aws-cdk/aws-s3:grantWriteWithoutAcl` feature flag is set to `true` in the `context` key of your cdk.json file. If you've already updated, but still need the principal to have permissions to modify the ACLs, use the `grantPutAcl` method. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* any --- ##### `grantWrite` ```typescript public grantWrite(identity: IGrantable, objectsKeyPattern?: any, allowedActionPatterns?: string[]): Grant ``` Grant write permissions to this bucket to an IAM principal. If encryption is used, permission to use the key to encrypt the contents of written files will also be granted to the same principal. Before CDK version 1.85.0, this method granted the `s3:PutObject*` permission that included `s3:PutObjectAcl`, which could be used to grant read/write object access to IAM principals in other accounts. If you want to get rid of that behavior, update your CDK version to 1.85.0 or later, and make sure the `@aws-cdk/aws-s3:grantWriteWithoutAcl` feature flag is set to `true` in the `context` key of your cdk.json file. If you've already updated, but still need the principal to have permissions to modify the ACLs, use the `grantPutAcl` method. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* any --- ###### `allowedActionPatterns`^Optional - *Type:* string[] --- ##### `onCloudTrailEvent` ```typescript public onCloudTrailEvent(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Define a CloudWatch event that triggers when something happens to this repository. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `onCloudTrailPutObject` ```typescript public onCloudTrailPutObject(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Defines an AWS CloudWatch event that triggers when an object is uploaded to the specified paths (keys) in this bucket using the PutObject API call. Note that some tools like `aws s3 cp` will automatically use either PutObject or the multipart upload API depending on the file size, so using `onCloudTrailWriteObject` may be preferable. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `onCloudTrailWriteObject` ```typescript public onCloudTrailWriteObject(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Defines an AWS CloudWatch event that triggers when an object at the specified paths (keys) in this bucket are written to. This includes the events PutObject, CopyObject, and CompleteMultipartUpload. Note that some tools like `aws s3 cp` will automatically use either PutObject or the multipart upload API depending on the file size, so using this method may be preferable to `onCloudTrailPutObject`. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `s3UrlForObject` ```typescript public s3UrlForObject(key?: string): string ``` The S3 URL of an S3 object. For example:. `s3://onlybucket` - `s3://bucket/key` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the S3 URL of the bucket is returned. --- ##### `transferAccelerationUrlForObject` ```typescript public transferAccelerationUrlForObject(key?: string, options?: TransferAccelerationUrlOptions): string ``` The https Transfer Acceleration URL of an S3 object. Specify `dualStack: true` at the options for dual-stack endpoint (connect to the bucket over IPv6). For example: - `https://bucket.s3-accelerate.amazonaws.com` - `https://bucket.s3-accelerate.amazonaws.com/key` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.TransferAccelerationUrlOptions Options for generating URL. --- ##### `urlForObject` ```typescript public urlForObject(key?: string): string ``` The https URL of an S3 object. Specify `regional: false` at the options for non-regional URLs. For example:. `https://s3.us-west-1.amazonaws.com/onlybucket` - `https://s3.us-west-1.amazonaws.com/bucket/key` - `https://s3.cn-north-1.amazonaws.com.cn/china-bucket/mykey` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ##### `virtualHostedUrlForObject` ```typescript public virtualHostedUrlForObject(key?: string, options?: VirtualHostedStyleUrlOptions): string ``` The virtual hosted-style URL of an S3 object. Specify `regional: false` at the options for non-regional URL. For example:. `https://only-bucket.s3.us-west-1.amazonaws.com` - `https://bucket.s3.us-west-1.amazonaws.com/key` - `https://bucket.s3.amazonaws.com/key` - `https://china-bucket.s3.cn-north-1.amazonaws.com.cn/mykey` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.VirtualHostedStyleUrlOptions Options for generating URL. --- ##### `addCorsRule` ```typescript public addCorsRule(rule: CorsRule): void ``` Adds a cross-origin access configuration for objects in an Amazon S3 bucket. ###### `rule`^Required - *Type:* aws-cdk-lib.aws_s3.CorsRule The CORS configuration rule to add. --- ##### `addInventory` ```typescript public addInventory(inventory: Inventory): void ``` Add an inventory configuration. ###### `inventory`^Required - *Type:* aws-cdk-lib.aws_s3.Inventory configuration to add. --- ##### `addLifecycleRule` ```typescript public addLifecycleRule(rule: LifecycleRule): void ``` Add a lifecycle rule to the bucket. ###### `rule`^Required - *Type:* aws-cdk-lib.aws_s3.LifecycleRule The rule to add. --- ##### `addMetric` ```typescript public addMetric(metric: BucketMetrics): void ``` Adds a metrics configuration for the CloudWatch request metrics from the bucket. ###### `metric`^Required - *Type:* aws-cdk-lib.aws_s3.BucketMetrics The metric configuration to add. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | isOwnedResource | Returns true if the construct was created by CDK, and false otherwise. | | isResource | Check whether the given construct is a Resource. | | fromBucketArn | *No description.* | | fromBucketAttributes | Creates a Bucket construct that represents an external bucket. | | fromBucketName | *No description.* | | fromCfnBucket | Create a mutable `IBucket` based on a low-level `CfnBucket`. | | validateBucketName | Thrown an exception if the given bucket name is not valid. | --- ##### `isConstruct` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `isOwnedResource` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.isOwnedResource(construct: IConstruct) ``` Returns true if the construct was created by CDK, and false otherwise. ###### `construct`^Required - *Type:* constructs.IConstruct --- ##### `isResource` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.isResource(construct: IConstruct) ``` Check whether the given construct is a Resource. ###### `construct`^Required - *Type:* constructs.IConstruct --- ##### `fromBucketArn` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.fromBucketArn(scope: Construct, id: string, bucketArn: string) ``` ###### `scope`^Required - *Type:* constructs.Construct --- ###### `id`^Required - *Type:* string --- ###### `bucketArn`^Required - *Type:* string --- ##### `fromBucketAttributes` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.fromBucketAttributes(scope: Construct, id: string, attrs: BucketAttributes) ``` Creates a Bucket construct that represents an external bucket. ###### `scope`^Required - *Type:* constructs.Construct The parent creating construct (usually `this`). --- ###### `id`^Required - *Type:* string The construct's name. --- ###### `attrs`^Required - *Type:* aws-cdk-lib.aws_s3.BucketAttributes A `BucketAttributes` object. Can be obtained from a call to `bucket.export()` or manually created. --- ##### `fromBucketName` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.fromBucketName(scope: Construct, id: string, bucketName: string) ``` ###### `scope`^Required - *Type:* constructs.Construct --- ###### `id`^Required - *Type:* string --- ###### `bucketName`^Required - *Type:* string --- ##### `fromCfnBucket` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.fromCfnBucket(cfnBucket: CfnBucket) ``` Create a mutable `IBucket` based on a low-level `CfnBucket`. ###### `cfnBucket`^Required - *Type:* aws-cdk-lib.aws_s3.CfnBucket --- ##### `validateBucketName` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AccessLogsBucket.validateBucketName(physicalName: string, allowLegacyBucketNaming?: boolean) ``` Thrown an exception if the given bucket name is not valid. ###### `physicalName`^Required - *Type:* string name of the bucket. --- ###### `allowLegacyBucketNaming`^Optional - *Type:* boolean allow legacy bucket naming style, default is false. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | env | aws-cdk-lib.ResourceEnvironment | The environment this resource belongs to. | | stack | aws-cdk-lib.Stack | The stack in which this resource is defined. | | bucketArn | string | The ARN of the bucket. | | bucketDomainName | string | The IPv4 DNS name of the specified bucket. | | bucketDualStackDomainName | string | The IPv6 DNS name of the specified bucket. | | bucketName | string | The name of the bucket. | | bucketRegionalDomainName | string | The regional domain name of the specified bucket. | | bucketWebsiteDomainName | string | The Domain name of the static website. | | bucketWebsiteUrl | string | The URL of the static website. | | encryptionKey | aws-cdk-lib.aws_kms.IKey | Optional KMS encryption key associated with this bucket. | | isWebsite | boolean | If this bucket has been configured for static website hosting. | | policy | aws-cdk-lib.aws_s3.BucketPolicy | The resource policy associated with this bucket. | | replicationRoleArn | string | Role used to set up permissions on this bucket for replication. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `env`^Required ```typescript public readonly env: ResourceEnvironment; ``` - *Type:* aws-cdk-lib.ResourceEnvironment The environment this resource belongs to. For resources that are created and managed by the CDK (generally, those created by creating new class instances like Role, Bucket, etc.), this is always the same as the environment of the stack they belong to; however, for imported resources (those obtained from static methods like fromRoleArn, fromBucketName, etc.), that might be different than the stack they were imported into. --- ##### `stack`^Required ```typescript public readonly stack: Stack; ``` - *Type:* aws-cdk-lib.Stack The stack in which this resource is defined. --- ##### `bucketArn`^Required ```typescript public readonly bucketArn: string; ``` - *Type:* string The ARN of the bucket. --- ##### `bucketDomainName`^Required ```typescript public readonly bucketDomainName: string; ``` - *Type:* string The IPv4 DNS name of the specified bucket. --- ##### `bucketDualStackDomainName`^Required ```typescript public readonly bucketDualStackDomainName: string; ``` - *Type:* string The IPv6 DNS name of the specified bucket. --- ##### `bucketName`^Required ```typescript public readonly bucketName: string; ``` - *Type:* string The name of the bucket. --- ##### `bucketRegionalDomainName`^Required ```typescript public readonly bucketRegionalDomainName: string; ``` - *Type:* string The regional domain name of the specified bucket. --- ##### `bucketWebsiteDomainName`^Required ```typescript public readonly bucketWebsiteDomainName: string; ``` - *Type:* string The Domain name of the static website. --- ##### `bucketWebsiteUrl`^Required ```typescript public readonly bucketWebsiteUrl: string; ``` - *Type:* string The URL of the static website. --- ##### `encryptionKey`^Optional ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey Optional KMS encryption key associated with this bucket. --- ##### `isWebsite`^Optional ```typescript public readonly isWebsite: boolean; ``` - *Type:* boolean If this bucket has been configured for static website hosting. --- ##### `policy`^Optional ```typescript public readonly policy: BucketPolicy; ``` - *Type:* aws-cdk-lib.aws_s3.BucketPolicy The resource policy associated with this bucket. If `autoCreatePolicy` is true, a `BucketPolicy` will be created upon the first call to addToResourcePolicy(s). --- ##### `replicationRoleArn`^Optional ```typescript public readonly replicationRoleArn: string; ``` - *Type:* string Role used to set up permissions on this bucket for replication. --- ### AnalyticsBucket Amazon S3 Bucket configured with best-practices and defaults for analytics. The default bucket name is `analytics---` > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/analytics-bucket](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/analytics-bucket) *Example* ```typescript import { Key } from 'aws-cdk-lib/aws-kms'; // Set context value for global data removal policy (or set in cdk.json). this.node.setContext('@data-solutions-framework-on-aws/removeDataOnDestroy', true); const encryptionKey = new Key(this, 'DataKey', { removalPolicy: cdk.RemovalPolicy.DESTROY, enableKeyRotation: true, }); new dsf.storage.AnalyticsBucket(this, 'MyAnalyticsBucket', { encryptionKey, removalPolicy: cdk.RemovalPolicy.DESTROY, }); ``` #### Initializers ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' new storage.AnalyticsBucket(scope: Construct, id: string, props: AnalyticsBucketProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucketProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucketProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | applyRemovalPolicy | Apply the given removal policy to this resource. | | addEventNotification | Adds a bucket notification event destination. | | addObjectCreatedNotification | Subscribes a destination to receive notifications when an object is created in the bucket. | | addObjectRemovedNotification | Subscribes a destination to receive notifications when an object is removed from the bucket. | | addReplicationPolicy | Function to add required permissions to the destination bucket for cross account replication. | | addToResourcePolicy | Adds a statement to the resource policy for a principal (i.e. account/role/service) to perform actions on this bucket and/or its contents. Use `bucketArn` and `arnForObjects(keys)` to obtain ARNs for this bucket or objects. | | arnForObjects | Returns an ARN that represents all objects within the bucket that match the key pattern specified. | | enableEventBridgeNotification | Enables event bridge notification, causing all events below to be sent to EventBridge:. | | grantDelete | Grants s3:DeleteObject* permission to an IAM principal for objects in this bucket. | | grantPublicAccess | Allows unrestricted access to objects from this bucket. | | grantPut | Grants s3:PutObject* and s3:Abort* permissions for this bucket to an IAM principal. | | grantPutAcl | Grant the given IAM identity permissions to modify the ACLs of objects in the given Bucket. | | grantRead | Grant read permissions for this bucket and it's contents to an IAM principal (Role/Group/User). | | grantReadWrite | Grants read/write permissions for this bucket and it's contents to an IAM principal (Role/Group/User). | | grantWrite | Grant write permissions to this bucket to an IAM principal. | | onCloudTrailEvent | Define a CloudWatch event that triggers when something happens to this repository. | | onCloudTrailPutObject | Defines an AWS CloudWatch event that triggers when an object is uploaded to the specified paths (keys) in this bucket using the PutObject API call. | | onCloudTrailWriteObject | Defines an AWS CloudWatch event that triggers when an object at the specified paths (keys) in this bucket are written to. | | s3UrlForObject | The S3 URL of an S3 object. For example:. | | transferAccelerationUrlForObject | The https Transfer Acceleration URL of an S3 object. | | urlForObject | The https URL of an S3 object. Specify `regional: false` at the options for non-regional URLs. For example:. | | virtualHostedUrlForObject | The virtual hosted-style URL of an S3 object. Specify `regional: false` at the options for non-regional URL. For example:. | | addCorsRule | Adds a cross-origin access configuration for objects in an Amazon S3 bucket. | | addInventory | Add an inventory configuration. | | addLifecycleRule | Add a lifecycle rule to the bucket. | | addMetric | Adds a metrics configuration for the CloudWatch request metrics from the bucket. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `applyRemovalPolicy` ```typescript public applyRemovalPolicy(policy: RemovalPolicy): void ``` Apply the given removal policy to this resource. The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced. The resource can be deleted (`RemovalPolicy.DESTROY`), or left in your AWS account for data recovery and cleanup later (`RemovalPolicy.RETAIN`). ###### `policy`^Required - *Type:* aws-cdk-lib.RemovalPolicy --- ##### `addEventNotification` ```typescript public addEventNotification(event: EventType, dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Adds a bucket notification event destination. > [https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html](https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html) *Example* ```typescript declare const myLambda: lambda.Function; const bucket = new s3.Bucket(this, 'MyBucket'); bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(myLambda), {prefix: 'home/myusername/*'}); ``` ###### `event`^Required - *Type:* aws-cdk-lib.aws_s3.EventType The event to trigger the notification. --- ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (Lambda, SNS Topic or SQS Queue). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] S3 object key filter rules to determine which objects trigger this event. Each filter must include a `prefix` and/or `suffix` that will be matched against the s3 object key. Refer to the S3 Developer Guide for details about allowed filter rules. --- ##### `addObjectCreatedNotification` ```typescript public addObjectCreatedNotification(dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Subscribes a destination to receive notifications when an object is created in the bucket. This is identical to calling `onEvent(EventType.OBJECT_CREATED)`. ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (see onEvent). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] Filters (see onEvent). --- ##### `addObjectRemovedNotification` ```typescript public addObjectRemovedNotification(dest: IBucketNotificationDestination, filters: ...NotificationKeyFilter[]): void ``` Subscribes a destination to receive notifications when an object is removed from the bucket. This is identical to calling `onEvent(EventType.OBJECT_REMOVED)`. ###### `dest`^Required - *Type:* aws-cdk-lib.aws_s3.IBucketNotificationDestination The notification destination (see onEvent). --- ###### `filters`^Required - *Type:* ...aws-cdk-lib.aws_s3.NotificationKeyFilter[] Filters (see onEvent). --- ##### `addReplicationPolicy` ```typescript public addReplicationPolicy(roleArn: string, accessControlTransition?: boolean, account?: string): void ``` Function to add required permissions to the destination bucket for cross account replication. These permissions will be added as a resource based policy on the bucket > [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-accesscontroltranslation.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-accesscontroltranslation.html) ###### `roleArn`^Required - *Type:* string --- ###### `accessControlTransition`^Optional - *Type:* boolean --- ###### `account`^Optional - *Type:* string --- ##### `addToResourcePolicy` ```typescript public addToResourcePolicy(permission: PolicyStatement): AddToResourcePolicyResult ``` Adds a statement to the resource policy for a principal (i.e. account/role/service) to perform actions on this bucket and/or its contents. Use `bucketArn` and `arnForObjects(keys)` to obtain ARNs for this bucket or objects. Note that the policy statement may or may not be added to the policy. For example, when an `IBucket` is created from an existing bucket, it's not possible to tell whether the bucket already has a policy attached, let alone to re-use that policy to add more statements to it. So it's safest to do nothing in these cases. ###### `permission`^Required - *Type:* aws-cdk-lib.aws_iam.PolicyStatement the policy statement to be added to the bucket's policy. --- ##### `arnForObjects` ```typescript public arnForObjects(keyPattern: string): string ``` Returns an ARN that represents all objects within the bucket that match the key pattern specified. To represent all keys, specify ``"*"``. If you need to specify a keyPattern with multiple components, concatenate them into a single string, e.g.: arnForObjects(`home/${team}/${user}/*`) ###### `keyPattern`^Required - *Type:* string --- ##### `enableEventBridgeNotification` ```typescript public enableEventBridgeNotification(): void ``` Enables event bridge notification, causing all events below to be sent to EventBridge:. Object Deleted (DeleteObject) - Object Deleted (Lifecycle expiration) - Object Restore Initiated - Object Restore Completed - Object Restore Expired - Object Storage Class Changed - Object Access Tier Changed - Object ACL Updated - Object Tags Added - Object Tags Deleted ##### `grantDelete` ```typescript public grantDelete(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants s3:DeleteObject* permission to an IAM principal for objects in this bucket. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantPublicAccess` ```typescript public grantPublicAccess(allowedActions: ...string[], keyPrefix?: string): Grant ``` Allows unrestricted access to objects from this bucket. IMPORTANT: This permission allows anyone to perform actions on S3 objects in this bucket, which is useful for when you configure your bucket as a website and want everyone to be able to read objects in the bucket without needing to authenticate. Without arguments, this method will grant read ("s3:GetObject") access to all objects ("*") in the bucket. The method returns the `iam.Grant` object, which can then be modified as needed. For example, you can add a condition that will restrict access only to an IPv4 range like this: const grant = bucket.grantPublicAccess(); grant.resourceStatement!.addCondition(‘IpAddress’, { “aws:SourceIp”: “54.240.143.0/24” }); Note that if this `IBucket` refers to an existing bucket, possibly not managed by CloudFormation, this method will have no effect, since it's impossible to modify the policy of an existing bucket. ###### `allowedActions`^Required - *Type:* ...string[] the set of S3 actions to allow. Default is "s3:GetObject". --- ###### `keyPrefix`^Optional - *Type:* string the prefix of S3 object keys (e.g. `home/*`). Default is "*". --- ##### `grantPut` ```typescript public grantPut(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants s3:PutObject* and s3:Abort* permissions for this bucket to an IAM principal. If encryption is used, permission to use the key to encrypt the contents of written files will also be granted to the same principal. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantPutAcl` ```typescript public grantPutAcl(identity: IGrantable, objectsKeyPattern?: string): Grant ``` Grant the given IAM identity permissions to modify the ACLs of objects in the given Bucket. If your application has the '@aws-cdk/aws-s3:grantWriteWithoutAcl' feature flag set, calling `grantWrite` or `grantReadWrite` no longer grants permissions to modify the ACLs of the objects; in this case, if you need to modify object ACLs, call this method explicitly. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* string --- ##### `grantRead` ```typescript public grantRead(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grant read permissions for this bucket and it's contents to an IAM principal (Role/Group/User). If encryption is used, permission to use the key to decrypt the contents of the bucket will also be granted to the same principal. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable The principal. --- ###### `objectsKeyPattern`^Optional - *Type:* any Restrict the permission to a certain key pattern (default '*'). Parameter type is `any` but `string` should be passed in. --- ##### `grantReadWrite` ```typescript public grantReadWrite(identity: IGrantable, objectsKeyPattern?: any): Grant ``` Grants read/write permissions for this bucket and it's contents to an IAM principal (Role/Group/User). If an encryption key is used, permission to use the key for encrypt/decrypt will also be granted. Before CDK version 1.85.0, this method granted the `s3:PutObject*` permission that included `s3:PutObjectAcl`, which could be used to grant read/write object access to IAM principals in other accounts. If you want to get rid of that behavior, update your CDK version to 1.85.0 or later, and make sure the `@aws-cdk/aws-s3:grantWriteWithoutAcl` feature flag is set to `true` in the `context` key of your cdk.json file. If you've already updated, but still need the principal to have permissions to modify the ACLs, use the `grantPutAcl` method. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* any --- ##### `grantWrite` ```typescript public grantWrite(identity: IGrantable, objectsKeyPattern?: any, allowedActionPatterns?: string[]): Grant ``` Grant write permissions to this bucket to an IAM principal. If encryption is used, permission to use the key to encrypt the contents of written files will also be granted to the same principal. Before CDK version 1.85.0, this method granted the `s3:PutObject*` permission that included `s3:PutObjectAcl`, which could be used to grant read/write object access to IAM principals in other accounts. If you want to get rid of that behavior, update your CDK version to 1.85.0 or later, and make sure the `@aws-cdk/aws-s3:grantWriteWithoutAcl` feature flag is set to `true` in the `context` key of your cdk.json file. If you've already updated, but still need the principal to have permissions to modify the ACLs, use the `grantPutAcl` method. ###### `identity`^Required - *Type:* aws-cdk-lib.aws_iam.IGrantable --- ###### `objectsKeyPattern`^Optional - *Type:* any --- ###### `allowedActionPatterns`^Optional - *Type:* string[] --- ##### `onCloudTrailEvent` ```typescript public onCloudTrailEvent(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Define a CloudWatch event that triggers when something happens to this repository. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `onCloudTrailPutObject` ```typescript public onCloudTrailPutObject(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Defines an AWS CloudWatch event that triggers when an object is uploaded to the specified paths (keys) in this bucket using the PutObject API call. Note that some tools like `aws s3 cp` will automatically use either PutObject or the multipart upload API depending on the file size, so using `onCloudTrailWriteObject` may be preferable. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `onCloudTrailWriteObject` ```typescript public onCloudTrailWriteObject(id: string, options?: OnCloudTrailBucketEventOptions): Rule ``` Defines an AWS CloudWatch event that triggers when an object at the specified paths (keys) in this bucket are written to. This includes the events PutObject, CopyObject, and CompleteMultipartUpload. Note that some tools like `aws s3 cp` will automatically use either PutObject or the multipart upload API depending on the file size, so using this method may be preferable to `onCloudTrailPutObject`. Requires that there exists at least one CloudTrail Trail in your account that captures the event. This method will not create the Trail. ###### `id`^Required - *Type:* string The id of the rule. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.OnCloudTrailBucketEventOptions Options for adding the rule. --- ##### `s3UrlForObject` ```typescript public s3UrlForObject(key?: string): string ``` The S3 URL of an S3 object. For example:. `s3://onlybucket` - `s3://bucket/key` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the S3 URL of the bucket is returned. --- ##### `transferAccelerationUrlForObject` ```typescript public transferAccelerationUrlForObject(key?: string, options?: TransferAccelerationUrlOptions): string ``` The https Transfer Acceleration URL of an S3 object. Specify `dualStack: true` at the options for dual-stack endpoint (connect to the bucket over IPv6). For example: - `https://bucket.s3-accelerate.amazonaws.com` - `https://bucket.s3-accelerate.amazonaws.com/key` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.TransferAccelerationUrlOptions Options for generating URL. --- ##### `urlForObject` ```typescript public urlForObject(key?: string): string ``` The https URL of an S3 object. Specify `regional: false` at the options for non-regional URLs. For example:. `https://s3.us-west-1.amazonaws.com/onlybucket` - `https://s3.us-west-1.amazonaws.com/bucket/key` - `https://s3.cn-north-1.amazonaws.com.cn/china-bucket/mykey` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ##### `virtualHostedUrlForObject` ```typescript public virtualHostedUrlForObject(key?: string, options?: VirtualHostedStyleUrlOptions): string ``` The virtual hosted-style URL of an S3 object. Specify `regional: false` at the options for non-regional URL. For example:. `https://only-bucket.s3.us-west-1.amazonaws.com` - `https://bucket.s3.us-west-1.amazonaws.com/key` - `https://bucket.s3.amazonaws.com/key` - `https://china-bucket.s3.cn-north-1.amazonaws.com.cn/mykey` ###### `key`^Optional - *Type:* string The S3 key of the object. If not specified, the URL of the bucket is returned. --- ###### `options`^Optional - *Type:* aws-cdk-lib.aws_s3.VirtualHostedStyleUrlOptions Options for generating URL. --- ##### `addCorsRule` ```typescript public addCorsRule(rule: CorsRule): void ``` Adds a cross-origin access configuration for objects in an Amazon S3 bucket. ###### `rule`^Required - *Type:* aws-cdk-lib.aws_s3.CorsRule The CORS configuration rule to add. --- ##### `addInventory` ```typescript public addInventory(inventory: Inventory): void ``` Add an inventory configuration. ###### `inventory`^Required - *Type:* aws-cdk-lib.aws_s3.Inventory configuration to add. --- ##### `addLifecycleRule` ```typescript public addLifecycleRule(rule: LifecycleRule): void ``` Add a lifecycle rule to the bucket. ###### `rule`^Required - *Type:* aws-cdk-lib.aws_s3.LifecycleRule The rule to add. --- ##### `addMetric` ```typescript public addMetric(metric: BucketMetrics): void ``` Adds a metrics configuration for the CloudWatch request metrics from the bucket. ###### `metric`^Required - *Type:* aws-cdk-lib.aws_s3.BucketMetrics The metric configuration to add. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | isOwnedResource | Returns true if the construct was created by CDK, and false otherwise. | | isResource | Check whether the given construct is a Resource. | | fromBucketArn | *No description.* | | fromBucketAttributes | Creates a Bucket construct that represents an external bucket. | | fromBucketName | *No description.* | | fromCfnBucket | Create a mutable `IBucket` based on a low-level `CfnBucket`. | | validateBucketName | Thrown an exception if the given bucket name is not valid. | --- ##### `isConstruct` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `isOwnedResource` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.isOwnedResource(construct: IConstruct) ``` Returns true if the construct was created by CDK, and false otherwise. ###### `construct`^Required - *Type:* constructs.IConstruct --- ##### `isResource` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.isResource(construct: IConstruct) ``` Check whether the given construct is a Resource. ###### `construct`^Required - *Type:* constructs.IConstruct --- ##### `fromBucketArn` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.fromBucketArn(scope: Construct, id: string, bucketArn: string) ``` ###### `scope`^Required - *Type:* constructs.Construct --- ###### `id`^Required - *Type:* string --- ###### `bucketArn`^Required - *Type:* string --- ##### `fromBucketAttributes` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.fromBucketAttributes(scope: Construct, id: string, attrs: BucketAttributes) ``` Creates a Bucket construct that represents an external bucket. ###### `scope`^Required - *Type:* constructs.Construct The parent creating construct (usually `this`). --- ###### `id`^Required - *Type:* string The construct's name. --- ###### `attrs`^Required - *Type:* aws-cdk-lib.aws_s3.BucketAttributes A `BucketAttributes` object. Can be obtained from a call to `bucket.export()` or manually created. --- ##### `fromBucketName` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.fromBucketName(scope: Construct, id: string, bucketName: string) ``` ###### `scope`^Required - *Type:* constructs.Construct --- ###### `id`^Required - *Type:* string --- ###### `bucketName`^Required - *Type:* string --- ##### `fromCfnBucket` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.fromCfnBucket(cfnBucket: CfnBucket) ``` Create a mutable `IBucket` based on a low-level `CfnBucket`. ###### `cfnBucket`^Required - *Type:* aws-cdk-lib.aws_s3.CfnBucket --- ##### `validateBucketName` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.AnalyticsBucket.validateBucketName(physicalName: string, allowLegacyBucketNaming?: boolean) ``` Thrown an exception if the given bucket name is not valid. ###### `physicalName`^Required - *Type:* string name of the bucket. --- ###### `allowLegacyBucketNaming`^Optional - *Type:* boolean allow legacy bucket naming style, default is false. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | env | aws-cdk-lib.ResourceEnvironment | The environment this resource belongs to. | | stack | aws-cdk-lib.Stack | The stack in which this resource is defined. | | bucketArn | string | The ARN of the bucket. | | bucketDomainName | string | The IPv4 DNS name of the specified bucket. | | bucketDualStackDomainName | string | The IPv6 DNS name of the specified bucket. | | bucketName | string | The name of the bucket. | | bucketRegionalDomainName | string | The regional domain name of the specified bucket. | | bucketWebsiteDomainName | string | The Domain name of the static website. | | bucketWebsiteUrl | string | The URL of the static website. | | encryptionKey | aws-cdk-lib.aws_kms.IKey | Optional KMS encryption key associated with this bucket. | | isWebsite | boolean | If this bucket has been configured for static website hosting. | | policy | aws-cdk-lib.aws_s3.BucketPolicy | The resource policy associated with this bucket. | | replicationRoleArn | string | Role used to set up permissions on this bucket for replication. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `env`^Required ```typescript public readonly env: ResourceEnvironment; ``` - *Type:* aws-cdk-lib.ResourceEnvironment The environment this resource belongs to. For resources that are created and managed by the CDK (generally, those created by creating new class instances like Role, Bucket, etc.), this is always the same as the environment of the stack they belong to; however, for imported resources (those obtained from static methods like fromRoleArn, fromBucketName, etc.), that might be different than the stack they were imported into. --- ##### `stack`^Required ```typescript public readonly stack: Stack; ``` - *Type:* aws-cdk-lib.Stack The stack in which this resource is defined. --- ##### `bucketArn`^Required ```typescript public readonly bucketArn: string; ``` - *Type:* string The ARN of the bucket. --- ##### `bucketDomainName`^Required ```typescript public readonly bucketDomainName: string; ``` - *Type:* string The IPv4 DNS name of the specified bucket. --- ##### `bucketDualStackDomainName`^Required ```typescript public readonly bucketDualStackDomainName: string; ``` - *Type:* string The IPv6 DNS name of the specified bucket. --- ##### `bucketName`^Required ```typescript public readonly bucketName: string; ``` - *Type:* string The name of the bucket. --- ##### `bucketRegionalDomainName`^Required ```typescript public readonly bucketRegionalDomainName: string; ``` - *Type:* string The regional domain name of the specified bucket. --- ##### `bucketWebsiteDomainName`^Required ```typescript public readonly bucketWebsiteDomainName: string; ``` - *Type:* string The Domain name of the static website. --- ##### `bucketWebsiteUrl`^Required ```typescript public readonly bucketWebsiteUrl: string; ``` - *Type:* string The URL of the static website. --- ##### `encryptionKey`^Optional ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey Optional KMS encryption key associated with this bucket. --- ##### `isWebsite`^Optional ```typescript public readonly isWebsite: boolean; ``` - *Type:* boolean If this bucket has been configured for static website hosting. --- ##### `policy`^Optional ```typescript public readonly policy: BucketPolicy; ``` - *Type:* aws-cdk-lib.aws_s3.BucketPolicy The resource policy associated with this bucket. If `autoCreatePolicy` is true, a `BucketPolicy` will be created upon the first call to addToResourcePolicy(s). --- ##### `replicationRoleArn`^Optional ```typescript public readonly replicationRoleArn: string; ``` - *Type:* string Role used to set up permissions on this bucket for replication. --- ### ApplicationStage ApplicationStage class that creates a CDK Pipelines Stage from an ApplicationStackFactory. #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.ApplicationStage(scope: Construct, id: string, props: ApplicationStageProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.utils.ApplicationStageProps | the ApplicationStage properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.ApplicationStageProps the ApplicationStage properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | synth | Synthesize this stage into a cloud assembly. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `synth` ```typescript public synth(options?: StageSynthesisOptions): CloudAssembly ``` Synthesize this stage into a cloud assembly. Once an assembly has been synthesized, it cannot be modified. Subsequent calls will return the same assembly. ###### `options`^Optional - *Type:* aws-cdk-lib.StageSynthesisOptions --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | isStage | Test whether the given construct is a stage. | | of | Return the stage this construct is contained with, if available. | --- ##### `isConstruct` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.ApplicationStage.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `isStage` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.ApplicationStage.isStage(x: any) ``` Test whether the given construct is a stage. ###### `x`^Required - *Type:* any --- ##### `of` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.ApplicationStage.of(construct: IConstruct) ``` Return the stage this construct is contained with, if available. If called on a nested stage, returns its parent. ###### `construct`^Required - *Type:* constructs.IConstruct --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | artifactId | string | Artifact ID of the assembly if it is a nested stage. The root stage (app) will return an empty string. | | assetOutdir | string | The cloud assembly asset output directory. | | outdir | string | The cloud assembly output directory. | | policyValidationBeta1 | aws-cdk-lib.IPolicyValidationPluginBeta1[] | Validation plugins to run during synthesis. | | stageName | string | The name of the stage. | | account | string | The default account for all resources defined within this stage. | | parentStage | aws-cdk-lib.Stage | The parent stage or `undefined` if this is the app. | | region | string | The default region for all resources defined within this stage. | | integrationTestStack | @cdklabs/aws-data-solutions-framework.utils.IntegrationTestStack | The created integration test stack. | | stackOutputsEnv | {[ key: string ]: aws-cdk-lib.CfnOutput} | The list of CfnOutputs created by the CDK Stack. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `artifactId`^Required ```typescript public readonly artifactId: string; ``` - *Type:* string Artifact ID of the assembly if it is a nested stage. The root stage (app) will return an empty string. Derived from the construct path. --- ##### `assetOutdir`^Required ```typescript public readonly assetOutdir: string; ``` - *Type:* string The cloud assembly asset output directory. --- ##### `outdir`^Required ```typescript public readonly outdir: string; ``` - *Type:* string The cloud assembly output directory. --- ##### `policyValidationBeta1`^Required ```typescript public readonly policyValidationBeta1: IPolicyValidationPluginBeta1[]; ``` - *Type:* aws-cdk-lib.IPolicyValidationPluginBeta1[] - *Default:* no validation plugins are used Validation plugins to run during synthesis. If any plugin reports any violation, synthesis will be interrupted and the report displayed to the user. --- ##### `stageName`^Required ```typescript public readonly stageName: string; ``` - *Type:* string The name of the stage. Based on names of the parent stages separated by hypens. --- ##### `account`^Optional ```typescript public readonly account: string; ``` - *Type:* string The default account for all resources defined within this stage. --- ##### `parentStage`^Optional ```typescript public readonly parentStage: Stage; ``` - *Type:* aws-cdk-lib.Stage The parent stage or `undefined` if this is the app. * --- ##### `region`^Optional ```typescript public readonly region: string; ``` - *Type:* string The default region for all resources defined within this stage. --- ##### `integrationTestStack`^Optional ```typescript public readonly integrationTestStack: IntegrationTestStack; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.IntegrationTestStack The created integration test stack. --- ##### `stackOutputsEnv`^Optional ```typescript public readonly stackOutputsEnv: {[ key: string ]: CfnOutput}; ``` - *Type:* {[ key: string ]: aws-cdk-lib.CfnOutput} The list of CfnOutputs created by the CDK Stack. --- ### AthenaWorkGroup An Amazon Athena Workgroup configured with default result bucket. *Example* ```typescript new dsf.consumption.AthenaWorkGroup(this, 'AthenaWorkGroupDefault', { name: 'athena-default', resultLocationPrefix: 'athena-default-results/' }) ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.AthenaWorkGroup(scope: Construct, id: string, props: AthenaWorkgroupProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.AthenaWorkgroupProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.AthenaWorkgroupProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | grantRunQueries | Grants running queries access to Principal. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `grantRunQueries` ```typescript public grantRunQueries(principal: IPrincipal): void ``` Grants running queries access to Principal. ###### `principal`^Required - *Type:* aws-cdk-lib.aws_iam.IPrincipal Principal to attach query access to Athena Workgroup. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.AthenaWorkGroup.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | resultBucket | aws-cdk-lib.aws_s3.IBucket | S3 Bucket used for query results. | | resultsEncryptionKey | aws-cdk-lib.aws_kms.IKey | KMS Key to encrypt the query results. | | workGroup | aws-cdk-lib.aws_athena.CfnWorkGroup | Athena Workgroup that is created. | | workGroupName | string | WorkGroup name with the randomized suffix. | | executionRole | aws-cdk-lib.aws_iam.IRole | Role used to access user resources in an Athena for Apache Spark session. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `resultBucket`^Required ```typescript public readonly resultBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket S3 Bucket used for query results. --- ##### `resultsEncryptionKey`^Required ```typescript public readonly resultsEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey KMS Key to encrypt the query results. --- ##### `workGroup`^Required ```typescript public readonly workGroup: CfnWorkGroup; ``` - *Type:* aws-cdk-lib.aws_athena.CfnWorkGroup Athena Workgroup that is created. --- ##### `workGroupName`^Required ```typescript public readonly workGroupName: string; ``` - *Type:* string WorkGroup name with the randomized suffix. --- ##### `executionRole`^Optional ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole Role used to access user resources in an Athena for Apache Spark session. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### BaseRedshiftDataAccess #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.BaseRedshiftDataAccess(scope: Construct, id: string, props: RedshiftDataProps, trackedConstructProps: any) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataProps | *No description.* | | trackedConstructProps | any | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataProps --- ##### `trackedConstructProps`^Required - *Type:* any --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.BaseRedshiftDataAccess.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | customResourceSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the Custom Resource when deployed in a VPC. | | vpcEndpoint | aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint | The created Redshift Data API interface vpc endpoint when deployed in a VPC. | | vpcEndpointSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the VPC Endpoint when deployed in a VPC. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `customResourceSecurityGroup`^Optional ```typescript public readonly customResourceSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the Custom Resource when deployed in a VPC. --- ##### `vpcEndpoint`^Optional ```typescript public readonly vpcEndpoint: IInterfaceVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint The created Redshift Data API interface vpc endpoint when deployed in a VPC. --- ##### `vpcEndpointSecurityGroup`^Optional ```typescript public readonly vpcEndpointSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the VPC Endpoint when deployed in a VPC. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### CreateServiceLinkedRole Create service linked role for the indicated service if it doesn't exists. *Example* ```typescript const slr = new dsf.utils.CreateServiceLinkedRole(this, 'CreateSLR') slr.create(dsf.utils.ServiceLinkedRoleService.REDSHIFT) ``` #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.CreateServiceLinkedRole(scope: Construct, id: string, props?: CreateServiceLinkedRoleProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.utils.CreateServiceLinkedRoleProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.utils.CreateServiceLinkedRoleProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | create | Creates the service linked role associated to the provided AWS service. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `create` ```typescript public create(slrService: ServiceLinkedRoleService): CustomResource ``` Creates the service linked role associated to the provided AWS service. ###### `slrService`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService See `ServiceLinkedRoleService` for supported service constant. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.CreateServiceLinkedRole.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ### DataCatalogDatabase An AWS Glue Data Catalog Database configured with the location and a crawler. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Governance/data-catalog-database](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Governance/data-catalog-database) *Example* ```typescript import { Bucket } from 'aws-cdk-lib/aws-s3'; new dsf.governance.DataCatalogDatabase(this, 'ExampleDatabase', { locationBucket: new Bucket(scope, 'LocationBucket'), locationPrefix: '/databasePath', name: 'example-db' }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataCatalogDatabase(scope: Construct, id: string, props: DataCatalogDatabaseProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabaseProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabaseProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | grantReadOnlyAccess | Grants read access via identity based policy to the principal. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `grantReadOnlyAccess` ```typescript public grantReadOnlyAccess(principal: IPrincipal): AddToPrincipalPolicyResult ``` Grants read access via identity based policy to the principal. This would attach an IAM Policy to the principal allowing read access to the Glue Database and all its Glue Tables. Only valid for IAM permission model. ###### `principal`^Required - *Type:* aws-cdk-lib.aws_iam.IPrincipal Principal to attach the Glue Database read access to. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataCatalogDatabase.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | database | aws-cdk-lib.aws_glue.CfnDatabase | The Glue Database that's created. | | databaseName | string | The Glue Database name with the randomized suffix to prevent name collisions in the catalog. | | crawler | aws-cdk-lib.aws_glue.CfnCrawler | The Glue Crawler created when `autoCrawl` is set to `true` (default value). | | crawlerLakeFormationDatabaseGrant | aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions | The Lake Formation grant on the database for the Crawler when Lake Formation or Hybrid is used. | | crawlerLakeFormationLocationGrant | aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions | The Lake Formation grant on the data location for the Crawler when Lake Formation or Hybrid is used. | | crawlerLakeFormationTablesGrant | aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions | The Lake Formation grant on the tables for the Crawler when Lake Formation or Hybrid is used. | | crawlerLogEncryptionKey | aws-cdk-lib.aws_kms.IKey | KMS encryption Key used by the Crawler. | | crawlerRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Glue crawler when created. | | crawlerSecurityConfiguration | aws-cdk-lib.aws_glue.CfnSecurityConfiguration | The Glue security configuration used by the Glue Crawler when created. | | dataLakeLocation | aws-cdk-lib.aws_lakeformation.CfnResource | The Lake Formation data lake location. | | dataLakeSettings | aws-cdk-lib.aws_lakeformation.CfnDataLakeSettings | The DataLakeSettings for Lake Formation. | | lakeFormationDataAccessRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by Lake Formation to access data. | | lakeFormationRevokeRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used to revoke LakeFormation IAMAllowedPrincipals. | | revokeIamAllowedPrincipal | aws-cdk-lib.custom_resources.AwsCustomResource | The custom resource for revoking IAM permissions from the database. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `database`^Required ```typescript public readonly database: CfnDatabase; ``` - *Type:* aws-cdk-lib.aws_glue.CfnDatabase The Glue Database that's created. --- ##### `databaseName`^Required ```typescript public readonly databaseName: string; ``` - *Type:* string The Glue Database name with the randomized suffix to prevent name collisions in the catalog. --- ##### `crawler`^Optional ```typescript public readonly crawler: CfnCrawler; ``` - *Type:* aws-cdk-lib.aws_glue.CfnCrawler The Glue Crawler created when `autoCrawl` is set to `true` (default value). This property can be undefined if `autoCrawl` is set to `false`. --- ##### `crawlerLakeFormationDatabaseGrant`^Optional ```typescript public readonly crawlerLakeFormationDatabaseGrant: CfnPrincipalPermissions; ``` - *Type:* aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions The Lake Formation grant on the database for the Crawler when Lake Formation or Hybrid is used. --- ##### `crawlerLakeFormationLocationGrant`^Optional ```typescript public readonly crawlerLakeFormationLocationGrant: CfnPrincipalPermissions; ``` - *Type:* aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions The Lake Formation grant on the data location for the Crawler when Lake Formation or Hybrid is used. --- ##### `crawlerLakeFormationTablesGrant`^Optional ```typescript public readonly crawlerLakeFormationTablesGrant: CfnPrincipalPermissions; ``` - *Type:* aws-cdk-lib.aws_lakeformation.CfnPrincipalPermissions The Lake Formation grant on the tables for the Crawler when Lake Formation or Hybrid is used. --- ##### `crawlerLogEncryptionKey`^Optional ```typescript public readonly crawlerLogEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey KMS encryption Key used by the Crawler. --- ##### `crawlerRole`^Optional ```typescript public readonly crawlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Glue crawler when created. --- ##### `crawlerSecurityConfiguration`^Optional ```typescript public readonly crawlerSecurityConfiguration: CfnSecurityConfiguration; ``` - *Type:* aws-cdk-lib.aws_glue.CfnSecurityConfiguration The Glue security configuration used by the Glue Crawler when created. --- ##### `dataLakeLocation`^Optional ```typescript public readonly dataLakeLocation: CfnResource; ``` - *Type:* aws-cdk-lib.aws_lakeformation.CfnResource The Lake Formation data lake location. --- ##### `dataLakeSettings`^Optional ```typescript public readonly dataLakeSettings: CfnDataLakeSettings; ``` - *Type:* aws-cdk-lib.aws_lakeformation.CfnDataLakeSettings The DataLakeSettings for Lake Formation. --- ##### `lakeFormationDataAccessRole`^Optional ```typescript public readonly lakeFormationDataAccessRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by Lake Formation to access data. --- ##### `lakeFormationRevokeRole`^Optional ```typescript public readonly lakeFormationRevokeRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used to revoke LakeFormation IAMAllowedPrincipals. --- ##### `revokeIamAllowedPrincipal`^Optional ```typescript public readonly revokeIamAllowedPrincipal: AwsCustomResource; ``` - *Type:* aws-cdk-lib.custom_resources.AwsCustomResource The custom resource for revoking IAM permissions from the database. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataLakeCatalog Creates a Data Lake Catalog on top of a `DataLakeStorage`. The Data Lake Catalog is composed of 3 `DataCatalogDatabase`, one for each storage layer. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Governance/data-lake-catalog](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Governance/data-lake-catalog) *Example* ```typescript import { Key } from 'aws-cdk-lib/aws-kms'; const logEncryptionKey = new Key(this, 'ExampleLogKey'); const storage = new dsf.storage.DataLakeStorage(this, "ExampleStorage"); const dataLakeCatalog = new dsf.governance.DataLakeCatalog(this, "ExampleDataLakeCatalog", { dataLakeStorage: storage, databaseName: "exampledb", crawlerLogEncryptionKey: logEncryptionKey }) ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataLakeCatalog(scope: Construct, id: string, props: DataLakeCatalogProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.governance.DataLakeCatalogProps | the DataLakeCatalog properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataLakeCatalogProps the DataLakeCatalog properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataLakeCatalog.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | bronzeCatalogDatabase | @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase | The Glue Database for the Bronze S3 Bucket. | | goldCatalogDatabase | @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase | The Glue Database for the Gold S3 Bucket. | | silverCatalogDatabase | @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase | The Glue Database for the Silver S3 Bucket. | | crawlerLogEncryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt the Glue Crawler logs. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `bronzeCatalogDatabase`^Required ```typescript public readonly bronzeCatalogDatabase: DataCatalogDatabase; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase The Glue Database for the Bronze S3 Bucket. --- ##### `goldCatalogDatabase`^Required ```typescript public readonly goldCatalogDatabase: DataCatalogDatabase; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase The Glue Database for the Gold S3 Bucket. --- ##### `silverCatalogDatabase`^Required ```typescript public readonly silverCatalogDatabase: DataCatalogDatabase; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataCatalogDatabase The Glue Database for the Silver S3 Bucket. --- ##### `crawlerLogEncryptionKey`^Optional ```typescript public readonly crawlerLogEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used to encrypt the Glue Crawler logs. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataLakeStorage Creates the storage layer for a data lake, composed of 3 `AnalyticsBucket` for Bronze, Silver, and Gold data. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/data-lake-storage](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Storage/data-lake-storage) *Example* ```typescript // Set the context value for global data removal policy this.node.setContext('@data-solutions-framework-on-aws/removeDataOnDestroy', true); new dsf.storage.DataLakeStorage(this, 'MyDataLakeStorage', { bronzeBucketName: 'my-bronze', bronzeBucketInfrequentAccessDelay: 90, bronzeBucketArchiveDelay: 180, silverBucketName: 'my-silver', silverBucketInfrequentAccessDelay: 180, silverBucketArchiveDelay: 360, goldBucketName: 'my-gold', goldBucketInfrequentAccessDelay: 180, goldBucketArchiveDelay: 360, removalPolicy: cdk.RemovalPolicy.DESTROY, }); ``` #### Initializers ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' new storage.DataLakeStorage(scope: Construct, id: string, props?: DataLakeStorageProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.storage.DataLakeStorageProps | the DataLakeStorageProps properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.storage.DataLakeStorageProps the DataLakeStorageProps properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' storage.DataLakeStorage.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | accessLogsBucket | @cdklabs/aws-data-solutions-framework.storage.AccessLogsBucket | The S3 Bucket for access logs. | | bronzeBucket | @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket | The S3 Bucket for Bronze layer. | | dataLakeKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt all DataLakeStorage S3 buckets. | | goldBucket | @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket | The S3 Bucket for Gold layer. | | silverBucket | @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket | The S3 Bucket for Silver layer. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `accessLogsBucket`^Required ```typescript public readonly accessLogsBucket: AccessLogsBucket; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.AccessLogsBucket The S3 Bucket for access logs. --- ##### `bronzeBucket`^Required ```typescript public readonly bronzeBucket: AnalyticsBucket; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket The S3 Bucket for Bronze layer. --- ##### `dataLakeKey`^Required ```typescript public readonly dataLakeKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used to encrypt all DataLakeStorage S3 buckets. --- ##### `goldBucket`^Required ```typescript public readonly goldBucket: AnalyticsBucket; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket The S3 Bucket for Gold layer. --- ##### `silverBucket`^Required ```typescript public readonly silverBucket: AnalyticsBucket; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.AnalyticsBucket The S3 Bucket for Silver layer. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataVpc Creates a VPC with best practices for securely deploying data solutions. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Utils/data-vpc](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Utils/data-vpc) *Example* ```typescript const vpc = new dsf.utils.DataVpc(this, 'DataVpc', { vpcCidr: '10.0.0.0/16', }); vpc.tagVpc('Name', 'My VPC'); ``` #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.DataVpc(scope: Construct, id: string, props: DataVpcProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.utils.DataVpcProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.DataVpcProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | tagVpc | Tag the VPC and the subnets. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `tagVpc` ```typescript public tagVpc(key: string, value: string): void ``` Tag the VPC and the subnets. ###### `key`^Required - *Type:* string the tag key. --- ###### `value`^Required - *Type:* string the tag value. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.DataVpc.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | flowLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group created for the VPC flow logs. | | flowLogKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt VPC flow logs. | | flowLogRole | aws-cdk-lib.aws_iam.IRole | The IAM role used to publish VPC Flow Logs. | | s3VpcEndpoint | aws-cdk-lib.aws_ec2.IGatewayVpcEndpoint | The S3 VPC endpoint gateway. | | vpc | aws-cdk-lib.aws_ec2.Vpc | The amazon VPC created. | | clientVpnEndpoint | aws-cdk-lib.aws_ec2.ClientVpnEndpoint | The Client VPN Endpoint. | | vpnLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The log group for Client VPN Endpoint. | | vpnSecurityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The security group for Client VPN Endpoint. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `flowLogGroup`^Required ```typescript public readonly flowLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group created for the VPC flow logs. --- ##### `flowLogKey`^Required ```typescript public readonly flowLogKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used to encrypt VPC flow logs. --- ##### `flowLogRole`^Required ```typescript public readonly flowLogRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role used to publish VPC Flow Logs. --- ##### `s3VpcEndpoint`^Required ```typescript public readonly s3VpcEndpoint: IGatewayVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IGatewayVpcEndpoint The S3 VPC endpoint gateway. --- ##### `vpc`^Required ```typescript public readonly vpc: Vpc; ``` - *Type:* aws-cdk-lib.aws_ec2.Vpc The amazon VPC created. --- ##### `clientVpnEndpoint`^Optional ```typescript public readonly clientVpnEndpoint: ClientVpnEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.ClientVpnEndpoint The Client VPN Endpoint. --- ##### `vpnLogGroup`^Optional ```typescript public readonly vpnLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The log group for Client VPN Endpoint. --- ##### `vpnSecurityGroups`^Optional ```typescript public readonly vpnSecurityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The security group for Client VPN Endpoint. --- ### DataZoneCustomAssetTypeFactory Factory construct providing resources to create a DataZone custom asset type. *Example* ```typescript new dsf.governance.DataZoneCustomAssetTypeFactory(this, 'CustomAssetTypeFactory', { domainId: 'aba_dc999t9ime9sss', }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneCustomAssetTypeFactory(scope: Construct, id: string, props: DataZoneCustomAssetTypeFactoryProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.governance.DataZoneCustomAssetTypeFactoryProps | The DataZoneCustomAssetTypeFactory properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneCustomAssetTypeFactoryProps The DataZoneCustomAssetTypeFactory properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | createCustomAssetType | Creates a DataZone custom asset type based on the provided properties. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `createCustomAssetType` ```typescript public createCustomAssetType(id: string, customAssetType: DataZoneCustomAssetTypeProps): CustomAssetType ``` Creates a DataZone custom asset type based on the provided properties. ###### `id`^Required - *Type:* string the ID of the CDK Construct. --- ###### `customAssetType`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneCustomAssetTypeProps the properties of the custom asset type. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneCustomAssetTypeFactory.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | createFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the DataZone custom asset type creation. | | createLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Logs Log Group for the DataZone custom asset type creation. | | createRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the DataZone custom asset type creation. | | serviceToken | string | The service token for the custom resource. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `createFunction`^Required ```typescript public readonly createFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the DataZone custom asset type creation. --- ##### `createLogGroup`^Required ```typescript public readonly createLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Logs Log Group for the DataZone custom asset type creation. --- ##### `createRole`^Required ```typescript public readonly createRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the DataZone custom asset type creation. --- ##### `serviceToken`^Required ```typescript public readonly serviceToken: string; ``` - *Type:* string The service token for the custom resource. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataZoneGsrMskDataSource A DataZone custom data source for MSK (Managed Streaming for Kafka) with integration for Glue Schema Registry. The construct creates assets with the MskTopicAssetType in DataZone based on schema definitions in a Glue Schema Registry. It can be either scheduled or react on Schema Registry events (Create Schema, Update Schema, Create Schema Revision). *Example* ```typescript import { Schedule } from 'aws-cdk-lib/aws-events'; new dsf.governance.DataZoneGsrMskDataSource(this, 'MskDatasource', { domainId: 'aba_dc999t9ime9sss', projectId: '999999b3m5cpz', registryName: 'MyRegistry', clusterName: 'MyCluster', runSchedule: Schedule.cron({ minute: '0', hour: '12' }), // Trigger daily at noon enableSchemaRegistryEvent: true, // Enable events for Glue Schema Registry changes }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneGsrMskDataSource(scope: Construct, id: string, props: DataZoneGsrMskDataSourceProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.governance.DataZoneGsrMskDataSourceProps | The DataZoneGsrMskDataSourceProps properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneGsrMskDataSourceProps The DataZoneGsrMskDataSourceProps properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneGsrMskDataSource.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | dataZoneMembership | aws-cdk-lib.aws_datazone.CfnProjectMembership | The membership of the Lambda Role on the DataZone Project. | | lambdaFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function creating DataZone Inventory Assets. | | lambdaLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The Log Group for the Lambda Function creating DataZone Inventory Assets. | | lambdaRole | aws-cdk-lib.aws_iam.IRole | The IAM Role of the Lambda Function interacting with DataZone API to create inventory assets. | | ssmParameterKey | aws-cdk-lib.aws_kms.Key | The KMS Key used to encrypt the SSM Parameter storing assets information. | | createUpdateEventRule | aws-cdk-lib.aws_events.IRule | The Event Bridge Rule for schema creation and update. | | deleteEventRule | aws-cdk-lib.aws_events.IRule | The Event Bridge Rule for schema deletion. | | scheduleRule | aws-cdk-lib.aws_events.IRule | The Event Bridge Rule for trigger the data source execution. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `dataZoneMembership`^Required ```typescript public readonly dataZoneMembership: CfnProjectMembership; ``` - *Type:* aws-cdk-lib.aws_datazone.CfnProjectMembership The membership of the Lambda Role on the DataZone Project. --- ##### `lambdaFunction`^Required ```typescript public readonly lambdaFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function creating DataZone Inventory Assets. --- ##### `lambdaLogGroup`^Required ```typescript public readonly lambdaLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The Log Group for the Lambda Function creating DataZone Inventory Assets. --- ##### `lambdaRole`^Required ```typescript public readonly lambdaRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role of the Lambda Function interacting with DataZone API to create inventory assets. --- ##### `ssmParameterKey`^Required ```typescript public readonly ssmParameterKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key The KMS Key used to encrypt the SSM Parameter storing assets information. --- ##### `createUpdateEventRule`^Optional ```typescript public readonly createUpdateEventRule: IRule; ``` - *Type:* aws-cdk-lib.aws_events.IRule The Event Bridge Rule for schema creation and update. --- ##### `deleteEventRule`^Optional ```typescript public readonly deleteEventRule: IRule; ``` - *Type:* aws-cdk-lib.aws_events.IRule The Event Bridge Rule for schema deletion. --- ##### `scheduleRule`^Optional ```typescript public readonly scheduleRule: IRule; ``` - *Type:* aws-cdk-lib.aws_events.IRule The Event Bridge Rule for trigger the data source execution. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataZoneMskAssetType A DataZone custom asset type representing an MSK topic. *Example* ```typescript new dsf.governance.DataZoneMskAssetType(this, 'MskAssetType', { domainId: 'aba_dc999t9ime9sss', projectId: '999999b3m5cpz', }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneMskAssetType(scope: Construct, id: string, props: DataZoneMskAssetTypeProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.governance.DataZoneMskAssetTypeProps | The DataZoneMskAssetTypeProps properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneMskAssetTypeProps The DataZoneMskAssetTypeProps properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneMskAssetType.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | mskCustomAssetType | @cdklabs/aws-data-solutions-framework.governance.CustomAssetType | The custom asset type for MSK. | | owningProject | aws-cdk-lib.aws_datazone.CfnProject | The project owning the MSK asset type. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `mskCustomAssetType`^Required ```typescript public readonly mskCustomAssetType: CustomAssetType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.CustomAssetType The custom asset type for MSK. --- ##### `owningProject`^Optional ```typescript public readonly owningProject: CfnProject; ``` - *Type:* aws-cdk-lib.aws_datazone.CfnProject The project owning the MSK asset type. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### DataZoneMskCentralAuthorizer A central authorizer workflow for granting read access to Kafka topics. The workflow is triggered by an event sent to the DataZone event bus. First, it collects metadata from DataZone about the Kafka topics. Then, it grants access to the relevant IAM roles. Finally acknowledge the subscription grant in DataZone. *Example* ```typescript new dsf.governance.DataZoneMskCentralAuthorizer(this, 'MskAuthorizer', { domainId: 'aba_dc999t9ime9sss', }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneMskCentralAuthorizer(scope: Construct, id: string, props: DataZoneMskCentralAuthorizerProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.governance.DataZoneMskCentralAuthorizerProps | The DataZoneMskCentralAuthorizer properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneMskCentralAuthorizerProps The DataZoneMskCentralAuthorizer properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | registerAccount | Connect the central authorizer workflow with environment authorizer workflows in other accounts. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `registerAccount` ```typescript public registerAccount(id: string, accountId: string): void ``` Connect the central authorizer workflow with environment authorizer workflows in other accounts. This method grants the environment workflow to send events in the default Event Bridge bus for orchestration. ###### `id`^Required - *Type:* string The construct ID to use. --- ###### `accountId`^Required - *Type:* string The account ID to register the authorizer with. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneMskCentralAuthorizer.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | datazoneCallbackFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function used to acknowledge the subscription grant in DataZone. | | datazoneCallbackLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The Cloudwatch Log Group for logging the datazone callback. | | datazoneCallbackRole | aws-cdk-lib.aws_iam.IRole | The role used to acknowledge the subscription grant in DataZone. | | datazoneEventRole | aws-cdk-lib.aws_iam.IRole | The role used by the DataZone event to trigger the authorizer workflow. | | datazoneEventRule | aws-cdk-lib.aws_events.IRule | The event rule used to trigger the authorizer workflow. | | deadLetterKey | aws-cdk-lib.aws_kms.IKey | The key used to encrypt the dead letter queue. | | deadLetterQueue | aws-cdk-lib.aws_sqs.IQueue | The SQS Queue used as a dead letter queue for the authorizer workflow. | | metadataCollectorFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function used to collect metadata from DataZone. | | metadataCollectorLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The Cloudwatch Log Group for logging the metadata collector. | | metadataCollectorRole | aws-cdk-lib.aws_iam.IRole | The role used to collect metadata from DataZone. | | stateMachine | aws-cdk-lib.aws_stepfunctions.StateMachine | The state machine used to orchestrate the authorizer workflow. | | stateMachineCallbackRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the authorizer workflow callback. | | stateMachineRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the authorizer workflow State Machine. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used to log the authorizer state machine. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `datazoneCallbackFunction`^Required ```typescript public readonly datazoneCallbackFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function used to acknowledge the subscription grant in DataZone. --- ##### `datazoneCallbackLogGroup`^Required ```typescript public readonly datazoneCallbackLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The Cloudwatch Log Group for logging the datazone callback. --- ##### `datazoneCallbackRole`^Required ```typescript public readonly datazoneCallbackRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The role used to acknowledge the subscription grant in DataZone. --- ##### `datazoneEventRole`^Required ```typescript public readonly datazoneEventRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The role used by the DataZone event to trigger the authorizer workflow. --- ##### `datazoneEventRule`^Required ```typescript public readonly datazoneEventRule: IRule; ``` - *Type:* aws-cdk-lib.aws_events.IRule The event rule used to trigger the authorizer workflow. --- ##### `deadLetterKey`^Required ```typescript public readonly deadLetterKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The key used to encrypt the dead letter queue. --- ##### `deadLetterQueue`^Required ```typescript public readonly deadLetterQueue: IQueue; ``` - *Type:* aws-cdk-lib.aws_sqs.IQueue The SQS Queue used as a dead letter queue for the authorizer workflow. --- ##### `metadataCollectorFunction`^Required ```typescript public readonly metadataCollectorFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function used to collect metadata from DataZone. --- ##### `metadataCollectorLogGroup`^Required ```typescript public readonly metadataCollectorLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The Cloudwatch Log Group for logging the metadata collector. --- ##### `metadataCollectorRole`^Required ```typescript public readonly metadataCollectorRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The role used to collect metadata from DataZone. --- ##### `stateMachine`^Required ```typescript public readonly stateMachine: StateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.StateMachine The state machine used to orchestrate the authorizer workflow. --- ##### `stateMachineCallbackRole`^Required ```typescript public readonly stateMachineCallbackRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role The IAM Role used by the authorizer workflow callback. --- ##### `stateMachineRole`^Required ```typescript public readonly stateMachineRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role The IAM Role used by the authorizer workflow State Machine. --- ##### `stateMachineLogGroup`^Required ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used to log the authorizer state machine. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | AUTHORIZER_NAME | string | The name of the authorizer. | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | | MSK_ASSET_TYPE | string | The asset type for the DataZone custom asset type. | --- ##### `AUTHORIZER_NAME`^Required ```typescript public readonly AUTHORIZER_NAME: string; ``` - *Type:* string The name of the authorizer. --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ##### `MSK_ASSET_TYPE`^Required ```typescript public readonly MSK_ASSET_TYPE: string; ``` - *Type:* string The asset type for the DataZone custom asset type. --- ### DataZoneMskEnvironmentAuthorizer An environment authorizer workflow for granting read access to Kafka topics. The workflow is triggered by an event sent by the central authorizer construct. It creates IAM policies required for the Kafka client to access the relevant topics. It supports MSK provisioned and serverless, in single and cross accounts, and grant/revoke requests. *Example* ```typescript new dsf.governance.DataZoneMskEnvironmentAuthorizer(this, 'MskAuthorizer', { domainId: 'aba_dc999t9ime9sss', }); ``` #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneMskEnvironmentAuthorizer(scope: Construct, id: string, props: DataZoneMskEnvironmentAuthorizerProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | The CDK Construct scope. | | id | string | The CDK Construct id. | | props | @cdklabs/aws-data-solutions-framework.governance.DataZoneMskEnvironmentAuthorizerProps | The props for the DataZoneMskEnvironmentAuthorizer construct. | --- ##### `scope`^Required - *Type:* constructs.Construct The CDK Construct scope. --- ##### `id`^Required - *Type:* string The CDK Construct id. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneMskEnvironmentAuthorizerProps The props for the DataZoneMskEnvironmentAuthorizer construct. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneMskEnvironmentAuthorizer.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | grantFunction | aws-cdk-lib.aws_lambda.IFunction | The lambda function used to grant access to Kafka topics. | | grantLogGroup | aws-cdk-lib.aws_logs.LogGroup | The CloudWatch Log Group used by the grant function. | | grantRole | aws-cdk-lib.aws_iam.IRole | The IAM role used to grant access to Kafka topics. | | stateMachine | aws-cdk-lib.aws_stepfunctions.IStateMachine | The environment authorizer State Machine. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Step Functions state machine. | | stateMachineRole | aws-cdk-lib.aws_iam.IRole | The IAM role used by the environment authorizer State Machine. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `grantFunction`^Required ```typescript public readonly grantFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The lambda function used to grant access to Kafka topics. --- ##### `grantLogGroup`^Required ```typescript public readonly grantLogGroup: LogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.LogGroup The CloudWatch Log Group used by the grant function. --- ##### `grantRole`^Required ```typescript public readonly grantRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role used to grant access to Kafka topics. --- ##### `stateMachine`^Required ```typescript public readonly stateMachine: IStateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.IStateMachine The environment authorizer State Machine. --- ##### `stateMachineLogGroup`^Required ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Step Functions state machine. --- ##### `stateMachineRole`^Required ```typescript public readonly stateMachineRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role used by the environment authorizer State Machine. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | | PERMISSIONS_BOUNDARY_STATEMENTS | aws-cdk-lib.aws_iam.PolicyStatement | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ##### `PERMISSIONS_BOUNDARY_STATEMENTS`^Required ```typescript public readonly PERMISSIONS_BOUNDARY_STATEMENTS: PolicyStatement; ``` - *Type:* aws-cdk-lib.aws_iam.PolicyStatement --- ### IntegrationTestStack Stack that contains resources related to running the integration test. #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.IntegrationTestStack(scope: Construct, id: string, stage: CICDStage, integScriptPath: string, integTestCommand: string, stackOutputsEnv?: {[ key: string ]: CfnOutput}, integTestPermissions?: PolicyStatement[]) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the scope of the construct. | | id | string | the id of the construct. | | stage | @cdklabs/aws-data-solutions-framework.utils.CICDStage | the stage in the CI/CD pipeline. | | integScriptPath | string | the path of the integration test script. | | integTestCommand | string | the command to be used to run the integration test script. | | stackOutputsEnv | {[ key: string ]: aws-cdk-lib.CfnOutput} | the output from the application stack to use as input to the test script. | | integTestPermissions | aws-cdk-lib.aws_iam.PolicyStatement[] | IAM permissions for the integration test script. | --- ##### `scope`^Required - *Type:* constructs.Construct the scope of the construct. --- ##### `id`^Required - *Type:* string the id of the construct. --- ##### `stage`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.CICDStage the stage in the CI/CD pipeline. --- ##### `integScriptPath`^Required - *Type:* string the path of the integration test script. --- ##### `integTestCommand`^Required - *Type:* string the command to be used to run the integration test script. --- ##### `stackOutputsEnv`^Optional - *Type:* {[ key: string ]: aws-cdk-lib.CfnOutput} the output from the application stack to use as input to the test script. --- ##### `integTestPermissions`^Optional - *Type:* aws-cdk-lib.aws_iam.PolicyStatement[] IAM permissions for the integration test script. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | addDependency | Add a dependency between this stack and another stack. | | addMetadata | Adds an arbitrary key-value pair, with information you want to record about the stack. | | addTransform | Add a Transform to this stack. A Transform is a macro that AWS CloudFormation uses to process your template. | | exportStringListValue | Create a CloudFormation Export for a string list value. | | exportValue | Create a CloudFormation Export for a string value. | | formatArn | Creates an ARN from components. | | getLogicalId | Allocates a stack-unique CloudFormation-compatible logical identity for a specific resource. | | regionalFact | Look up a fact value for the given fact for the region of this stack. | | renameLogicalId | Rename a generated logical identities. | | reportMissingContextKey | Indicate that a context key was expected. | | resolve | Resolve a tokenized value in the context of the current stack. | | splitArn | Splits the provided ARN into its components. | | toJsonString | Convert an object, potentially containing tokens, to a JSON string. | | toYamlString | Convert an object, potentially containing tokens, to a YAML string. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `addDependency` ```typescript public addDependency(target: Stack, reason?: string): void ``` Add a dependency between this stack and another stack. This can be used to define dependencies between any two stacks within an app, and also supports nested stacks. ###### `target`^Required - *Type:* aws-cdk-lib.Stack --- ###### `reason`^Optional - *Type:* string --- ##### `addMetadata` ```typescript public addMetadata(key: string, value: any): void ``` Adds an arbitrary key-value pair, with information you want to record about the stack. These get translated to the Metadata section of the generated template. > [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html) ###### `key`^Required - *Type:* string --- ###### `value`^Required - *Type:* any --- ##### `addTransform` ```typescript public addTransform(transform: string): void ``` Add a Transform to this stack. A Transform is a macro that AWS CloudFormation uses to process your template. Duplicate values are removed when stack is synthesized. > [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html) *Example* ```typescript declare const stack: Stack; stack.addTransform('AWS::Serverless-2016-10-31') ``` ###### `transform`^Required - *Type:* string The transform to add. --- ##### `exportStringListValue` ```typescript public exportStringListValue(exportedValue: any, options?: ExportValueOptions): string[] ``` Create a CloudFormation Export for a string list value. Returns a string list representing the corresponding `Fn.importValue()` expression for this Export. The export expression is automatically wrapped with an `Fn::Join` and the import value with an `Fn::Split`, since CloudFormation can only export strings. You can control the name for the export by passing the `name` option. If you don't supply a value for `name`, the value you're exporting must be a Resource attribute (for example: `bucket.bucketName`) and it will be given the same name as the automatic cross-stack reference that would be created if you used the attribute in another Stack. One of the uses for this method is to *remove* the relationship between two Stacks established by automatic cross-stack references. It will temporarily ensure that the CloudFormation Export still exists while you remove the reference from the consuming stack. After that, you can remove the resource and the manual export. See `exportValue` for an example of this process. ###### `exportedValue`^Required - *Type:* any --- ###### `options`^Optional - *Type:* aws-cdk-lib.ExportValueOptions --- ##### `exportValue` ```typescript public exportValue(exportedValue: any, options?: ExportValueOptions): string ``` Create a CloudFormation Export for a string value. Returns a string representing the corresponding `Fn.importValue()` expression for this Export. You can control the name for the export by passing the `name` option. If you don't supply a value for `name`, the value you're exporting must be a Resource attribute (for example: `bucket.bucketName`) and it will be given the same name as the automatic cross-stack reference that would be created if you used the attribute in another Stack. One of the uses for this method is to *remove* the relationship between two Stacks established by automatic cross-stack references. It will temporarily ensure that the CloudFormation Export still exists while you remove the reference from the consuming stack. After that, you can remove the resource and the manual export. Here is how the process works. Let's say there are two stacks, `producerStack` and `consumerStack`, and `producerStack` has a bucket called `bucket`, which is referenced by `consumerStack` (perhaps because an AWS Lambda Function writes into it, or something like that). It is not safe to remove `producerStack.bucket` because as the bucket is being deleted, `consumerStack` might still be using it. Instead, the process takes two deployments: **Deployment 1: break the relationship**: - Make sure `consumerStack` no longer references `bucket.bucketName` (maybe the consumer stack now uses its own bucket, or it writes to an AWS DynamoDB table, or maybe you just remove the Lambda Function altogether). - In the `ProducerStack` class, call `this.exportValue(this.bucket.bucketName)`. This will make sure the CloudFormation Export continues to exist while the relationship between the two stacks is being broken. - Deploy (this will effectively only change the `consumerStack`, but it's safe to deploy both). **Deployment 2: remove the bucket resource**: - You are now free to remove the `bucket` resource from `producerStack`. - Don't forget to remove the `exportValue()` call as well. - Deploy again (this time only the `producerStack` will be changed -- the bucket will be deleted). ###### `exportedValue`^Required - *Type:* any --- ###### `options`^Optional - *Type:* aws-cdk-lib.ExportValueOptions --- ##### `formatArn` ```typescript public formatArn(components: ArnComponents): string ``` Creates an ARN from components. If `partition`, `region` or `account` are not specified, the stack's partition, region and account will be used. If any component is the empty string, an empty string will be inserted into the generated ARN at the location that component corresponds to. The ARN will be formatted as follows: arn:{partition}:{service}:{region}:{account}:{resource}{sep}{resource-name} The required ARN pieces that are omitted will be taken from the stack that the 'scope' is attached to. If all ARN pieces are supplied, the supplied scope can be 'undefined'. ###### `components`^Required - *Type:* aws-cdk-lib.ArnComponents --- ##### `getLogicalId` ```typescript public getLogicalId(element: CfnElement): string ``` Allocates a stack-unique CloudFormation-compatible logical identity for a specific resource. This method is called when a `CfnElement` is created and used to render the initial logical identity of resources. Logical ID renames are applied at this stage. This method uses the protected method `allocateLogicalId` to render the logical ID for an element. To modify the naming scheme, extend the `Stack` class and override this method. ###### `element`^Required - *Type:* aws-cdk-lib.CfnElement The CloudFormation element for which a logical identity is needed. --- ##### `regionalFact` ```typescript public regionalFact(factName: string, defaultValue?: string): string ``` Look up a fact value for the given fact for the region of this stack. Will return a definite value only if the region of the current stack is resolved. If not, a lookup map will be added to the stack and the lookup will be done at CDK deployment time. What regions will be included in the lookup map is controlled by the `@aws-cdk/core:target-partitions` context value: it must be set to a list of partitions, and only regions from the given partitions will be included. If no such context key is set, all regions will be included. This function is intended to be used by construct library authors. Application builders can rely on the abstractions offered by construct libraries and do not have to worry about regional facts. If `defaultValue` is not given, it is an error if the fact is unknown for the given region. ###### `factName`^Required - *Type:* string --- ###### `defaultValue`^Optional - *Type:* string --- ##### `renameLogicalId` ```typescript public renameLogicalId(oldId: string, newId: string): void ``` Rename a generated logical identities. To modify the naming scheme strategy, extend the `Stack` class and override the `allocateLogicalId` method. ###### `oldId`^Required - *Type:* string --- ###### `newId`^Required - *Type:* string --- ##### `reportMissingContextKey` ```typescript public reportMissingContextKey(report: MissingContext): void ``` Indicate that a context key was expected. Contains instructions which will be emitted into the cloud assembly on how the key should be supplied. ###### `report`^Required - *Type:* aws-cdk-lib.cloud_assembly_schema.MissingContext The set of parameters needed to obtain the context. --- ##### `resolve` ```typescript public resolve(obj: any): any ``` Resolve a tokenized value in the context of the current stack. ###### `obj`^Required - *Type:* any --- ##### `splitArn` ```typescript public splitArn(arn: string, arnFormat: ArnFormat): ArnComponents ``` Splits the provided ARN into its components. Works both if 'arn' is a string like 'arn:aws:s3:::bucket', and a Token representing a dynamic CloudFormation expression (in which case the returned components will also be dynamic CloudFormation expressions, encoded as Tokens). ###### `arn`^Required - *Type:* string the ARN to split into its components. --- ###### `arnFormat`^Required - *Type:* aws-cdk-lib.ArnFormat the expected format of 'arn' - depends on what format the service 'arn' represents uses. --- ##### `toJsonString` ```typescript public toJsonString(obj: any, space?: number): string ``` Convert an object, potentially containing tokens, to a JSON string. ###### `obj`^Required - *Type:* any --- ###### `space`^Optional - *Type:* number --- ##### `toYamlString` ```typescript public toYamlString(obj: any): string ``` Convert an object, potentially containing tokens, to a YAML string. ###### `obj`^Required - *Type:* any --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | isStack | Return whether the given object is a Stack. | | of | Looks up the first stack scope in which `construct` is defined. | --- ##### `isConstruct` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.IntegrationTestStack.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `isStack` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.IntegrationTestStack.isStack(x: any) ``` Return whether the given object is a Stack. We do attribute detection since we can't reliably use 'instanceof'. ###### `x`^Required - *Type:* any --- ##### `of` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.IntegrationTestStack.of(construct: IConstruct) ``` Looks up the first stack scope in which `construct` is defined. Fails if there is no stack up the tree. ###### `construct`^Required - *Type:* constructs.IConstruct The construct to start the search from. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | account | string | The AWS account into which this stack will be deployed. | | artifactId | string | The ID of the cloud assembly artifact for this stack. | | availabilityZones | string[] | Returns the list of AZs that are available in the AWS environment (account/region) associated with this stack. | | bundlingRequired | boolean | Indicates whether the stack requires bundling or not. | | dependencies | aws-cdk-lib.Stack[] | Return the stacks this stack depends on. | | environment | string | The environment coordinates in which this stack is deployed. | | nested | boolean | Indicates if this is a nested stack, in which case `parentStack` will include a reference to it's parent. | | notificationArns | string[] | Returns the list of notification Amazon Resource Names (ARNs) for the current stack. | | partition | string | The partition in which this stack is defined. | | region | string | The AWS region into which this stack will be deployed (e.g. `us-west-2`). | | stackId | string | The ID of the stack. | | stackName | string | The concrete CloudFormation physical stack name. | | synthesizer | aws-cdk-lib.IStackSynthesizer | Synthesis method for this stack. | | tags | aws-cdk-lib.TagManager | Tags to be applied to the stack. | | templateFile | string | The name of the CloudFormation template file emitted to the output directory during synthesis. | | templateOptions | aws-cdk-lib.ITemplateOptions | Options for CloudFormation template (like version, transform, description). | | urlSuffix | string | The Amazon domain suffix for the region in which this stack is defined. | | nestedStackParent | aws-cdk-lib.Stack | If this is a nested stack, returns it's parent stack. | | nestedStackResource | aws-cdk-lib.CfnResource | If this is a nested stack, this represents its `AWS::CloudFormation::Stack` resource. | | terminationProtection | boolean | Whether termination protection is enabled for this stack. | | integrationTestCodeBuildProject | aws-cdk-lib.aws_codebuild.Project | The CodeBuild Project that's going to run the integration test. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `account`^Required ```typescript public readonly account: string; ``` - *Type:* string The AWS account into which this stack will be deployed. This value is resolved according to the following rules: 1. The value provided to `env.account` when the stack is defined. This can either be a concrete account (e.g. `585695031111`) or the `Aws.ACCOUNT_ID` token. 3. `Aws.ACCOUNT_ID`, which represents the CloudFormation intrinsic reference `{ "Ref": "AWS::AccountId" }` encoded as a string token. Preferably, you should use the return value as an opaque string and not attempt to parse it to implement your logic. If you do, you must first check that it is a concrete value an not an unresolved token. If this value is an unresolved token (`Token.isUnresolved(stack.account)` returns `true`), this implies that the user wishes that this stack will synthesize into a **account-agnostic template**. In this case, your code should either fail (throw an error, emit a synth error using `Annotations.of(construct).addError()`) or implement some other region-agnostic behavior. --- ##### `artifactId`^Required ```typescript public readonly artifactId: string; ``` - *Type:* string The ID of the cloud assembly artifact for this stack. --- ##### `availabilityZones`^Required ```typescript public readonly availabilityZones: string[]; ``` - *Type:* string[] Returns the list of AZs that are available in the AWS environment (account/region) associated with this stack. If the stack is environment-agnostic (either account and/or region are tokens), this property will return an array with 2 tokens that will resolve at deploy-time to the first two availability zones returned from CloudFormation's `Fn::GetAZs` intrinsic function. If they are not available in the context, returns a set of dummy values and reports them as missing, and let the CLI resolve them by calling EC2 `DescribeAvailabilityZones` on the target environment. To specify a different strategy for selecting availability zones override this method. --- ##### `bundlingRequired`^Required ```typescript public readonly bundlingRequired: boolean; ``` - *Type:* boolean Indicates whether the stack requires bundling or not. --- ##### `dependencies`^Required ```typescript public readonly dependencies: Stack[]; ``` - *Type:* aws-cdk-lib.Stack[] Return the stacks this stack depends on. --- ##### `environment`^Required ```typescript public readonly environment: string; ``` - *Type:* string The environment coordinates in which this stack is deployed. In the form `aws://account/region`. Use `stack.account` and `stack.region` to obtain the specific values, no need to parse. You can use this value to determine if two stacks are targeting the same environment. If either `stack.account` or `stack.region` are not concrete values (e.g. `Aws.ACCOUNT_ID` or `Aws.REGION`) the special strings `unknown-account` and/or `unknown-region` will be used respectively to indicate this stack is region/account-agnostic. --- ##### `nested`^Required ```typescript public readonly nested: boolean; ``` - *Type:* boolean Indicates if this is a nested stack, in which case `parentStack` will include a reference to it's parent. --- ##### `notificationArns`^Required ```typescript public readonly notificationArns: string[]; ``` - *Type:* string[] Returns the list of notification Amazon Resource Names (ARNs) for the current stack. --- ##### `partition`^Required ```typescript public readonly partition: string; ``` - *Type:* string The partition in which this stack is defined. --- ##### `region`^Required ```typescript public readonly region: string; ``` - *Type:* string The AWS region into which this stack will be deployed (e.g. `us-west-2`). This value is resolved according to the following rules: 1. The value provided to `env.region` when the stack is defined. This can either be a concrete region (e.g. `us-west-2`) or the `Aws.REGION` token. 3. `Aws.REGION`, which is represents the CloudFormation intrinsic reference `{ "Ref": "AWS::Region" }` encoded as a string token. Preferably, you should use the return value as an opaque string and not attempt to parse it to implement your logic. If you do, you must first check that it is a concrete value an not an unresolved token. If this value is an unresolved token (`Token.isUnresolved(stack.region)` returns `true`), this implies that the user wishes that this stack will synthesize into a **region-agnostic template**. In this case, your code should either fail (throw an error, emit a synth error using `Annotations.of(construct).addError()`) or implement some other region-agnostic behavior. --- ##### `stackId`^Required ```typescript public readonly stackId: string; ``` - *Type:* string The ID of the stack. --- *Example* ```typescript // After resolving, looks like 'arn:aws:cloudformation:us-west-2:123456789012:stack/teststack/51af3dc0-da77-11e4-872e-1234567db123' ``` ##### `stackName`^Required ```typescript public readonly stackName: string; ``` - *Type:* string The concrete CloudFormation physical stack name. This is either the name defined explicitly in the `stackName` prop or allocated based on the stack's location in the construct tree. Stacks that are directly defined under the app use their construct `id` as their stack name. Stacks that are defined deeper within the tree will use a hashed naming scheme based on the construct path to ensure uniqueness. If you wish to obtain the deploy-time AWS::StackName intrinsic, you can use `Aws.STACK_NAME` directly. --- ##### `synthesizer`^Required ```typescript public readonly synthesizer: IStackSynthesizer; ``` - *Type:* aws-cdk-lib.IStackSynthesizer Synthesis method for this stack. --- ##### `tags`^Required ```typescript public readonly tags: TagManager; ``` - *Type:* aws-cdk-lib.TagManager Tags to be applied to the stack. --- ##### `templateFile`^Required ```typescript public readonly templateFile: string; ``` - *Type:* string The name of the CloudFormation template file emitted to the output directory during synthesis. Example value: `MyStack.template.json` --- ##### `templateOptions`^Required ```typescript public readonly templateOptions: ITemplateOptions; ``` - *Type:* aws-cdk-lib.ITemplateOptions Options for CloudFormation template (like version, transform, description). --- ##### `urlSuffix`^Required ```typescript public readonly urlSuffix: string; ``` - *Type:* string The Amazon domain suffix for the region in which this stack is defined. --- ##### `nestedStackParent`^Optional ```typescript public readonly nestedStackParent: Stack; ``` - *Type:* aws-cdk-lib.Stack If this is a nested stack, returns it's parent stack. --- ##### `nestedStackResource`^Optional ```typescript public readonly nestedStackResource: CfnResource; ``` - *Type:* aws-cdk-lib.CfnResource If this is a nested stack, this represents its `AWS::CloudFormation::Stack` resource. `undefined` for top-level (non-nested) stacks. --- ##### `terminationProtection`^Required ```typescript public readonly terminationProtection: boolean; ``` - *Type:* boolean Whether termination protection is enabled for this stack. --- ##### `integrationTestCodeBuildProject`^Required ```typescript public readonly integrationTestCodeBuildProject: Project; ``` - *Type:* aws-cdk-lib.aws_codebuild.Project The CodeBuild Project that's going to run the integration test. --- ### KafkaApi A construct to create a Kafka API admin client. > [https://awslabs.github.io/data-solutions-framework-on-aws/](https://awslabs.github.io/data-solutions-framework-on-aws/) #### Initializers ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' new streaming.KafkaApi(scope: Construct, id: string, props: KafkaApiProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.streaming.KafkaApiProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaApiProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | grantConsume | Grant a principal permissions to consume from a topic. | | grantProduce | Grant a principal permissions to produce to a topic. | | retrieveVersion | Retrieve DSF package.json version. | | setAcl | Creates a ACL in the MSK Cluster. | | setTopic | Creates a topic in the MSK Cluster. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `grantConsume` ```typescript public grantConsume(id: string, topicName: string, clientAuthentication: Authentication, principal: string | IPrincipal, host?: string, removalPolicy?: RemovalPolicy, customResourceAuthentication?: Authentication): CustomResource ``` Grant a principal permissions to consume from a topic. ###### `id`^Required - *Type:* string the CDK resource ID. --- ###### `topicName`^Required - *Type:* string the target topic to grant consume permissions on. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication mode of the consumer. --- ###### `principal`^Required - *Type:* string | aws-cdk-lib.aws_iam.IPrincipal the principal receiveing grant consume permissions. --- ###### `host`^Optional - *Type:* string the host of the consumer. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy the removal policy to apply to the grant. --- ###### `customResourceAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `grantProduce` ```typescript public grantProduce(id: string, topicName: string, clientAuthentication: Authentication, principal: string | IPrincipal, host?: string, removalPolicy?: RemovalPolicy, customResourceAuthentication?: Authentication): CustomResource ``` Grant a principal permissions to produce to a topic. ###### `id`^Required - *Type:* string the CDK resource ID. --- ###### `topicName`^Required - *Type:* string the target topic to grant produce permissions on. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication mode of the producer. --- ###### `principal`^Required - *Type:* string | aws-cdk-lib.aws_iam.IPrincipal the principal receiving grant produce permissions. --- ###### `host`^Optional - *Type:* string the host of the producer. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy the removal policy to apply to the grant. --- ###### `customResourceAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. ##### `setAcl` ```typescript public setAcl(id: string, aclDefinition: Acl, removalPolicy?: RemovalPolicy, clientAuthentication?: Authentication): CustomResource ``` Creates a ACL in the MSK Cluster. ###### `id`^Required - *Type:* string the CDK ID of the ACL. --- ###### `aclDefinition`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Acl the Kafka ACL definition. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy Wether to keep the ACL or delete it when removing the resource from the Stack. --- ###### `clientAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `setTopic` ```typescript public setTopic(id: string, clientAuthentication: Authentication, topicDefinition: MskTopic, removalPolicy?: RemovalPolicy, waitForLeaders?: boolean, timeout?: number): CustomResource ``` Creates a topic in the MSK Cluster. ###### `id`^Required - *Type:* string the CDK ID for Topic. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the topic. --- ###### `topicDefinition`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskTopic the Kafka topic definition. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy Wether to keep the topic or delete it when removing the resource from the Stack. --- ###### `waitForLeaders`^Optional - *Type:* boolean If set to true, waits until metadata for the new topics doesn't throw LEADER_NOT_AVAILABLE. --- ###### `timeout`^Optional - *Type:* number The time in ms to wait for a topic to be completely created on the controller node. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.KafkaApi.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | mskAclFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function used by the Custom Resource provider when MSK is using mTLS authentication. | | mskAclLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The Cloudwatch Log Group used by the Custom Resource provider when MSK is using mTLS authentication. | | mskAclRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Custom Resource provider when MSK is using mTLS authentication. | | mskAclSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Custom Resource provider when MSK is using mTLS authentication. | | mskIamFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function used by the Custom Resource provider when MSK is using IAM authentication. | | mskIamLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The Cloudwatch Log Group used by the Custom Resource provider when MSK is using IAM authentication. | | mskIamRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Custom Resource provider when MSK is using IAM authentication. | | mskIamSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Custom Resource provider when MSK is using IAM authentication. | | serviceToken | string | If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `mskAclFunction`^Optional ```typescript public readonly mskAclFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function used by the Custom Resource provider when MSK is using mTLS authentication. --- ##### `mskAclLogGroup`^Optional ```typescript public readonly mskAclLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The Cloudwatch Log Group used by the Custom Resource provider when MSK is using mTLS authentication. --- ##### `mskAclRole`^Optional ```typescript public readonly mskAclRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Custom Resource provider when MSK is using mTLS authentication. --- ##### `mskAclSecurityGroup`^Optional ```typescript public readonly mskAclSecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Custom Resource provider when MSK is using mTLS authentication. --- ##### `mskIamFunction`^Optional ```typescript public readonly mskIamFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function used by the Custom Resource provider when MSK is using IAM authentication. --- ##### `mskIamLogGroup`^Optional ```typescript public readonly mskIamLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The Cloudwatch Log Group used by the Custom Resource provider when MSK is using IAM authentication. --- ##### `mskIamRole`^Optional ```typescript public readonly mskIamRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Custom Resource provider when MSK is using IAM authentication. --- ##### `mskIamSecurityGroup`^Optional ```typescript public readonly mskIamSecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Custom Resource provider when MSK is using IAM authentication. --- ##### `serviceToken`^Optional ```typescript public readonly serviceToken: string; ``` - *Type:* string If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### MskProvisioned A construct to create an MSK Provisioned cluster. > [https://awslabs.github.io/data-solutions-framework-on-aws/](https://awslabs.github.io/data-solutions-framework-on-aws/) *Example* ```typescript const msk = new dsf.streaming.MskProvisioned(this, 'cluster'); ``` #### Initializers ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' new streaming.MskProvisioned(scope: Construct, id: string, props?: MskProvisionedProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.streaming.MskProvisionedProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskProvisionedProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | addClusterPolicy | Add a cluster policy. | | deleteClusterPolicy | *No description.* | | getBootstrapBrokers | Method to get bootstrap broker connection string based on the authentication mode. | | grantConsume | Grant a principal permissions to consume from a topic. | | grantProduce | Grant a principal permissions to produce to a topic. | | putClusterPolicy | *No description.* | | retrieveVersion | Retrieve DSF package.json version. | | setAcl | Creates ACL in the Msk Cluster. | | setTopic | Creates a topic in the Msk Cluster. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `addClusterPolicy` ```typescript public addClusterPolicy(policy: PolicyDocument, id: string): CfnClusterPolicy ``` Add a cluster policy. ###### `policy`^Required - *Type:* aws-cdk-lib.aws_iam.PolicyDocument the IAM principal to grand the consume action. --- ###### `id`^Required - *Type:* string the CDK id for the Cluster Policy. --- ##### `deleteClusterPolicy` ```typescript public deleteClusterPolicy(): void ``` ##### `getBootstrapBrokers` ```typescript public getBootstrapBrokers(authentication: Authentication): string ``` Method to get bootstrap broker connection string based on the authentication mode. ###### `authentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication the authentication mode. --- ##### `grantConsume` ```typescript public grantConsume(id: string, topicName: string, clientAuthentication: Authentication, principal: string | IPrincipal, host?: string, removalPolicy?: RemovalPolicy, customResourceAuthentication?: Authentication): CustomResource ``` Grant a principal permissions to consume from a topic. ###### `id`^Required - *Type:* string the CDK resource ID. --- ###### `topicName`^Required - *Type:* string the target topic to grant consume permissions on. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication mode of the consumer. --- ###### `principal`^Required - *Type:* string | aws-cdk-lib.aws_iam.IPrincipal the principal receiveing grant consume permissions. --- ###### `host`^Optional - *Type:* string the host of the consumer. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy the removal policy to apply to the grant. --- ###### `customResourceAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `grantProduce` ```typescript public grantProduce(id: string, topicName: string, clientAuthentication: Authentication, principal: string | IPrincipal, host?: string, removalPolicy?: RemovalPolicy, customResourceAuthentication?: Authentication): CustomResource ``` Grant a principal permissions to produce to a topic. ###### `id`^Required - *Type:* string the CDK resource ID. --- ###### `topicName`^Required - *Type:* string the target topic to grant produce permissions on. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication mode of the producer. --- ###### `principal`^Required - *Type:* string | aws-cdk-lib.aws_iam.IPrincipal the principal receiving grant produce permissions. --- ###### `host`^Optional - *Type:* string the host of the producer. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy the removal policy to apply to the grant. --- ###### `customResourceAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `putClusterPolicy` ```typescript public putClusterPolicy(policy: string, id: string, currentVersion?: string): void ``` ###### `policy`^Required - *Type:* string --- ###### `id`^Required - *Type:* string --- ###### `currentVersion`^Optional - *Type:* string --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. ##### `setAcl` ```typescript public setAcl(id: string, aclDefinition: Acl, removalPolicy?: RemovalPolicy, clientAuthentication?: Authentication): CustomResource ``` Creates ACL in the Msk Cluster. ###### `id`^Required - *Type:* string the CDK ID of the ACL. --- ###### `aclDefinition`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Acl the Kafka Acl definition. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy Wether to keep the ACL or delete it when removing the resource from the Stack {@default RemovalPolicy.RETAIN}. --- ###### `clientAuthentication`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the ACL. --- ##### `setTopic` ```typescript public setTopic(id: string, clientAuthentication: Authentication, topicDefinition: MskTopic, removalPolicy?: RemovalPolicy, waitForLeaders?: boolean, timeout?: number): CustomResource ``` Creates a topic in the Msk Cluster. ###### `id`^Required - *Type:* string the CDK ID of the Topic. --- ###### `clientAuthentication`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.Authentication The authentication used by the Kafka API admin client to create the topic. --- ###### `topicDefinition`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskTopic the Kafka topic definition. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy Wether to keep the topic or delete it when removing the resource from the Stack {@default RemovalPolicy.RETAIN}. --- ###### `waitForLeaders`^Optional - *Type:* boolean If this is true it will wait until metadata for the new topics doesn't throw LEADER_NOT_AVAILABLE. --- ###### `timeout`^Optional - *Type:* number The time in ms to wait for a topic to be completely created on the controller node. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | createClusterConfiguration | *No description.* | --- ##### `isConstruct` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.MskProvisioned.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `createClusterConfiguration` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.MskProvisioned.createClusterConfiguration(scope: Construct, id: string, name: string, serverPropertiesFilePath: string, kafkaVersions?: KafkaVersion[], configurationDescription?: string, latestRevision?: LatestRevisionProperty) ``` ###### `scope`^Required - *Type:* constructs.Construct --- ###### `id`^Required - *Type:* string --- ###### `name`^Required - *Type:* string --- ###### `serverPropertiesFilePath`^Required - *Type:* string --- ###### `kafkaVersions`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion[] --- ###### `configurationDescription`^Optional - *Type:* string --- ###### `latestRevision`^Optional - *Type:* aws-cdk-lib.aws_msk.CfnConfiguration.LatestRevisionProperty --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | cluster | aws-cdk-lib.aws_msk.CfnCluster | The MSK cluster created by the construct. | | encryptionAtRestKey | aws-cdk-lib.aws_kms.IKey | The KMS CMK key for encrypting data within the cluster. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where the MSK cluster is deployed. | | applyConfigurationFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function responsible for applying MSK configuration. | | applyConfigurationLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Lambda responsible for applying MSK configuration. | | applyConfigurationRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Lambda responsible for applying MSK configuration. | | applyConfigurationSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Lambda responsible for applying MSK configuration. | | brokerLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch log group associated with brokers activity. | | brokerSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The security group associated with the MSK brokers. | | clusterConfiguration | aws-cdk-lib.aws_msk.CfnConfiguration | The MSK cluster configuration. | | iamCrudAdminFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function responsible for CRUD operations via IAM authentication. | | iamCrudAdminLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Lambda responsible for CRUD operations via IAM authentication. | | iamCrudAdminRole | aws-cdk-lib.aws_iam.IRole | The IAM role used by the Lambda responsible for CRUD operations via IAM authentication. | | iamCrudAdminSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Lambda responsible for CRUD operations via IAM authentication. | | inClusterAclFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function responsible for CRUD operations via mTLS authentication. | | inClusterAclLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Lambda responsible for CRUD operations via mTLS authentication. | | inClusterAclRole | aws-cdk-lib.aws_iam.IRole | The IAM role used by the Lambda responsible for CRUD operations via mTLS authentication. | | inClusterAclSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Lambda responsible for CRUD operations via mTLS authentication. | | serviceToken | string | If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. | | updateConnectivityFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function responsible for updating MSK Connectivity. | | updateConnectivityLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Lambda responsible for updating MSK Connectivity. | | updateConnectivityRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Lambda responsible for updating MSK Connectivity. | | updateConnectivitySecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The Security Group used by the Lambda responsible for updating MSK Connectivity. | | updateZookepeerFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function responsible for updating Zookeeper. | | updateZookepeerLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the Lambda responsible for updating Zookeeper. | | updateZookepeerRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Lambda responsible for updating Zookeeper. | | updateZookepeerSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | THe Security Group associated to the Lambda responsible for updating Zookeeper. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `cluster`^Required ```typescript public readonly cluster: CfnCluster; ``` - *Type:* aws-cdk-lib.aws_msk.CfnCluster The MSK cluster created by the construct. --- ##### `encryptionAtRestKey`^Required ```typescript public readonly encryptionAtRestKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS CMK key for encrypting data within the cluster. --- ##### `vpc`^Required ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc The VPC where the MSK cluster is deployed. --- ##### `applyConfigurationFunction`^Optional ```typescript public readonly applyConfigurationFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function responsible for applying MSK configuration. --- ##### `applyConfigurationLogGroup`^Optional ```typescript public readonly applyConfigurationLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Lambda responsible for applying MSK configuration. --- ##### `applyConfigurationRole`^Optional ```typescript public readonly applyConfigurationRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Lambda responsible for applying MSK configuration. --- ##### `applyConfigurationSecurityGroup`^Optional ```typescript public readonly applyConfigurationSecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Lambda responsible for applying MSK configuration. --- ##### `brokerLogGroup`^Optional ```typescript public readonly brokerLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch log group associated with brokers activity. --- ##### `brokerSecurityGroup`^Optional ```typescript public readonly brokerSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The security group associated with the MSK brokers. --- ##### `clusterConfiguration`^Optional ```typescript public readonly clusterConfiguration: CfnConfiguration; ``` - *Type:* aws-cdk-lib.aws_msk.CfnConfiguration The MSK cluster configuration. --- ##### `iamCrudAdminFunction`^Optional ```typescript public readonly iamCrudAdminFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function responsible for CRUD operations via IAM authentication. --- ##### `iamCrudAdminLogGroup`^Optional ```typescript public readonly iamCrudAdminLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Lambda responsible for CRUD operations via IAM authentication. --- ##### `iamCrudAdminRole`^Optional ```typescript public readonly iamCrudAdminRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role used by the Lambda responsible for CRUD operations via IAM authentication. --- ##### `iamCrudAdminSecurityGroup`^Optional ```typescript public readonly iamCrudAdminSecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Lambda responsible for CRUD operations via IAM authentication. --- ##### `inClusterAclFunction`^Optional ```typescript public readonly inClusterAclFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function responsible for CRUD operations via mTLS authentication. --- ##### `inClusterAclLogGroup`^Optional ```typescript public readonly inClusterAclLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Lambda responsible for CRUD operations via mTLS authentication. --- ##### `inClusterAclRole`^Optional ```typescript public readonly inClusterAclRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role used by the Lambda responsible for CRUD operations via mTLS authentication. --- ##### `inClusterAclSecurityGroup`^Optional ```typescript public readonly inClusterAclSecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Lambda responsible for CRUD operations via mTLS authentication. --- ##### `serviceToken`^Optional ```typescript public readonly serviceToken: string; ``` - *Type:* string If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. --- ##### `updateConnectivityFunction`^Optional ```typescript public readonly updateConnectivityFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function responsible for updating MSK Connectivity. --- ##### `updateConnectivityLogGroup`^Optional ```typescript public readonly updateConnectivityLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Lambda responsible for updating MSK Connectivity. --- ##### `updateConnectivityRole`^Optional ```typescript public readonly updateConnectivityRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Lambda responsible for updating MSK Connectivity. --- ##### `updateConnectivitySecurityGroup`^Optional ```typescript public readonly updateConnectivitySecurityGroup: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The Security Group used by the Lambda responsible for updating MSK Connectivity. --- ##### `updateZookepeerFunction`^Optional ```typescript public readonly updateZookepeerFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function responsible for updating Zookeeper. --- ##### `updateZookepeerLogGroup`^Optional ```typescript public readonly updateZookepeerLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the Lambda responsible for updating Zookeeper. --- ##### `updateZookepeerRole`^Optional ```typescript public readonly updateZookepeerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the Lambda responsible for updating Zookeeper. --- ##### `updateZookepeerSecurityGroup`^Optional ```typescript public readonly updateZookepeerSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup THe Security Group associated to the Lambda responsible for updating Zookeeper. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | | MSK_DEFAULT_VERSION | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ##### `MSK_DEFAULT_VERSION`^Required ```typescript public readonly MSK_DEFAULT_VERSION: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion --- ### MskServerless A construct to create an MSK Serverless cluster. > [https://awslabs.github.io/data-solutions-framework-on-aws/](https://awslabs.github.io/data-solutions-framework-on-aws/) *Example* ```typescript const msk = new dsf.streaming.MskServerless(this, 'cluster'); ``` #### Initializers ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' new streaming.MskServerless(scope: Construct, id: string, props?: MskServerlessProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.streaming.MskServerlessProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Optional - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskServerlessProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | addClusterPolicy | Add a cluster policy. | | addTopic | Creates a topic in the MSK Serverless. | | grantConsume | Grant a principal the right to consume data from a topic. | | grantProduce | Grant a principal to produce data to a topic. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `addClusterPolicy` ```typescript public addClusterPolicy(policy: PolicyDocument, id: string): CfnClusterPolicy ``` Add a cluster policy. ###### `policy`^Required - *Type:* aws-cdk-lib.aws_iam.PolicyDocument the IAM principal to grand the consume action. --- ###### `id`^Required - *Type:* string the CDK id for the Cluster Policy. --- ##### `addTopic` ```typescript public addTopic(id: string, topicDefinition: MskTopic, removalPolicy?: RemovalPolicy, waitForLeaders?: boolean, timeout?: number): CustomResource ``` Creates a topic in the MSK Serverless. ###### `id`^Required - *Type:* string the CDK id for the topic. --- ###### `topicDefinition`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskTopic the Kafka topic definition. --- ###### `removalPolicy`^Optional - *Type:* aws-cdk-lib.RemovalPolicy Wether to keep the topic or delete it when removing the resource from the Stack. --- ###### `waitForLeaders`^Optional - *Type:* boolean Wait until metadata for the new topics doesn't throw LEADER_NOT_AVAILABLE. --- ###### `timeout`^Optional - *Type:* number The time in ms to wait for a topic to be completely created on the controller node. --- ##### `grantConsume` ```typescript public grantConsume(topicName: string, principal: IPrincipal): CustomResource ``` Grant a principal the right to consume data from a topic. ###### `topicName`^Required - *Type:* string the topic to which the principal can consume data from. --- ###### `principal`^Required - *Type:* aws-cdk-lib.aws_iam.IPrincipal the IAM principal to grand the consume action. --- ##### `grantProduce` ```typescript public grantProduce(topicName: string, principal: IPrincipal): CustomResource ``` Grant a principal to produce data to a topic. ###### `topicName`^Required - *Type:* string the name of the topic to grant producer permissions. --- ###### `principal`^Required - *Type:* aws-cdk-lib.aws_iam.IPrincipal the IAM principal to grand producer permissions. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.MskServerless.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | cluster | aws-cdk-lib.aws_msk.CfnServerlessCluster | The MSK cluster as a CloudFormation resource. | | clusterBoostrapBrokers | string | The list of bootstrap servers for client to connect. | | clusterName | string | The name of the cluster. | | lambdaSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The security group used by the configuration Lambda. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where the MSK cluster is deployed. | | brokerSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The security group used by the cluster. | | serviceToken | string | If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `cluster`^Required ```typescript public readonly cluster: CfnServerlessCluster; ``` - *Type:* aws-cdk-lib.aws_msk.CfnServerlessCluster The MSK cluster as a CloudFormation resource. --- ##### `clusterBoostrapBrokers`^Required ```typescript public readonly clusterBoostrapBrokers: string; ``` - *Type:* string The list of bootstrap servers for client to connect. --- ##### `clusterName`^Required ```typescript public readonly clusterName: string; ``` - *Type:* string The name of the cluster. --- ##### `lambdaSecurityGroup`^Required ```typescript public readonly lambdaSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The security group used by the configuration Lambda. --- ##### `vpc`^Required ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc The VPC where the MSK cluster is deployed. --- ##### `brokerSecurityGroup`^Optional ```typescript public readonly brokerSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The security group used by the cluster. --- ##### `serviceToken`^Optional ```typescript public readonly serviceToken: string; ``` - *Type:* string If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### OpenSearchApi A construct to create an OpenSearch API client. *Example* ```typescript const domainEndpoint='search-XXXXXX.XXXXXX.es.amazonaws.com'; const apiRole = cdk.iam.Role.fromRoleName(this, 'ApiRole', ''); const osApi = new dsf.consumption.OpenSearchApi(this, 'MyOpenSearchApi',{ iamHandlerRole:apiRole, openSearchEndpoint:domainEndpoint, openSearchClusterType:dsf.consumption.OpenSearchClusterType.PROVISIONED, removalPolicy:cdk.RemovalPolicy.DESTROY }); const roleMap = osApi.addRoleMapping('DashBoardUser', 'dashboards_user',''); const add1Cr = osApi.callOpenSearchApi('AddData1', 'movies-01/_doc/1111',{"title": "Rush", "year": 2013}, 'PUT'); add1Cr.node.addDependency(roleMap); const add2Cr = osApi.callOpenSearchApi('AddData3', 'movies-01/_doc/2222',{"title": "Toy Story", "year": 2014}, 'PUT'); add2Cr.node.addDependency(roleMap); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.OpenSearchApi(scope: Construct, id: string, props: OpenSearchApiProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | Construct. | | id | string | unique ID for the construct. | | props | @cdklabs/aws-data-solutions-framework.consumption.OpenSearchApiProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct Construct. --- ##### `id`^Required - *Type:* string unique ID for the construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.OpenSearchApiProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | addRoleMapping | *No description.* | | callOpenSearchApi | Calls OpenSearch API using custom resource. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `addRoleMapping` ```typescript public addRoleMapping(id: string, name: string, role: string, persist?: boolean): CustomResource ``` > [https://opensearch.org/docs/2.9/security/access-control/users-roles/#predefined-roles](https://opensearch.org/docs/2.9/security/access-control/users-roles/#predefined-roles) ###### `id`^Required - *Type:* string The CDK resource ID. --- ###### `name`^Required - *Type:* string OpenSearch role name. --- ###### `role`^Required - *Type:* string list of IAM roles. For IAM Identity center provide SAML group Id as a role --- ###### `persist`^Optional - *Type:* boolean Set to true if you want to prevent the roles to be ovewritten by subsequent PUT API calls. Default false. --- ##### `callOpenSearchApi` ```typescript public callOpenSearchApi(id: string, apiPath: string, body: any, method?: string): CustomResource ``` Calls OpenSearch API using custom resource. ###### `id`^Required - *Type:* string The CDK resource ID. --- ###### `apiPath`^Required - *Type:* string OpenSearch API path. --- ###### `body`^Required - *Type:* any OpenSearch API request body. --- ###### `method`^Optional - *Type:* string Opensearch API method,. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.OpenSearchApi.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | apiFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function used for the onEventHandler. | | apiLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the onEventHandler Lambda Function. | | apiRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the onEventHandler Lambba Function. | | apiSecurityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The list of EC2 Security Groups used by the Lambda Functions. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `apiFunction`^Required ```typescript public readonly apiFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function used for the onEventHandler. --- ##### `apiLogGroup`^Required ```typescript public readonly apiLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the onEventHandler Lambda Function. --- ##### `apiRole`^Required ```typescript public readonly apiRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the onEventHandler Lambba Function. --- ##### `apiSecurityGroups`^Optional ```typescript public readonly apiSecurityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The list of EC2 Security Groups used by the Lambda Functions. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### OpenSearchCluster A construct to provision Amazon OpenSearch Cluster and OpenSearch Dashboards. Uses IAM Identity Center SAML authentication. If OpenSearch cluster is deployed in vpc created using DataVpc construct, ClientVPNEndpoint will be provisioned automatically for secure access to OpenSearch Dashboards. *Example* ```typescript const osCluster = new dsf.consumption.OpenSearchCluster(this, 'MyOpenSearchCluster',{ domainName:"mycluster1", samlEntityId:'', samlMetadataContent:'', samlMasterBackendRole:'', deployInVpc:true, removalPolicy:cdk.RemovalPolicy.DESTROY }); osCluster.addRoleMapping('DashBoardUser', 'dashboards_user',''); osCluster.addRoleMapping('ReadAllRole', 'readall',''); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.OpenSearchCluster(scope: Construct, id: string, props: OpenSearchClusterProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the AWS CDK Construct. | | id | string | the ID of the AWS CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.consumption.OpenSearchClusterProps | the OpenSearchCluster [properties]{@link OpenSearchClusterProps}. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the AWS CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the AWS CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.OpenSearchClusterProps the OpenSearchCluster [properties]{@link OpenSearchClusterProps}. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | addRoleMapping | *No description.* | | callOpenSearchApi | Calls OpenSearch API using custom resource. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `addRoleMapping` ```typescript public addRoleMapping(id: string, name: string, role: string, persist?: boolean): CustomResource ``` > [https://opensearch.org/docs/2.9/security/access-control/users-roles/#predefined-roles](https://opensearch.org/docs/2.9/security/access-control/users-roles/#predefined-roles) ###### `id`^Required - *Type:* string The CDK resource ID. --- ###### `name`^Required - *Type:* string OpenSearch role name. --- ###### `role`^Required - *Type:* string list of IAM roles. For IAM Identity center provide SAML group Id as a role --- ###### `persist`^Optional - *Type:* boolean Set to true if you want to prevent the roles to be ovewritten by subsequent PUT API calls. Default false. --- ##### `callOpenSearchApi` ```typescript public callOpenSearchApi(id: string, apiPath: string, body: any, method?: string): CustomResource ``` Calls OpenSearch API using custom resource. ###### `id`^Required - *Type:* string The CDK resource ID. --- ###### `apiPath`^Required - *Type:* string OpenSearch API path. --- ###### `body`^Required - *Type:* any OpenSearch API request body. --- ###### `method`^Optional - *Type:* string Opensearch API method,. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.OpenSearchCluster.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | domain | aws-cdk-lib.aws_opensearchservice.IDomain | OpenSearchCluster domain. | | encryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt data and logs. | | logGroup | aws-cdk-lib.aws_logs.ILogGroup | CloudWatch Logs Log Group to store OpenSearch cluster logs. | | masterRole | aws-cdk-lib.aws_iam.IRole | IAM Role used to provision and configure OpenSearch domain. | | openSearchApiFunction | aws-cdk-lib.aws_lambda.IFunction | Lambda Function used for OpenSearch API. | | openSearchApiLogGroup | aws-cdk-lib.aws_logs.ILogGroup | CloudWatch Logs Log Group to store OpenSearch API logs. | | apiSecurityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | Security Group for OpenSearch API Lambda Function. | | vpc | aws-cdk-lib.aws_ec2.IVpc | VPC OpenSearch cluster is provisioned in. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `domain`^Required ```typescript public readonly domain: IDomain; ``` - *Type:* aws-cdk-lib.aws_opensearchservice.IDomain OpenSearchCluster domain. --- ##### `encryptionKey`^Required ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used to encrypt data and logs. --- ##### `logGroup`^Required ```typescript public readonly logGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup CloudWatch Logs Log Group to store OpenSearch cluster logs. --- ##### `masterRole`^Required ```typescript public readonly masterRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole IAM Role used to provision and configure OpenSearch domain. --- ##### `openSearchApiFunction`^Required ```typescript public readonly openSearchApiFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction Lambda Function used for OpenSearch API. > [OpenSearchApi](OpenSearchApi) --- ##### `openSearchApiLogGroup`^Required ```typescript public readonly openSearchApiLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup CloudWatch Logs Log Group to store OpenSearch API logs. --- ##### `apiSecurityGroups`^Optional ```typescript public readonly apiSecurityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] Security Group for OpenSearch API Lambda Function. > [OpenSearchApi](OpenSearchApi) --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc VPC OpenSearch cluster is provisioned in. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### PySparkApplicationPackage A construct that takes your PySpark application, packages its virtual environment and uploads it along its entrypoint to an Amazon S3 bucket This construct requires Docker daemon installed locally to run. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/pyspark-application-package](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/pyspark-application-package) *Example* ```typescript let pysparkPacker = new dsf.processing.PySparkApplicationPackage (this, 'pysparkPacker', { applicationName: 'my-pyspark', entrypointPath: '/Users/my-user/my-spark-job/app/app-pyspark.py', dependenciesFolder: '/Users/my-user/my-spark-job/app', removalPolicy: cdk.RemovalPolicy.DESTROY, }); ``` #### Initializers ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' new processing.PySparkApplicationPackage(scope: Construct, id: string, props: PySparkApplicationPackageProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.processing.PySparkApplicationPackageProps | {@link PySparkApplicationPackageProps}. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.processing.PySparkApplicationPackageProps {@link PySparkApplicationPackageProps}. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.PySparkApplicationPackage.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | artifactsBucket | aws-cdk-lib.aws_s3.IBucket | The S3 Bucket for storing the artifacts (entrypoint and virtual environment archive). | | assetUploadManagedPolicy | aws-cdk-lib.aws_iam.IManagedPolicy | The IAM Managed Policy used by the custom resource for the assets deployment. | | assetUploadRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the BucketDeployment to upload the artifacts to an s3 bucket. | | entrypointUri | string | The location (generally it's an S3 URI) where the entry point is saved. | | artifactsAccessLogsBucket | @cdklabs/aws-data-solutions-framework.storage.AccessLogsBucket | The access logs bucket to log accesses on the artifacts bucket. | | sparkVenvConf | string | The Spark Config containing the configuration of virtual environment archive with all dependencies. | | venvArchiveUri | string | The location (generally an S3 URI) where the archive of the Python virtual environment with all dependencies is stored. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `artifactsBucket`^Required ```typescript public readonly artifactsBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The S3 Bucket for storing the artifacts (entrypoint and virtual environment archive). --- ##### `assetUploadManagedPolicy`^Required ```typescript public readonly assetUploadManagedPolicy: IManagedPolicy; ``` - *Type:* aws-cdk-lib.aws_iam.IManagedPolicy The IAM Managed Policy used by the custom resource for the assets deployment. --- ##### `assetUploadRole`^Required ```typescript public readonly assetUploadRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the BucketDeployment to upload the artifacts to an s3 bucket. In case you provide your own S3 Bucket for storing the artifacts (entrypoint and virtual environment archive), you must provide S3 write access to this role to upload the artifacts. --- ##### `entrypointUri`^Required ```typescript public readonly entrypointUri: string; ``` - *Type:* string The location (generally it's an S3 URI) where the entry point is saved. You can pass this location to your Spark job. --- ##### `artifactsAccessLogsBucket`^Optional ```typescript public readonly artifactsAccessLogsBucket: AccessLogsBucket; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.AccessLogsBucket The access logs bucket to log accesses on the artifacts bucket. --- ##### `sparkVenvConf`^Optional ```typescript public readonly sparkVenvConf: string; ``` - *Type:* string The Spark Config containing the configuration of virtual environment archive with all dependencies. --- ##### `venvArchiveUri`^Optional ```typescript public readonly venvArchiveUri: string; ``` - *Type:* string The location (generally an S3 URI) where the archive of the Python virtual environment with all dependencies is stored. You can pass this location to your Spark job. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | ARTIFACTS_PREFIX | string | The prefix used to store artifacts on the artifact bucket. | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `ARTIFACTS_PREFIX`^Required ```typescript public readonly ARTIFACTS_PREFIX: string; ``` - *Type:* string The prefix used to store artifacts on the artifact bucket. --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### RedshiftData Creates an asynchronous custom resource that handles the execution of SQL using Redshift's Data API. If `vpc` and `vpcSubnets` are passed, this construct would also create the Redshift Data Interface VPC endpoint and configure the custom resource in the same VPC subnet. *Example* ```typescript const namespace = new dsf.consumption.RedshiftServerlessNamespace(this, 'RedshiftNamespace', { name: "default", dbName: 'defaultdb', }); const workgroup = new dsf.consumption.RedshiftServerlessWorkgroup(this, "RedshiftWorkgroup", { name: "redshift-workgroup", namespace: namespace, }); const rsData = workgroup.accessData('DataApi'); rsData.createDbRole("EngineeringRole", "defaultdb", "engineering"); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.RedshiftData(scope: Construct, id: string, props: RedshiftDataProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | | assignDbRolesToIAMRole | Assigns Redshift DB roles to IAM role vs the `RedshiftDbRoles` tag. | | createDbRole | Creates a new DB role. | | grantDbAllPrivilegesToRole | Grants both read and write permissions on all the tables in the `schema` to the DB role. | | grantDbSchemaToRole | Grants access to the schema to the DB role. | | grantSchemaReadToRole | Grants read permission on all the tables in the `schema` to the DB role. | | ingestData | Ingest data from S3 into a Redshift table. | | mergeToTargetTable | Run the `MERGE` query using simplified mode. | | runCustomSQL | Runs a custom SQL. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. ##### `assignDbRolesToIAMRole` ```typescript public assignDbRolesToIAMRole(dbRoles: string[], targetRole: IRole): void ``` Assigns Redshift DB roles to IAM role vs the `RedshiftDbRoles` tag. ###### `dbRoles`^Required - *Type:* string[] List of Redshift DB roles to assign to IAM role. --- ###### `targetRole`^Required - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role to assign the Redshift DB roles to. --- ##### `createDbRole` ```typescript public createDbRole(id: string, databaseName: string, roleName: string): CustomResource ``` Creates a new DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `roleName`^Required - *Type:* string The name of the role to create. --- ##### `grantDbAllPrivilegesToRole` ```typescript public grantDbAllPrivilegesToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants both read and write permissions on all the tables in the `schema` to the DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `grantDbSchemaToRole` ```typescript public grantDbSchemaToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants access to the schema to the DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `grantSchemaReadToRole` ```typescript public grantSchemaReadToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants read permission on all the tables in the `schema` to the DB role. ###### `id`^Required - *Type:* string --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `ingestData` ```typescript public ingestData(id: string, databaseName: string, targetTable: string, sourceBucket: IBucket, sourcePrefix: string, ingestAdditionalOptions?: string, role?: IRole): CustomResource ``` Ingest data from S3 into a Redshift table. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `targetTable`^Required - *Type:* string The target table to load the data into. --- ###### `sourceBucket`^Required - *Type:* aws-cdk-lib.aws_s3.IBucket The bucket where the source data would be coming from. --- ###### `sourcePrefix`^Required - *Type:* string The location inside the bucket where the data would be ingested from. --- ###### `ingestAdditionalOptions`^Optional - *Type:* string Optional. Additional options to pass to the `COPY` command. For example, `delimiter '|'` or `ignoreheader 1` --- ###### `role`^Optional - *Type:* aws-cdk-lib.aws_iam.IRole Optional. The IAM Role to use to access the data in S3. If not provided, it would use the default IAM role configured in the Redshift Namespace --- ##### `mergeToTargetTable` ```typescript public mergeToTargetTable(id: string, databaseName: string, sourceTable: string, targetTable: string, sourceColumnId?: string, targetColumnId?: string): CustomResource ``` Run the `MERGE` query using simplified mode. This command would do an upsert into the target table. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `sourceTable`^Required - *Type:* string The source table name. Schema can also be included using the following format: `schemaName.tableName` --- ###### `targetTable`^Required - *Type:* string The target table name. Schema can also be included using the following format: `schemaName.tableName` --- ###### `sourceColumnId`^Optional - *Type:* string The column in the source table that's used to determine whether the rows in the `sourceTable` can be matched with rows in the `targetTable`. Default is `id` --- ###### `targetColumnId`^Optional - *Type:* string The column in the target table that's used to determine whether the rows in the `sourceTable` can be matched with rows in the `targetTable`. Default is `id` --- ##### `runCustomSQL` ```typescript public runCustomSQL(id: string, databaseName: string, sql: string, deleteSql?: string): CustomResource ``` Runs a custom SQL. Once the custom resource finishes execution, the attribute `Data` contains an attribute `execId` which contains the Redshift Data API execution ID. You can then use this to retrieve execution results via the `GetStatementResult` API. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `sql`^Required - *Type:* string The sql to run. --- ###### `deleteSql`^Optional - *Type:* string Optional. The sql to run when this resource gets deleted --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.RedshiftData.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | customResourceSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the Custom Resource when deployed in a VPC. | | vpcEndpoint | aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint | The created Redshift Data API interface vpc endpoint when deployed in a VPC. | | vpcEndpointSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the VPC Endpoint when deployed in a VPC. | | dataAccessTargetProps | @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataAccessTargetProps | Contains normalized details of the target Redshift cluster/workgroup for data access. | | executionRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the Redshift Data API execution. | | statusFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the Redshift Data API status checks. | | statusLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data API status checks. | | submitFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the Redshift Data submission. | | submitLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data API submission. | | taggingManagedPolicy | aws-cdk-lib.aws_iam.IManagedPolicy | The managed IAM policy allowing IAM Role to retrieve tag information. | | cleanUpFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function for the S3 data copy cleaning up lambda. | | cleanUpLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data cleaning up lambda. | | cleanUpRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the the S3 data copy cleaning up lambda. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `customResourceSecurityGroup`^Optional ```typescript public readonly customResourceSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the Custom Resource when deployed in a VPC. --- ##### `vpcEndpoint`^Optional ```typescript public readonly vpcEndpoint: IInterfaceVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint The created Redshift Data API interface vpc endpoint when deployed in a VPC. --- ##### `vpcEndpointSecurityGroup`^Optional ```typescript public readonly vpcEndpointSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the VPC Endpoint when deployed in a VPC. --- ##### `dataAccessTargetProps`^Required ```typescript public readonly dataAccessTargetProps: RedshiftDataAccessTargetProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataAccessTargetProps Contains normalized details of the target Redshift cluster/workgroup for data access. --- ##### `executionRole`^Required ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the Redshift Data API execution. --- ##### `statusFunction`^Required ```typescript public readonly statusFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the Redshift Data API status checks. --- ##### `statusLogGroup`^Required ```typescript public readonly statusLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data API status checks. --- ##### `submitFunction`^Required ```typescript public readonly submitFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the Redshift Data submission. --- ##### `submitLogGroup`^Required ```typescript public readonly submitLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data API submission. --- ##### `taggingManagedPolicy`^Required ```typescript public readonly taggingManagedPolicy: IManagedPolicy; ``` - *Type:* aws-cdk-lib.aws_iam.IManagedPolicy The managed IAM policy allowing IAM Role to retrieve tag information. --- ##### `cleanUpFunction`^Optional ```typescript public readonly cleanUpFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function for the S3 data copy cleaning up lambda. --- ##### `cleanUpLogGroup`^Optional ```typescript public readonly cleanUpLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data cleaning up lambda. --- ##### `cleanUpRole`^Optional ```typescript public readonly cleanUpRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the the S3 data copy cleaning up lambda. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### RedshiftDataSharing Creates an asynchronous custom resource to manage the data sharing lifecycle for both data producers and data consumers. This also covers both same account and cross account access. *Example* ```typescript const redshiftAdminSecret = Secret.fromSecretPartialArn(this, 'RedshiftAdminCredentials', 'arn:aws:secretsmanager:us-east-1:XXXXXXXX:secret:YYYYYYYY'); const redshiftVpc = Vpc.fromLookup(this, 'RedshiftVpc', { vpcId: 'XXXXXXXX', }); const dataAccess = new dsf.consumption.RedshiftData(this, 'RedshiftDataAccess', { workgroupId: 'XXXXXXXXXXXXXXX', secret: redshiftAdminSecret, vpc: redshiftVpc, subnets: redshiftVpc.selectSubnets({ subnetGroupName: 'YYYYYYYY' }), createInterfaceVpcEndpoint: true, executionTimeout: Duration.minutes(10), }); const dataShare = new dsf.consumption.RedshiftDataSharing(this, 'RedshiftDataShare', { redshiftData: dataAccess, workgroupId: 'XXXXXXXXXXXXXXX', secret: redshiftAdminSecret, vpc: redshiftVpc, subnets: redshiftVpc.selectSubnets({ subnetGroupName: 'YYYYYYYY' }), createInterfaceVpcEndpoint: true, executionTimeout: Duration.minutes(10), }); const share = dataShare.createShare('ProducerShare', 'default', 'example_share', 'public', ['public.customers']); const grantToConsumer = dataShare.grant('GrantToConsumer', { dataShareName: 'example_share', databaseName: 'default', autoAuthorized: true, accountId: "", dataShareArn: '', }); dataShare.createDatabaseFromShare('ProducerShare', { consumerNamespaceArn: '', newDatabaseName: 'db_from_share', databaseName: 'default', dataShareName: 'example_share', dataShareArn: '', accountId: "", }); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.RedshiftDataSharing(scope: Construct, id: string, props: RedshiftDataSharingProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataSharingProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataSharingProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | | createDatabaseFromShare | Consume datashare by creating a new database pointing to the share. | | createShare | Create a new datashare. | | grant | Create a datashare grant to a namespace if it's in the same account, or to another account. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. ##### `createDatabaseFromShare` ```typescript public createDatabaseFromShare(id: string, props: RedshiftDataSharingCreateDbProps): RedshiftDataSharingCreateDbFromShareProps ``` Consume datashare by creating a new database pointing to the share. If datashare is coming from a different account, setting `autoAssociate` to true automatically associates the datashare to the cluster before the new database is created. ###### `id`^Required - *Type:* string the CDK ID of the resource. --- ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataSharingCreateDbProps `RedshiftDataSharingCreateDbProps`. --- ##### `createShare` ```typescript public createShare(id: string, databaseName: string, dataShareName: string, schema: string, tables: string[]): RedshiftNewShareProps ``` Create a new datashare. ###### `id`^Required - *Type:* string the CDK ID of the resource. --- ###### `databaseName`^Required - *Type:* string The name of the database to connect to. --- ###### `dataShareName`^Required - *Type:* string The name of the datashare. --- ###### `schema`^Required - *Type:* string The schema to add in the datashare. --- ###### `tables`^Required - *Type:* string[] The list of tables that would be included in the datashare. This must follow the format: `.` --- ##### `grant` ```typescript public grant(id: string, props: RedshiftDataSharingGrantProps): RedshiftDataSharingGrantedProps ``` Create a datashare grant to a namespace if it's in the same account, or to another account. ###### `id`^Required - *Type:* string the CDK ID of the resource. --- ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataSharingGrantProps `RedshiftDataSharingGrantProps`. --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.RedshiftDataSharing.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | customResourceSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the Custom Resource when deployed in a VPC. | | vpcEndpoint | aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint | The created Redshift Data API interface vpc endpoint when deployed in a VPC. | | vpcEndpointSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The Security Group used by the VPC Endpoint when deployed in a VPC. | | dataAccessTargetProps | @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataAccessTargetProps | Contains normalized details of the target Redshift cluster/workgroup for data access. | | executionRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the Redshift Data API execution. | | statusFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the Redshift Data Sharing status checks. | | statusLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data Sharing status checks. | | submitFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the Redshift Data Sharing submission. | | submitLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data Sharing submission. | | cleanUpFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function for the cleaning up lambda. | | cleanUpLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the Redshift Data Sharing cleaning up lambda. | | cleanUpRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the the cleaning up lambda. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `customResourceSecurityGroup`^Optional ```typescript public readonly customResourceSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the Custom Resource when deployed in a VPC. --- ##### `vpcEndpoint`^Optional ```typescript public readonly vpcEndpoint: IInterfaceVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint The created Redshift Data API interface vpc endpoint when deployed in a VPC. --- ##### `vpcEndpointSecurityGroup`^Optional ```typescript public readonly vpcEndpointSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The Security Group used by the VPC Endpoint when deployed in a VPC. --- ##### `dataAccessTargetProps`^Required ```typescript public readonly dataAccessTargetProps: RedshiftDataAccessTargetProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftDataAccessTargetProps Contains normalized details of the target Redshift cluster/workgroup for data access. --- ##### `executionRole`^Required ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the Redshift Data API execution. --- ##### `statusFunction`^Required ```typescript public readonly statusFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the Redshift Data Sharing status checks. --- ##### `statusLogGroup`^Required ```typescript public readonly statusLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data Sharing status checks. --- ##### `submitFunction`^Required ```typescript public readonly submitFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the Redshift Data Sharing submission. --- ##### `submitLogGroup`^Required ```typescript public readonly submitLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data Sharing submission. --- ##### `cleanUpFunction`^Optional ```typescript public readonly cleanUpFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function for the cleaning up lambda. --- ##### `cleanUpLogGroup`^Optional ```typescript public readonly cleanUpLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the Redshift Data Sharing cleaning up lambda. --- ##### `cleanUpRole`^Optional ```typescript public readonly cleanUpRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the the cleaning up lambda. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### RedshiftServerlessNamespace Create a Redshift Serverless Namespace with the admin credentials stored in Secrets Manager. *Example* ```typescript const namespace = new dsf.consumption.RedshiftServerlessNamespace(this, 'DefaultServerlessNamespace', { dbName: 'defaultdb', name: 'default' }); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.RedshiftServerlessNamespace(scope: Construct, id: string, props: RedshiftServerlessNamespaceProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespaceProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespaceProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.RedshiftServerlessNamespace.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | adminSecret | aws-cdk-lib.aws_secretsmanager.ISecret | The created Secrets Manager secret containing the admin credentials. | | adminSecretKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt the admin credentials secret. | | createFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the Redshift Serverless creation. | | createLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Logs Log Group for the Redshift Serverless creation. | | createRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the Redshift Serverless creation. | | customResource | aws-cdk-lib.CustomResource | The custom resource that creates the Namespace. | | dataKey | aws-cdk-lib.aws_kms.Key | KMS key used by the namespace to encrypt the data. | | dbName | string | The name of the database. | | namespaceArn | string | The ARN of the created namespace. | | namespaceId | string | The ID of the created namespace. | | namespaceName | string | The name of the created namespace. | | roles | {[ key: string ]: aws-cdk-lib.aws_iam.IRole} | The roles attached to the namespace in the form of `{RoleArn: IRole}`. | | statusFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the creation status check. | | statusLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Logs Log Group for the creation status check. | | statusRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the creation status check. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `adminSecret`^Required ```typescript public readonly adminSecret: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret The created Secrets Manager secret containing the admin credentials. --- ##### `adminSecretKey`^Required ```typescript public readonly adminSecretKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used to encrypt the admin credentials secret. --- ##### `createFunction`^Required ```typescript public readonly createFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the Redshift Serverless creation. --- ##### `createLogGroup`^Required ```typescript public readonly createLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Logs Log Group for the Redshift Serverless creation. --- ##### `createRole`^Required ```typescript public readonly createRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the Redshift Serverless creation. --- ##### `customResource`^Required ```typescript public readonly customResource: CustomResource; ``` - *Type:* aws-cdk-lib.CustomResource The custom resource that creates the Namespace. --- ##### `dataKey`^Required ```typescript public readonly dataKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key KMS key used by the namespace to encrypt the data. --- ##### `dbName`^Required ```typescript public readonly dbName: string; ``` - *Type:* string The name of the database. --- ##### `namespaceArn`^Required ```typescript public readonly namespaceArn: string; ``` - *Type:* string The ARN of the created namespace. --- ##### `namespaceId`^Required ```typescript public readonly namespaceId: string; ``` - *Type:* string The ID of the created namespace. --- ##### `namespaceName`^Required ```typescript public readonly namespaceName: string; ``` - *Type:* string The name of the created namespace. --- ##### `roles`^Required ```typescript public readonly roles: {[ key: string ]: IRole}; ``` - *Type:* {[ key: string ]: aws-cdk-lib.aws_iam.IRole} The roles attached to the namespace in the form of `{RoleArn: IRole}`. These roles are used to access other AWS services for ingestion, federated query, and data catalog access. > [https://docs.aws.amazon.com/redshift/latest/mgmt/redshift-iam-authentication-access-control.html](https://docs.aws.amazon.com/redshift/latest/mgmt/redshift-iam-authentication-access-control.html) --- ##### `statusFunction`^Required ```typescript public readonly statusFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the creation status check. --- ##### `statusLogGroup`^Required ```typescript public readonly statusLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Logs Log Group for the creation status check. --- ##### `statusRole`^Required ```typescript public readonly statusRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the creation status check. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### RedshiftServerlessWorkgroup - *Implements:* aws-cdk-lib.aws_ec2.IConnectable Create a Redshift Serverless Workgroup. A default namespace would be created if none is provided. *Example* ```typescript const workgroup = new dsf.consumption.RedshiftServerlessWorkgroup(this, "RedshiftWorkgroup", { name: "example-workgroup", namespace: new dsf.consumption.RedshiftServerlessNamespace(this, "RedshiftNamespace", { name: 'example-namespace', dbName: 'defaultdb', }) }); ``` #### Initializers ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' new consumption.RedshiftServerlessWorkgroup(scope: Construct, id: string, props: RedshiftServerlessWorkgroupProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessWorkgroupProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessWorkgroupProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | accessData | Creates an instance of `RedshiftData` to send custom SQLs to the workgroup. | | assignDbRolesToIAMRole | Assigns Redshift DB roles to IAM role vs the `RedshiftDbRoles` tag. | | catalogTables | Creates a new Glue data catalog database with a crawler using JDBC target type to connect to the Redshift Workgroup. | | createDatabaseFromShare | Consume datashare by creating a new database pointing to the share. | | createDbRole | Creates a new DB role. | | createShare | Create a new datashare. | | grantAccessToShare | Create a datashare grant to a namespace if it's in the same account, or to another account. | | grantDbAllPrivilegesToRole | Grants both read and write permissions on all the tables in the `schema` to the DB role. | | grantDbSchemaToRole | Grants access to the schema to the DB role. | | grantSchemaReadToRole | Grants read permission on all the tables in the `schema` to the DB role. | | ingestData | Ingest data from S3 into a Redshift table. | | mergeToTargetTable | Run the `MERGE` query using simplified mode. | | retrieveVersion | Retrieve DSF package.json version. | | runCustomSQL | Runs a custom SQL. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### ~~`accessData`~~ ```typescript public accessData(id: string, createVpcEndpoint?: boolean, existingInterfaceVPCEndpoint?: IInterfaceVpcEndpoint): RedshiftData ``` Creates an instance of `RedshiftData` to send custom SQLs to the workgroup. ###### `id`^Required - *Type:* string The CDK ID of the resource. --- ###### `createVpcEndpoint`^Optional - *Type:* boolean if set to true, create interface VPC endpoint for Redshift Data API. --- ###### `existingInterfaceVPCEndpoint`^Optional - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint if `createVpcEndpoint` is false, and if this is populated, then the Lambda function's security group would be added in the existing VPC endpoint's security group. --- ##### `assignDbRolesToIAMRole` ```typescript public assignDbRolesToIAMRole(dbRoles: string[], targetRole: IRole): void ``` Assigns Redshift DB roles to IAM role vs the `RedshiftDbRoles` tag. ###### `dbRoles`^Required - *Type:* string[] List of Redshift DB roles to assign to IAM role. --- ###### `targetRole`^Required - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role to assign the Redshift DB roles to. --- ##### `catalogTables` ```typescript public catalogTables(id: string, catalogDbName: string, pathToCrawl?: string): DataCatalogDatabase ``` Creates a new Glue data catalog database with a crawler using JDBC target type to connect to the Redshift Workgroup. ###### `id`^Required - *Type:* string The CDK ID of the resource. --- ###### `catalogDbName`^Required - *Type:* string The name of the Glue Database to create. --- ###### `pathToCrawl`^Optional - *Type:* string The path of Redshift tables to crawl. --- ##### `createDatabaseFromShare` ```typescript public createDatabaseFromShare(id: string, newDatabaseName: string, producerDataShareName: string, producerNamespaceId?: string, producerAccountId?: string): RedshiftDataSharingCreateDbFromShareProps ``` Consume datashare by creating a new database pointing to the share. If datashare is coming from a different account, setting `autoAssociate` to true automatically associates the datashare to the cluster before the new database is created. ###### `id`^Required - *Type:* string The CDK ID of the resource. --- ###### `newDatabaseName`^Required - *Type:* string The name of the database that would be created from the data share. --- ###### `producerDataShareName`^Required - *Type:* string The name of the data share from producer. --- ###### `producerNamespaceId`^Optional - *Type:* string The producer cluster namespace. --- ###### `producerAccountId`^Optional - *Type:* string The producer account ID. Required for cross account shares. --- ##### `createDbRole` ```typescript public createDbRole(id: string, databaseName: string, roleName: string): CustomResource ``` Creates a new DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `roleName`^Required - *Type:* string The name of the role to create. --- ##### `createShare` ```typescript public createShare(id: string, databaseName: string, dataShareName: string, schema: string, tables: string[]): RedshiftNewShareProps ``` Create a new datashare. ###### `id`^Required - *Type:* string The CDK ID of the resource. --- ###### `databaseName`^Required - *Type:* string The name of the database to connect to. --- ###### `dataShareName`^Required - *Type:* string The name of the datashare. --- ###### `schema`^Required - *Type:* string The schema to add in the datashare. --- ###### `tables`^Required - *Type:* string[] The list of tables that would be included in the datashare. This must follow the format: `.` --- ##### `grantAccessToShare` ```typescript public grantAccessToShare(id: string, dataShareDetails: RedshiftNewShareProps, consumerNamespaceId?: string, consumerAccountId?: string, autoAuthorized?: boolean): RedshiftDataSharingGrantedProps ``` Create a datashare grant to a namespace if it's in the same account, or to another account. ###### `id`^Required - *Type:* string The CDK ID of the resource. --- ###### `dataShareDetails`^Required - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftNewShareProps The details of the datashare. --- ###### `consumerNamespaceId`^Optional - *Type:* string The namespace of the consumer that you're sharing to. Either namespace or account Id must be provided. --- ###### `consumerAccountId`^Optional - *Type:* string The account ID of the consumer that you're sharing to. Either namespace or account Id must be provided. --- ###### `autoAuthorized`^Optional - *Type:* boolean --- ##### `grantDbAllPrivilegesToRole` ```typescript public grantDbAllPrivilegesToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants both read and write permissions on all the tables in the `schema` to the DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `grantDbSchemaToRole` ```typescript public grantDbSchemaToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants access to the schema to the DB role. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `grantSchemaReadToRole` ```typescript public grantSchemaReadToRole(id: string, databaseName: string, schema: string, roleName: string): CustomResource ``` Grants read permission on all the tables in the `schema` to the DB role. ###### `id`^Required - *Type:* string --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `schema`^Required - *Type:* string The schema where the tables are located in. --- ###### `roleName`^Required - *Type:* string The DB role to grant the permissions to. --- ##### `ingestData` ```typescript public ingestData(id: string, databaseName: string, targetTable: string, sourceBucket: IBucket, sourcePrefix: string, ingestAdditionalOptions?: string, role?: IRole): CustomResource ``` Ingest data from S3 into a Redshift table. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `targetTable`^Required - *Type:* string The target table to load the data into. --- ###### `sourceBucket`^Required - *Type:* aws-cdk-lib.aws_s3.IBucket The bucket where the source data would be coming from. --- ###### `sourcePrefix`^Required - *Type:* string The location inside the bucket where the data would be ingested from. --- ###### `ingestAdditionalOptions`^Optional - *Type:* string Optional. Additional options to pass to the `COPY` command. For example, `delimiter '|'` or `ignoreheader 1` --- ###### `role`^Optional - *Type:* aws-cdk-lib.aws_iam.IRole Optional. The IAM Role to use to access the data in S3. If not provided, it would use the default IAM role configured in the Redshift Namespace --- ##### `mergeToTargetTable` ```typescript public mergeToTargetTable(id: string, databaseName: string, sourceTable: string, targetTable: string, sourceColumnId?: string, targetColumnId?: string): CustomResource ``` Run the `MERGE` query using simplified mode. This command would do an upsert into the target table. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `sourceTable`^Required - *Type:* string The source table name. Schema can also be included using the following format: `schemaName.tableName` --- ###### `targetTable`^Required - *Type:* string The target table name. Schema can also be included using the following format: `schemaName.tableName` --- ###### `sourceColumnId`^Optional - *Type:* string The column in the source table that's used to determine whether the rows in the `sourceTable` can be matched with rows in the `targetTable`. Default is `id` --- ###### `targetColumnId`^Optional - *Type:* string The column in the target table that's used to determine whether the rows in the `sourceTable` can be matched with rows in the `targetTable`. Default is `id` --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. ##### `runCustomSQL` ```typescript public runCustomSQL(id: string, databaseName: string, sql: string, deleteSql?: string): CustomResource ``` Runs a custom SQL. Once the custom resource finishes execution, the attribute `Data` contains an attribute `execId` which contains the Redshift Data API execution ID. You can then use this to retrieve execution results via the `GetStatementResult` API. ###### `id`^Required - *Type:* string The CDK Construct ID. --- ###### `databaseName`^Required - *Type:* string The name of the database to run this command. --- ###### `sql`^Required - *Type:* string The sql to run. --- ###### `deleteSql`^Optional - *Type:* string Optional. The sql to run when this resource gets deleted --- #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' consumption.RedshiftServerlessWorkgroup.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | cfnResource | aws-cdk-lib.aws_redshiftserverless.CfnWorkgroup | The created Redshift Serverless Workgroup. | | connections | aws-cdk-lib.aws_ec2.Connections | Connections used by Workgroup security group. | | existingShares | {[ key: string ]: @cdklabs/aws-data-solutions-framework.consumption.RedshiftNewShareProps} | Index of existing shares. | | glueConnection | aws-cdk-lib.aws_glue.CfnConnection | The Glue Connection associated with the workgroup. | | namespace | @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespace | The associated Redshift Serverless Namespace. | | primarySecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The primary EC2 Security Group associated with the Redshift Serverless Workgroup. | | selectedSubnets | aws-cdk-lib.aws_ec2.SelectedSubnets | The subnets where the Redshift Serverless Workgroup is deployed. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where the Redshift Serverless Workgroup is deployed. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `cfnResource`^Required ```typescript public readonly cfnResource: CfnWorkgroup; ``` - *Type:* aws-cdk-lib.aws_redshiftserverless.CfnWorkgroup The created Redshift Serverless Workgroup. --- ##### `connections`^Required ```typescript public readonly connections: Connections; ``` - *Type:* aws-cdk-lib.aws_ec2.Connections Connections used by Workgroup security group. Used this to enable access from clients connecting to the workgroup --- ##### `existingShares`^Required ```typescript public readonly existingShares: {[ key: string ]: RedshiftNewShareProps}; ``` - *Type:* {[ key: string ]: @cdklabs/aws-data-solutions-framework.consumption.RedshiftNewShareProps} Index of existing shares. --- ##### `glueConnection`^Required ```typescript public readonly glueConnection: CfnConnection; ``` - *Type:* aws-cdk-lib.aws_glue.CfnConnection The Glue Connection associated with the workgroup. This can be used by Glue ETL Jobs to read/write data from/to Redshift workgroup --- ##### `namespace`^Required ```typescript public readonly namespace: RedshiftServerlessNamespace; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespace The associated Redshift Serverless Namespace. --- ##### `primarySecurityGroup`^Required ```typescript public readonly primarySecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The primary EC2 Security Group associated with the Redshift Serverless Workgroup. --- ##### `selectedSubnets`^Required ```typescript public readonly selectedSubnets: SelectedSubnets; ``` - *Type:* aws-cdk-lib.aws_ec2.SelectedSubnets The subnets where the Redshift Serverless Workgroup is deployed. --- ##### `vpc`^Required ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc The VPC where the Redshift Serverless Workgroup is deployed. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### S3DataCopy Copy data from one S3 bucket to another. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Utils/s3-data-copy](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Utils/s3-data-copy) *Example* ```typescript import { Bucket } from 'aws-cdk-lib/aws-s3'; const sourceBucket = Bucket.fromBucketName(this, 'SourceBucket', 'nyc-tlc'); const bucketName = `test-${this.region}-${this.account}-${dsf.utils.Utils.generateUniqueHash(this, 'TargetBucket')}`; const targetBucket = new Bucket(this, 'TargetBucket'); new dsf.utils.S3DataCopy(this, 'S3DataCopy', { sourceBucket, sourceBucketPrefix: 'trip data/', sourceBucketRegion: 'us-east-1', targetBucket, }); ``` #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.S3DataCopy(scope: Construct, id: string, props: S3DataCopyProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.utils.S3DataCopyProps | *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.S3DataCopyProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.S3DataCopy.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | copyFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda Function for the copy. | | copyLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the S3 data copy. | | copyRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the copy Lambba Function. | | cleanUpFunction | aws-cdk-lib.aws_lambda.IFunction | The Lambda function for the S3 data copy cleaning up lambda. | | cleanUpLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the S3 data copy cleaning up lambda. | | cleanUpRole | aws-cdk-lib.aws_iam.IRole | The IAM Role for the the S3 data copy cleaning up lambda. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The list of EC2 Security Groups used by the Lambda Functions. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `copyFunction`^Required ```typescript public readonly copyFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda Function for the copy. --- ##### `copyLogGroup`^Required ```typescript public readonly copyLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the S3 data copy. --- ##### `copyRole`^Required ```typescript public readonly copyRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the copy Lambba Function. --- ##### `cleanUpFunction`^Optional ```typescript public readonly cleanUpFunction: IFunction; ``` - *Type:* aws-cdk-lib.aws_lambda.IFunction The Lambda function for the S3 data copy cleaning up lambda. --- ##### `cleanUpLogGroup`^Optional ```typescript public readonly cleanUpLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the S3 data copy cleaning up lambda. --- ##### `cleanUpRole`^Optional ```typescript public readonly cleanUpRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role for the the S3 data copy cleaning up lambda. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] The list of EC2 Security Groups used by the Lambda Functions. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### SparkEmrCICDPipeline A CICD Pipeline to test and deploy a Spark application on Amazon EMR in cross-account environments using CDK Pipelines. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-cicd-pipeline](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-cicd-pipeline) *Example* ```typescript import { Bucket } from 'aws-cdk-lib/aws-s3'; import { CodePipelineSource } from 'aws-cdk-lib/pipelines'; interface MyApplicationStackProps extends cdk.StackProps { readonly stage: dsf.utils.CICDStage; } class MyApplicationStack extends cdk.Stack { constructor(scope: cdk.Stack, props?: MyApplicationStackProps) { super(scope, 'MyApplicationStack'); const bucket = new Bucket(this, 'TestBucket', { autoDeleteObjects: true, removalPolicy: cdk.RemovalPolicy.DESTROY, }); new cdk.CfnOutput(this, 'BucketName', { value: bucket.bucketName }); } } class MyStackFactory implements dsf.utils.ApplicationStackFactory { createStack(scope: cdk.Stack, stage: dsf.utils.CICDStage): cdk.Stack { return new MyApplicationStack(scope, { stage }); } } class MyCICDStack extends cdk.Stack { constructor(scope: Construct, id: string) { super(scope, id); new dsf.processing.SparkEmrCICDPipeline(this, 'TestConstruct', { sparkApplicationName: 'test', applicationStackFactory: new MyStackFactory(), cdkApplicationPath: 'cdk/', sparkApplicationPath: 'spark/', sparkImage: dsf.processing.SparkImage.EMR_6_12, integTestScript: 'cdk/integ-test.sh', integTestEnv: { TEST_BUCKET: 'BucketName', }, source: CodePipelineSource.connection('owner/weekly-job', 'mainline', { connectionArn: 'arn:aws:codeconnections:eu-west-1:123456789012:connection/aEXAMPLE-8aad-4d5d-8878-dfcab0bc441f' }), }); } } ``` #### Initializers ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' new processing.SparkEmrCICDPipeline(scope: Construct, id: string, props: SparkEmrCICDPipelineProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.processing.SparkEmrCICDPipelineProps | the SparkCICDPipelineProps properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.processing.SparkEmrCICDPipelineProps the SparkCICDPipelineProps properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkEmrCICDPipeline.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | artifactAccessLogsBucket | aws-cdk-lib.aws_s3.IBucket | The S3 Bucket for storing the access logs on the artifact S3 Bucket. | | artifactBucket | aws-cdk-lib.aws_s3.IBucket | The S3 Bucket for storing the artifacts. | | pipeline | aws-cdk-lib.pipelines.CodePipeline | The CodePipeline created as part of the Spark CICD Pipeline. | | pipelineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for storing the CodePipeline logs. | | integrationTestStack | @cdklabs/aws-data-solutions-framework.utils.IntegrationTestStack | The IntegrationTestStack containing resources for the running the integration tests. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `artifactAccessLogsBucket`^Required ```typescript public readonly artifactAccessLogsBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The S3 Bucket for storing the access logs on the artifact S3 Bucket. --- ##### `artifactBucket`^Required ```typescript public readonly artifactBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The S3 Bucket for storing the artifacts. --- ##### `pipeline`^Required ```typescript public readonly pipeline: CodePipeline; ``` - *Type:* aws-cdk-lib.pipelines.CodePipeline The CodePipeline created as part of the Spark CICD Pipeline. --- ##### `pipelineLogGroup`^Required ```typescript public readonly pipelineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for storing the CodePipeline logs. --- ##### `integrationTestStack`^Optional ```typescript public readonly integrationTestStack: IntegrationTestStack; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.IntegrationTestStack The IntegrationTestStack containing resources for the running the integration tests. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### SparkEmrContainersJob A construct to run Spark Jobs using EMR Container runtime (EMR on EKS). It creates a Step Functions State Machine that orchestrates the Spark Job. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-job](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-job) *Example* ```typescript import { JsonPath } from 'aws-cdk-lib/aws-stepfunctions'; const job = new dsf.processing.SparkEmrContainersJob(this, 'SparkJob', { jobConfig:{ "Name": JsonPath.format('ge_profile-{}', JsonPath.uuid()), "VirtualClusterId": "virtualClusterId", "ExecutionRoleArn": "ROLE-ARN", "JobDriver": { "SparkSubmit": { "EntryPoint": "s3://S3-BUCKET/pi.py", "EntryPointArguments": [], "SparkSubmitParameters": "--conf spark.executor.instances=2 --conf spark.executor.memory=2G --conf spark.driver.memory=2G --conf spark.executor.cores=4" }, } } } as dsf.processing.SparkEmrContainersJobApiProps); new cdk.CfnOutput(this, 'SparkJobStateMachine', { value: job.stateMachine!.stateMachineArn, }); ``` #### Initializers ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' new processing.SparkEmrContainersJob(scope: Construct, id: string, props: SparkEmrContainersJobProps | SparkEmrContainersJobApiProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | *No description.* | | id | string | *No description.* | | props |

@cdklabs/aws-data-solutions-framework.processing.SparkEmrContainersJobProps \| @cdklabs/aws-data-solutions-framework.processing.SparkEmrContainersJobApiProps

@cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessJobProps \| @cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessJobApiProps

| *No description.* | --- ##### `scope`^Required - *Type:* constructs.Construct --- ##### `id`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessJobProps | @cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessJobApiProps --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkEmrServerlessJob.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | stateMachine | aws-cdk-lib.aws_stepfunctions.StateMachine | The Step Functions State Machine created to orchestrate the Spark Job. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the State Machine. | | sparkJobExecutionRole | aws-cdk-lib.aws_iam.IRole | The Spark job execution role. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `stateMachine`^Optional ```typescript public readonly stateMachine: StateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.StateMachine The Step Functions State Machine created to orchestrate the Spark Job. --- ##### `stateMachineLogGroup`^Optional ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the State Machine. --- ##### `sparkJobExecutionRole`^Optional ```typescript public readonly sparkJobExecutionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The Spark job execution role. Use this property to add additional IAM permissions if necessary. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### SparkEmrServerlessRuntime A construct to create a Spark EMR Serverless Application, along with methods to create IAM roles having the least privilege. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-runtime](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-runtime) *Example* ```typescript import { Role, AccountRootPrincipal } from 'aws-cdk-lib/aws-iam'; const serverlessRuntime = new dsf.processing.SparkEmrServerlessRuntime(this, 'EmrApp', { name: 'SparkRuntimeServerless', }); const executionRole = dsf.processing.SparkEmrServerlessRuntime.createExecutionRole(this, 'ExecutionRole') const submitterRole = new Role (this, 'SubmitterRole', { assumedBy: new AccountRootPrincipal(), }); dsf.processing.SparkEmrServerlessRuntime.grantStartJobExecution(submitterRole, [executionRole.roleArn], ['EMR-serverless-app-ID']); ``` #### Initializers ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' new processing.SparkEmrServerlessRuntime(scope: Construct, id: string, props: SparkEmrServerlessRuntimeProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | props | @cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessRuntimeProps | {@link SparkEmrServerlessRuntimeProps}. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.processing.SparkEmrServerlessRuntimeProps {@link SparkEmrServerlessRuntimeProps}. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | grantStartExecution | A method which will grant an IAM Role the right to start and monitor a job. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `grantStartExecution` ```typescript public grantStartExecution(startJobRole: IRole, executionRoleArn: string): void ``` A method which will grant an IAM Role the right to start and monitor a job. The method will also attach an iam:PassRole permission to limited to the IAM Job Execution roles passed. The excution role will be able to submit job to the EMR Serverless application created by the construct. ###### `startJobRole`^Required - *Type:* aws-cdk-lib.aws_iam.IRole the role that will call the start job api and which need to have the iam:PassRole permission. --- ###### `executionRoleArn`^Required - *Type:* string the role use by EMR Serverless to access resources during the job execution. --- ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | | createExecutionRole | A static method creating an execution IAM role that can be assumed by EMR Serverless The method returns the role it creates. | | grantStartJobExecution | A static method granting the right to start and monitor a job to an IAM Role. | --- ##### `isConstruct` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkEmrServerlessRuntime.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- ##### `createExecutionRole` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkEmrServerlessRuntime.createExecutionRole(scope: Construct, id: string, executionRolePolicyDocument?: PolicyDocument, iamPolicyName?: string) ``` A static method creating an execution IAM role that can be assumed by EMR Serverless The method returns the role it creates. If no `executionRolePolicyDocument` or `iamPolicyName` The method will return a role with only a trust policy to EMR Servereless service principal. You can use this role then to grant access to any resources you control. ###### `scope`^Required - *Type:* constructs.Construct the scope in which to create the role. --- ###### `id`^Required - *Type:* string passed to the IAM Role construct object. --- ###### `executionRolePolicyDocument`^Optional - *Type:* aws-cdk-lib.aws_iam.PolicyDocument the inline policy document to attach to the role. These are IAM policies needed by the job. This parameter is mutually execlusive with iamPolicyName. --- ###### `iamPolicyName`^Optional - *Type:* string the IAM policy name to attach to the role, this is mutually execlusive with executionRolePolicyDocument. --- ##### `grantStartJobExecution` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkEmrServerlessRuntime.grantStartJobExecution(startJobRole: IRole, executionRoleArn: string[], applicationArns: string[]) ``` A static method granting the right to start and monitor a job to an IAM Role. The method will also attach an iam:PassRole permission limited to the IAM Job Execution roles passed ###### `startJobRole`^Required - *Type:* aws-cdk-lib.aws_iam.IRole the role that will call the start job api and which needs to have the iam:PassRole permission. --- ###### `executionRoleArn`^Required - *Type:* string[] the role used by EMR Serverless to access resources during the job execution. --- ###### `applicationArns`^Required - *Type:* string[] the EMR Serverless aplication ARN, this is used by the method to limit the EMR Serverless applications the role can submit job to. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | application | aws-cdk-lib.aws_emrserverless.CfnApplication | The EMR Serverless application. | | emrApplicationSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | If no VPC is provided, one is created by default along with a security group attached to the EMR Serverless Application This attribute is used to expose the security group, if you provide your own security group through the {@link SparkEmrServerlessRuntimeProps} the attribute will be `undefined`. | | flowLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for the VPC flow log when the VPC is created. | | flowLogKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used for the VPC flow log when the VPC is created. | | flowLogRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used for the VPC flow log when the VPC is created. | | s3VpcEndpoint | aws-cdk-lib.aws_ec2.IGatewayVpcEndpoint | *No description.* | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC used by the EKS cluster. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `application`^Required ```typescript public readonly application: CfnApplication; ``` - *Type:* aws-cdk-lib.aws_emrserverless.CfnApplication The EMR Serverless application. --- ##### `emrApplicationSecurityGroup`^Optional ```typescript public readonly emrApplicationSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup If no VPC is provided, one is created by default along with a security group attached to the EMR Serverless Application This attribute is used to expose the security group, if you provide your own security group through the {@link SparkEmrServerlessRuntimeProps} the attribute will be `undefined`. --- ##### `flowLogGroup`^Optional ```typescript public readonly flowLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for the VPC flow log when the VPC is created. --- ##### `flowLogKey`^Optional ```typescript public readonly flowLogKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used for the VPC flow log when the VPC is created. --- ##### `flowLogRole`^Optional ```typescript public readonly flowLogRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used for the VPC flow log when the VPC is created. --- ##### `s3VpcEndpoint`^Optional ```typescript public readonly s3VpcEndpoint: IGatewayVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IGatewayVpcEndpoint --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc The VPC used by the EKS cluster. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ### SparkJob A base construct to run Spark Jobs. Creates an AWS Step Functions State Machine that orchestrates the Spark Job. > [https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-job Available implementations: * {@link SparkEmrServerlessJob } for Emr Serverless implementation * {@link SparkEmrEksJob } for EMR On EKS implementation](https://awslabs.github.io/data-solutions-framework-on-aws/docs/constructs/library/Processing/spark-emr-serverless-job Available implementations: * {@link SparkEmrServerlessJob } for Emr Serverless implementation * {@link SparkEmrEksJob } for EMR On EKS implementation) #### Initializers ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' new processing.SparkJob(scope: Construct, id: string, trackingTag: string, props: SparkJobProps) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | scope | constructs.Construct | the Scope of the CDK Construct. | | id | string | the ID of the CDK Construct. | | trackingTag | string | *No description.* | | props | @cdklabs/aws-data-solutions-framework.processing.SparkJobProps | the SparkJobProps properties. | --- ##### `scope`^Required - *Type:* constructs.Construct the Scope of the CDK Construct. --- ##### `id`^Required - *Type:* string the ID of the CDK Construct. --- ##### `trackingTag`^Required - *Type:* string --- ##### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.processing.SparkJobProps the SparkJobProps properties. --- #### Methods | **Name** | **Description** | | --- | --- | | toString | Returns a string representation of this construct. | | retrieveVersion | Retrieve DSF package.json version. | --- ##### `toString` ```typescript public toString(): string ``` Returns a string representation of this construct. ##### `retrieveVersion` ```typescript public retrieveVersion(): any ``` Retrieve DSF package.json version. #### Static Functions | **Name** | **Description** | | --- | --- | | isConstruct | Checks if `x` is a construct. | --- ##### `isConstruct` ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' processing.SparkJob.isConstruct(x: any) ``` Checks if `x` is a construct. Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked. Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead. ###### `x`^Required - *Type:* any Any object. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | node | constructs.Node | The tree node. | | stateMachine | aws-cdk-lib.aws_stepfunctions.StateMachine | The Step Functions State Machine created to orchestrate the Spark Job. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group used by the State Machine. | --- ##### `node`^Required ```typescript public readonly node: Node; ``` - *Type:* constructs.Node The tree node. --- ##### `stateMachine`^Optional ```typescript public readonly stateMachine: StateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.StateMachine The Step Functions State Machine created to orchestrate the Spark Job. --- ##### `stateMachineLogGroup`^Optional ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group used by the State Machine. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | DSF_OWNED_TAG | string | *No description.* | | DSF_TRACKING_CODE | string | *No description.* | --- ##### `DSF_OWNED_TAG`^Required ```typescript public readonly DSF_OWNED_TAG: string; ``` - *Type:* string --- ##### `DSF_TRACKING_CODE`^Required ```typescript public readonly DSF_TRACKING_CODE: string; ``` - *Type:* string --- ## Structs ### Acl Kakfa ACL This is similar to the object used by `kafkajs`, for more information see this [link](https://kafka.js.org/docs/admin#create-acl). #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const acl: streaming.Acl = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | host | string | *No description.* | | operation | @cdklabs/aws-data-solutions-framework.streaming.AclOperationTypes | *No description.* | | permissionType | @cdklabs/aws-data-solutions-framework.streaming.AclPermissionTypes | *No description.* | | principal | string | *No description.* | | resourceName | string | *No description.* | | resourcePatternType | @cdklabs/aws-data-solutions-framework.streaming.ResourcePatternTypes | *No description.* | | resourceType | @cdklabs/aws-data-solutions-framework.streaming.AclResourceTypes | *No description.* | --- ##### `host`^Required ```typescript public readonly host: string; ``` - *Type:* string --- ##### `operation`^Required ```typescript public readonly operation: AclOperationTypes; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.AclOperationTypes --- ##### `permissionType`^Required ```typescript public readonly permissionType: AclPermissionTypes; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.AclPermissionTypes --- ##### `principal`^Required ```typescript public readonly principal: string; ``` - *Type:* string --- ##### `resourceName`^Required ```typescript public readonly resourceName: string; ``` - *Type:* string --- ##### `resourcePatternType`^Required ```typescript public readonly resourcePatternType: ResourcePatternTypes; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.ResourcePatternTypes --- ##### `resourceType`^Required ```typescript public readonly resourceType: AclResourceTypes; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.AclResourceTypes --- ### AclAdminProps This Props allow you to define the principals that will be adminstartor as well as the principal that will be used by the CDK Custom resources to. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const aclAdminProps: streaming.AclAdminProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | aclAdminPrincipal | string | This Principal will be used by the CDK custom resource to set ACLs and Topics. | | adminPrincipal | string | The Principal that will have administrator privilege in MSK The MSK construct does not have access to this principal Keep this principal in a secure storage and should be only used in case you put an ACL that lock MSK access. | | secretCertificate | aws-cdk-lib.aws_secretsmanager.ISecret | This is the TLS certificate of the Principal that is used by the CDK custom resource which set ACLs and Topics. | --- ##### `aclAdminPrincipal`^Required ```typescript public readonly aclAdminPrincipal: string; ``` - *Type:* string This Principal will be used by the CDK custom resource to set ACLs and Topics. --- ##### `adminPrincipal`^Required ```typescript public readonly adminPrincipal: string; ``` - *Type:* string The Principal that will have administrator privilege in MSK The MSK construct does not have access to this principal Keep this principal in a secure storage and should be only used in case you put an ACL that lock MSK access. --- ##### `secretCertificate`^Required ```typescript public readonly secretCertificate: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret This is the TLS certificate of the Principal that is used by the CDK custom resource which set ACLs and Topics. The secret in AWS secrets manager must be a JSON in the following format { "key" : "PRIVATE-KEY", "cert" : "CERTIFICATE" } You can use the following utility to generate the certificates https://github.com/aws-samples/amazon-msk-client-authentication --- ### AnalyticsBucketProps Properties for the `AnalyticsBucket` construct. #### Initializer ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' const analyticsBucketProps: storage.AnalyticsBucketProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | encryptionKey | aws-cdk-lib.aws_kms.IKey | External KMS Key to use for the S3 Bucket encryption. | | accessControl | aws-cdk-lib.aws_s3.BucketAccessControl | Specifies a canned ACL that grants predefined permissions to the bucket. | | autoDeleteObjects | boolean | Whether all objects should be automatically deleted when the S3 Bucket is removed from the stack or when the stack is deleted. | | blockPublicAccess | aws-cdk-lib.aws_s3.BlockPublicAccess | The block public access configuration of this bucket. | | bucketKeyEnabled | boolean | Whether Amazon S3 should use its own intermediary key to generate data keys. Only relevant when using KMS for encryption. | | bucketName | string | The physical name of this S3 Bucket. | | cors | aws-cdk-lib.aws_s3.CorsRule[] | The CORS configuration of this bucket. | | enforceSSL | boolean | Enforces SSL for requests. | | eventBridgeEnabled | boolean | Whether this S3 Bucket should send notifications to Amazon EventBridge or not. | | intelligentTieringConfigurations | aws-cdk-lib.aws_s3.IntelligentTieringConfiguration[] | Intelligent Tiering Configurations. | | inventories | aws-cdk-lib.aws_s3.Inventory[] | The inventory configuration of the S3 Bucket. | | lifecycleRules | aws-cdk-lib.aws_s3.LifecycleRule[] | Rules that define how Amazon S3 manages objects during their lifetime. | | metrics | aws-cdk-lib.aws_s3.BucketMetrics[] | The metrics configuration of this bucket. | | notificationsHandlerRole | aws-cdk-lib.aws_iam.IRole | The IAM Role to be used by the notifications handler. | | objectLockDefaultRetention | aws-cdk-lib.aws_s3.ObjectLockRetention | The default retention mode and rules for S3 Object Lock. | | objectLockEnabled | boolean | Enable object lock on the S3 Bucket. | | objectOwnership | aws-cdk-lib.aws_s3.ObjectOwnership | The objectOwnership of the S3 Bucket. | | publicReadAccess | boolean | Grants public read access to all objects in the S3 Bucket. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | serverAccessLogsBucket | aws-cdk-lib.aws_s3.IBucket | S3 Bucket destination for the server access logs. | | serverAccessLogsPrefix | string | Optional log file prefix to use for the S3 Bucket's access logs. | | transferAcceleration | boolean | Whether this S3 Bucket should have transfer acceleration turned on or not. | | versioned | boolean | Whether this S3 Bucket should have versioning turned on or not. | --- ##### `encryptionKey`^Required ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* If `encryption` is set to `KMS` and this property is undefined, a new KMS key will be created and associated with this bucket. External KMS Key to use for the S3 Bucket encryption. The `encryption` property must be either not specified or set to `KMS` or `DSSE`. An error will be emitted if `encryption` is set to `UNENCRYPTED` or `S3_MANAGED`. --- ##### `accessControl`^Optional ```typescript public readonly accessControl: BucketAccessControl; ``` - *Type:* aws-cdk-lib.aws_s3.BucketAccessControl - *Default:* BucketAccessControl.PRIVATE Specifies a canned ACL that grants predefined permissions to the bucket. --- ##### `autoDeleteObjects`^Optional ```typescript public readonly autoDeleteObjects: boolean; ``` - *Type:* boolean - *Default:* False Whether all objects should be automatically deleted when the S3 Bucket is removed from the stack or when the stack is deleted. Requires the `removalPolicy` to be set to `RemovalPolicy.DESTROY`. --- ##### `blockPublicAccess`^Optional ```typescript public readonly blockPublicAccess: BlockPublicAccess; ``` - *Type:* aws-cdk-lib.aws_s3.BlockPublicAccess - *Default:* CloudFormation defaults will apply. New buckets and objects don't allow public access, but users can modify bucket policies or object permissions to allow public access The block public access configuration of this bucket. --- ##### `bucketKeyEnabled`^Optional ```typescript public readonly bucketKeyEnabled: boolean; ``` - *Type:* boolean - *Default:* False Whether Amazon S3 should use its own intermediary key to generate data keys. Only relevant when using KMS for encryption. If not enabled, every object GET and PUT will cause an API call to KMS (with the attendant cost implications of that). - If enabled, S3 will use its own time-limited key instead. Only relevant, when Encryption is set to `BucketEncryption.KMS` or `BucketEncryption.KMS_MANAGED`. --- ##### `bucketName`^Optional ```typescript public readonly bucketName: string; ``` - *Type:* string - *Default:* `analytics---` The physical name of this S3 Bucket. --- ##### `cors`^Optional ```typescript public readonly cors: CorsRule[]; ``` - *Type:* aws-cdk-lib.aws_s3.CorsRule[] - *Default:* No CORS configuration. The CORS configuration of this bucket. --- ##### `enforceSSL`^Optional ```typescript public readonly enforceSSL: boolean; ``` - *Type:* boolean - *Default:* False Enforces SSL for requests. S3.5 of the AWS Foundational Security Best Practices Regarding S3. --- ##### `eventBridgeEnabled`^Optional ```typescript public readonly eventBridgeEnabled: boolean; ``` - *Type:* boolean - *Default:* False Whether this S3 Bucket should send notifications to Amazon EventBridge or not. --- ##### `intelligentTieringConfigurations`^Optional ```typescript public readonly intelligentTieringConfigurations: IntelligentTieringConfiguration[]; ``` - *Type:* aws-cdk-lib.aws_s3.IntelligentTieringConfiguration[] - *Default:* No Intelligent Tiiering Configurations. Intelligent Tiering Configurations. --- ##### `inventories`^Optional ```typescript public readonly inventories: Inventory[]; ``` - *Type:* aws-cdk-lib.aws_s3.Inventory[] - *Default:* No inventory configuration The inventory configuration of the S3 Bucket. --- ##### `lifecycleRules`^Optional ```typescript public readonly lifecycleRules: LifecycleRule[]; ``` - *Type:* aws-cdk-lib.aws_s3.LifecycleRule[] - *Default:* No lifecycle rules. Rules that define how Amazon S3 manages objects during their lifetime. --- ##### `metrics`^Optional ```typescript public readonly metrics: BucketMetrics[]; ``` - *Type:* aws-cdk-lib.aws_s3.BucketMetrics[] - *Default:* No metrics configuration. The metrics configuration of this bucket. --- ##### `notificationsHandlerRole`^Optional ```typescript public readonly notificationsHandlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new IAM Role will be created. The IAM Role to be used by the notifications handler. --- ##### `objectLockDefaultRetention`^Optional ```typescript public readonly objectLockDefaultRetention: ObjectLockRetention; ``` - *Type:* aws-cdk-lib.aws_s3.ObjectLockRetention - *Default:* No default retention period The default retention mode and rules for S3 Object Lock. Default retention can be configured after a bucket is created if the bucket already has object lock enabled. Enabling object lock for existing buckets is not supported. --- ##### `objectLockEnabled`^Optional ```typescript public readonly objectLockEnabled: boolean; ``` - *Type:* boolean - *Default:* False, unless objectLockDefaultRetention is set (then, true) Enable object lock on the S3 Bucket. Enabling object lock for existing buckets is not supported. Object lock must be enabled when the bucket is created. --- ##### `objectOwnership`^Optional ```typescript public readonly objectOwnership: ObjectOwnership; ``` - *Type:* aws-cdk-lib.aws_s3.ObjectOwnership - *Default:* No ObjectOwnership configuration, uploading account will own the object. The objectOwnership of the S3 Bucket. --- ##### `publicReadAccess`^Optional ```typescript public readonly publicReadAccess: boolean; ``` - *Type:* boolean - *Default:* False Grants public read access to all objects in the S3 Bucket. Similar to calling `bucket.grantPublicAccess()` --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `serverAccessLogsBucket`^Optional ```typescript public readonly serverAccessLogsBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket - *Default:* If "serverAccessLogsPrefix" undefined - access logs disabled, otherwise - log to current bucket. S3 Bucket destination for the server access logs. --- ##### `serverAccessLogsPrefix`^Optional ```typescript public readonly serverAccessLogsPrefix: string; ``` - *Type:* string - *Default:* No log file prefix Optional log file prefix to use for the S3 Bucket's access logs. If defined without "serverAccessLogsBucket", enables access logs to current S3 Bucket with this prefix. --- ##### `transferAcceleration`^Optional ```typescript public readonly transferAcceleration: boolean; ``` - *Type:* boolean - *Default:* False Whether this S3 Bucket should have transfer acceleration turned on or not. --- ##### `versioned`^Optional ```typescript public readonly versioned: boolean; ``` - *Type:* boolean - *Default:* False (unless object lock is enabled, then true) Whether this S3 Bucket should have versioning turned on or not. --- ### ApplicationStageProps Properties for the `ApplicationStage` class. #### Initializer ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' const applicationStageProps: utils.ApplicationStageProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | env | aws-cdk-lib.Environment | Default AWS environment (account/region) for `Stack`s in this `Stage`. | | outdir | string | The output directory into which to emit synthesized artifacts. | | permissionsBoundary | aws-cdk-lib.PermissionsBoundary | Options for applying a permissions boundary to all IAM Roles and Users created within this Stage. | | policyValidationBeta1 | aws-cdk-lib.IPolicyValidationPluginBeta1[] | Validation plugins to run during synthesis. | | stageName | string | Name of this stage. | | applicationStackFactory | @cdklabs/aws-data-solutions-framework.utils.ApplicationStackFactory | The application CDK Stack Factory used to create application Stacks. | | stage | @cdklabs/aws-data-solutions-framework.utils.CICDStage | The Stage to deploy the application CDK Stack in. | | attachIntegrationTest | boolean | Whether to attach the integration test with this stage or not. | | outputsEnv | {[ key: string ]: string} | The list of values to create CfnOutputs. | | stageProps | {[ key: string ]: any} | (Optional) These are parameters that are passed down to the integration test stack. | --- ##### `env`^Optional ```typescript public readonly env: Environment; ``` - *Type:* aws-cdk-lib.Environment - *Default:* The environments should be configured on the `Stack`s. Default AWS environment (account/region) for `Stack`s in this `Stage`. Stacks defined inside this `Stage` with either `region` or `account` missing from its env will use the corresponding field given here. If either `region` or `account`is is not configured for `Stack` (either on the `Stack` itself or on the containing `Stage`), the Stack will be *environment-agnostic*. Environment-agnostic stacks can be deployed to any environment, may not be able to take advantage of all features of the CDK. For example, they will not be able to use environmental context lookups, will not automatically translate Service Principals to the right format based on the environment's AWS partition, and other such enhancements. --- *Example* ```typescript // Use a concrete account and region to deploy this Stage to new Stage(app, 'Stage1', { env: { account: '123456789012', region: 'us-east-1' }, }); // Use the CLI's current credentials to determine the target environment new Stage(app, 'Stage2', { env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION }, }); ``` ##### `outdir`^Optional ```typescript public readonly outdir: string; ``` - *Type:* string - *Default:* for nested stages, outdir will be determined as a relative directory to the outdir of the app. For apps, if outdir is not specified, a temporary directory will be created. The output directory into which to emit synthesized artifacts. Can only be specified if this stage is the root stage (the app). If this is specified and this stage is nested within another stage, an error will be thrown. --- ##### `permissionsBoundary`^Optional ```typescript public readonly permissionsBoundary: PermissionsBoundary; ``` - *Type:* aws-cdk-lib.PermissionsBoundary - *Default:* no permissions boundary is applied Options for applying a permissions boundary to all IAM Roles and Users created within this Stage. --- ##### `policyValidationBeta1`^Optional ```typescript public readonly policyValidationBeta1: IPolicyValidationPluginBeta1[]; ``` - *Type:* aws-cdk-lib.IPolicyValidationPluginBeta1[] - *Default:* no validation plugins are used Validation plugins to run during synthesis. If any plugin reports any violation, synthesis will be interrupted and the report displayed to the user. --- ##### `stageName`^Optional ```typescript public readonly stageName: string; ``` - *Type:* string - *Default:* Derived from the id. Name of this stage. --- ##### `applicationStackFactory`^Required ```typescript public readonly applicationStackFactory: ApplicationStackFactory; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ApplicationStackFactory The application CDK Stack Factory used to create application Stacks. --- ##### `stage`^Required ```typescript public readonly stage: CICDStage; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.CICDStage - *Default:* No stage is passed to the application stack The Stage to deploy the application CDK Stack in. --- ##### `attachIntegrationTest`^Optional ```typescript public readonly attachIntegrationTest: boolean; ``` - *Type:* boolean - *Default:* No integration test is attached with this stage Whether to attach the integration test with this stage or not. --- ##### `outputsEnv`^Optional ```typescript public readonly outputsEnv: {[ key: string ]: string}; ``` - *Type:* {[ key: string ]: string} - *Default:* No CfnOutputs are created The list of values to create CfnOutputs. --- ##### `stageProps`^Optional ```typescript public readonly stageProps: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} (Optional) These are parameters that are passed down to the integration test stack. For example: the integration test script and where the test script is located. --- ### AthenaWorkgroupProps Properties for the AthenaWorkgroup Construct. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const athenaWorkgroupProps: consumption.AthenaWorkgroupProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | Name of the Workgroup. | | resultLocationPrefix | string | Specifies the location in Amazon S3 where query results are stored. | | bytesScannedCutoffPerQuery | number | Indicates the number of days after creation when objects are deleted from the Result bucket. | | enforceWorkGroupConfiguration | boolean | If set to "true", the settings for the workgroup override client-side settings. | | engineVersion | @cdklabs/aws-data-solutions-framework.consumption.EngineVersion | The engine version on which the query runs. | | executionRole | aws-cdk-lib.aws_iam.IRole | Role used to access user resources in an Athena for Apache Spark session. | | publishCloudWatchMetricsEnabled | boolean | Indicates that the Amazon CloudWatch metrics are enabled for the workgroup. | | recursiveDeleteOption | boolean | The option to delete a workgroup and its contents even if the workgroup contains any named queries. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | requesterPaysEnabled | boolean | Allows members assigned to a workgroup to reference Amazon S3 Requester Pays buckets in queries. | | resultBucket | aws-cdk-lib.aws_s3.IBucket | Amazon S3 Bucket where query results are stored. | | resultBucketName | string | Name for the S3 Bucket in case it should be created. | | resultsEncryptionKey | aws-cdk-lib.aws_kms.IKey | Encryption key used to encrypt query results. | | resultsRetentionPeriod | aws-cdk-lib.Duration | Indicates the number of days after creation when objects are deleted from the Result bucket. | | state | @cdklabs/aws-data-solutions-framework.consumption.State | The state of the Workgroup. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string Name of the Workgroup. --- ##### `resultLocationPrefix`^Required ```typescript public readonly resultLocationPrefix: string; ``` - *Type:* string Specifies the location in Amazon S3 where query results are stored. --- ##### `bytesScannedCutoffPerQuery`^Optional ```typescript public readonly bytesScannedCutoffPerQuery: number; ``` - *Type:* number Indicates the number of days after creation when objects are deleted from the Result bucket. --- ##### `enforceWorkGroupConfiguration`^Optional ```typescript public readonly enforceWorkGroupConfiguration: boolean; ``` - *Type:* boolean - *Default:* True. If set to "true", the settings for the workgroup override client-side settings. --- ##### `engineVersion`^Optional ```typescript public readonly engineVersion: EngineVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.EngineVersion - *Default:* AUTO. The engine version on which the query runs. --- ##### `executionRole`^Optional ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* The role is created if PySpark engine version is selected and no role is provided. Role used to access user resources in an Athena for Apache Spark session. --- ##### `publishCloudWatchMetricsEnabled`^Optional ```typescript public readonly publishCloudWatchMetricsEnabled: boolean; ``` - *Type:* boolean - *Default:* True. Indicates that the Amazon CloudWatch metrics are enabled for the workgroup. --- ##### `recursiveDeleteOption`^Optional ```typescript public readonly recursiveDeleteOption: boolean; ``` - *Type:* boolean - *Default:* Workgroup is retained. The option to delete a workgroup and its contents even if the workgroup contains any named queries. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `requesterPaysEnabled`^Optional ```typescript public readonly requesterPaysEnabled: boolean; ``` - *Type:* boolean - *Default:* False. Allows members assigned to a workgroup to reference Amazon S3 Requester Pays buckets in queries. --- ##### `resultBucket`^Optional ```typescript public readonly resultBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket - *Default:* Create a new bucket with SSE encryption using AnalyticsBucket if not provided. Amazon S3 Bucket where query results are stored. --- ##### `resultBucketName`^Optional ```typescript public readonly resultBucketName: string; ``` - *Type:* string - *Default:* Name will be provided. Name for the S3 Bucket in case it should be created. --- ##### `resultsEncryptionKey`^Optional ```typescript public readonly resultsEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* The key is created if Result Bucket is not provided. Encryption key used to encrypt query results. Has to be provided if Result bucket is provided. User needs to grant access to it for AthenaWorkGroup's executionRole (if Spark engine) or for principals that were granted to run queries using AthenaWorkGroup's grantRunQueries. --- ##### `resultsRetentionPeriod`^Optional ```typescript public readonly resultsRetentionPeriod: Duration; ``` - *Type:* aws-cdk-lib.Duration Indicates the number of days after creation when objects are deleted from the Result bucket. --- ##### `state`^Optional ```typescript public readonly state: State; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.State - *Default:* ENABLED. The state of the Workgroup. --- ### AuthorizerCentralWorflow Interface for the authorizer central workflow. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const authorizerCentralWorflow: governance.AuthorizerCentralWorflow = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | authorizerEventRole | aws-cdk-lib.aws_iam.IRole | The authorizer event role for allowing events to invoke the workflow. | | authorizerEventRule | aws-cdk-lib.aws_events.IRule | The authorizer event rule for triggering the workflow. | | callbackRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the State Machine Call Back. | | deadLetterKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used for encryption of the Dead Letter Queue. | | deadLetterQueue | aws-cdk-lib.aws_sqs.IQueue | The SQS Dead Letter Queue receiving events failure. | | stateMachine | aws-cdk-lib.aws_stepfunctions.StateMachine | The authorizer Step Functions state machine. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group for logging the state machine. | | stateMachineRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the State Machine. | --- ##### `authorizerEventRole`^Required ```typescript public readonly authorizerEventRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The authorizer event role for allowing events to invoke the workflow. --- ##### `authorizerEventRule`^Required ```typescript public readonly authorizerEventRule: IRule; ``` - *Type:* aws-cdk-lib.aws_events.IRule The authorizer event rule for triggering the workflow. --- ##### `callbackRole`^Required ```typescript public readonly callbackRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role The IAM Role used by the State Machine Call Back. --- ##### `deadLetterKey`^Required ```typescript public readonly deadLetterKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS Key used for encryption of the Dead Letter Queue. --- ##### `deadLetterQueue`^Required ```typescript public readonly deadLetterQueue: IQueue; ``` - *Type:* aws-cdk-lib.aws_sqs.IQueue The SQS Dead Letter Queue receiving events failure. --- ##### `stateMachine`^Required ```typescript public readonly stateMachine: StateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.StateMachine The authorizer Step Functions state machine. --- ##### `stateMachineLogGroup`^Required ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The CloudWatch Log Group for logging the state machine. --- ##### `stateMachineRole`^Required ```typescript public readonly stateMachineRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role The IAM Role used by the State Machine. --- ### AuthorizerEnvironmentWorflow The interface representing the environment custom authorizer workflow. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const authorizerEnvironmentWorflow: governance.AuthorizerEnvironmentWorflow = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | stateMachine | aws-cdk-lib.aws_stepfunctions.IStateMachine | The state machine that orchestrates the workflow. | | stateMachineLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The log group where the state machine logs are stored. | | stateMachineRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the State Machine. | --- ##### `stateMachine`^Required ```typescript public readonly stateMachine: IStateMachine; ``` - *Type:* aws-cdk-lib.aws_stepfunctions.IStateMachine The state machine that orchestrates the workflow. --- ##### `stateMachineLogGroup`^Required ```typescript public readonly stateMachineLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup The log group where the state machine logs are stored. --- ##### `stateMachineRole`^Required ```typescript public readonly stateMachineRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM Role used by the State Machine. --- ### BaseRedshiftDataSharingAccessProps The base interface for the different data sharing lifecycle properties. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const baseRedshiftDataSharingAccessProps: consumption.BaseRedshiftDataSharingAccessProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | databaseName | string | The name of the Redshift database used in the data sharing. | | dataShareName | string | The name of the data share. | | accountId | string | For cross-account grants, this is the consumer account ID. | | dataShareArn | string | The ARN of the datashare. | | namespaceId | string | For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. | --- ##### `databaseName`^Required ```typescript public readonly databaseName: string; ``` - *Type:* string The name of the Redshift database used in the data sharing. --- ##### `dataShareName`^Required ```typescript public readonly dataShareName: string; ``` - *Type:* string The name of the data share. --- ##### `accountId`^Optional ```typescript public readonly accountId: string; ``` - *Type:* string - *Default:* No account ID is used. For cross-account grants, this is the consumer account ID. For cross-account consumers, this is the producer account ID. --- ##### `dataShareArn`^Optional ```typescript public readonly dataShareArn: string; ``` - *Type:* string - *Default:* No data share ARN is used. The ARN of the datashare. This is required for any action that is cross account. --- ##### `namespaceId`^Optional ```typescript public readonly namespaceId: string; ``` - *Type:* string - *Default:* No namespace ID is used. For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. For consumers, this is the producer namespace ID. It is required for both single and cross account data sharing. --- ### BrokerLogging Configuration details related to broker logs. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const brokerLogging: streaming.BrokerLogging = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | cloudwatchLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Logs group that is the destination for broker logs. | | firehoseDeliveryStreamName | string | The Kinesis Data Firehose delivery stream that is the destination for broker logs. | | s3 | @cdklabs/aws-data-solutions-framework.streaming.S3LoggingConfiguration | Details of the Amazon S3 destination for broker logs. | --- ##### `cloudwatchLogGroup`^Optional ```typescript public readonly cloudwatchLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup - *Default:* disabled The CloudWatch Logs group that is the destination for broker logs. --- ##### `firehoseDeliveryStreamName`^Optional ```typescript public readonly firehoseDeliveryStreamName: string; ``` - *Type:* string - *Default:* disabled The Kinesis Data Firehose delivery stream that is the destination for broker logs. --- ##### `s3`^Optional ```typescript public readonly s3: S3LoggingConfiguration; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.S3LoggingConfiguration - *Default:* disabled Details of the Amazon S3 destination for broker logs. --- ### ClusterConfigurationInfo The Amazon MSK configuration to use for the cluster. Note: There is currently no Cloudformation Resource to create a Configuration #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const clusterConfigurationInfo: streaming.ClusterConfigurationInfo = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | arn | string | The Amazon Resource Name (ARN) of the MSK configuration to use. | | revision | number | The revision of the Amazon MSK configuration to use. | --- ##### `arn`^Required ```typescript public readonly arn: string; ``` - *Type:* string The Amazon Resource Name (ARN) of the MSK configuration to use. For example, arn:aws:kafka:us-east-1:123456789012:configuration/example-configuration-name/abcdabcd-1234-abcd-1234-abcd123e8e8e-1. --- ##### `revision`^Required ```typescript public readonly revision: number; ``` - *Type:* number The revision of the Amazon MSK configuration to use. --- ### CreateServiceLinkedRoleProps The properties of the `CreateServiceLinkedRole` construct. #### Initializer ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' const createServiceLinkedRoleProps: utils.CreateServiceLinkedRoleProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ### CustomAssetType Interface representing a DataZone custom asset type. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const customAssetType: governance.CustomAssetType = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | domainId | string | The domain identifier of the custom asset type. | | name | string | The name of the custom asset type. | | projectIdentifier | string | The project identifier owner of the custom asset type. | | revision | string | The revision of the custom asset type. | --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The domain identifier of the custom asset type. --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the custom asset type. --- ##### `projectIdentifier`^Required ```typescript public readonly projectIdentifier: string; ``` - *Type:* string The project identifier owner of the custom asset type. --- ##### `revision`^Required ```typescript public readonly revision: string; ``` - *Type:* string The revision of the custom asset type. --- ### DataCatalogDatabaseProps Properties for the `DataCatalogDatabase` construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataCatalogDatabaseProps: governance.DataCatalogDatabaseProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | Database name. | | autoCrawl | boolean | When enabled, this automatically creates a top level Glue Crawler that would run based on the defined schedule in the `autoCrawlSchedule` parameter. | | autoCrawlSchedule | aws-cdk-lib.aws_glue.CfnCrawler.ScheduleProperty | The schedule to run the Glue Crawler. | | crawlerLogEncryptionKey | aws-cdk-lib.aws_kms.IKey | KMS encryption Key used for the Glue Crawler logs. | | crawlerRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Glue Crawler when `autoCrawl` is set to `True`. | | crawlerTableLevelDepth | number | Directory depth where the table folders are located. | | glueConnectionName | string | The connection that would be used by the crawler. | | jdbcPath | string | The JDBC path that would be included by the crawler. | | jdbcSecret | aws-cdk-lib.aws_secretsmanager.ISecret | The secret associated with the JDBC connection. | | jdbcSecretKMSKey | aws-cdk-lib.aws_kms.IKey | The KMS key used by the JDBC secret. | | lakeFormationConfigurationRole | aws-cdk-lib.aws_iam.IRole | The IAM Role assumed by the construct resources to perform Lake Formation configuration. | | lakeFormationDataAccessRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by Lake Formation for [data access](https://docs.aws.amazon.com/lake-formation/latest/dg/registration-role.html). The role is assumed by Lake Formation to provide temporary credentials to query engines. Only needed when permissionModel is set to Lake Formation or Hybrid. | | locationBucket | aws-cdk-lib.aws_s3.IBucket | S3 bucket where data is stored. | | locationPrefix | string | Top level location where table data is stored. | | permissionModel | @cdklabs/aws-data-solutions-framework.utils.PermissionModel | The permission model to apply to the Glue Database. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string Database name. Construct would add a randomize suffix as part of the name to prevent name collisions. --- ##### `autoCrawl`^Optional ```typescript public readonly autoCrawl: boolean; ``` - *Type:* boolean - *Default:* True When enabled, this automatically creates a top level Glue Crawler that would run based on the defined schedule in the `autoCrawlSchedule` parameter. --- ##### `autoCrawlSchedule`^Optional ```typescript public readonly autoCrawlSchedule: ScheduleProperty; ``` - *Type:* aws-cdk-lib.aws_glue.CfnCrawler.ScheduleProperty - *Default:* `cron(1 0 * * ? *)` The schedule to run the Glue Crawler. Default is once a day at 00:01h. --- ##### `crawlerLogEncryptionKey`^Optional ```typescript public readonly crawlerLogEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* Create a new key if none is provided KMS encryption Key used for the Glue Crawler logs. --- ##### `crawlerRole`^Optional ```typescript public readonly crawlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* When `autoCrawl` is enabled, a new role is created with least privilege permissions to run the crawler The IAM Role used by the Glue Crawler when `autoCrawl` is set to `True`. Additional permissions are granted to this role such as S3 Bucket read only permissions and KMS encrypt/decrypt on the key used by the Glue Crawler logging to CloudWatch Logs. --- ##### `crawlerTableLevelDepth`^Optional ```typescript public readonly crawlerTableLevelDepth: number; ``` - *Type:* number - *Default:* calculated based on `locationPrefix` Directory depth where the table folders are located. This helps the Glue Crawler understand the layout of the folders in S3. --- ##### `glueConnectionName`^Optional ```typescript public readonly glueConnectionName: string; ``` - *Type:* string The connection that would be used by the crawler. --- ##### `jdbcPath`^Optional ```typescript public readonly jdbcPath: string; ``` - *Type:* string The JDBC path that would be included by the crawler. --- ##### `jdbcSecret`^Optional ```typescript public readonly jdbcSecret: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret The secret associated with the JDBC connection. --- ##### `jdbcSecretKMSKey`^Optional ```typescript public readonly jdbcSecretKMSKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The KMS key used by the JDBC secret. --- ##### `lakeFormationConfigurationRole`^Optional ```typescript public readonly lakeFormationConfigurationRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created The IAM Role assumed by the construct resources to perform Lake Formation configuration. The role is assumed by Lambda functions to perform Lake Formation related operations. Only needed when permissionModel is set to Lake Formation or Hybrid --- ##### `lakeFormationDataAccessRole`^Optional ```typescript public readonly lakeFormationDataAccessRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created The IAM Role used by Lake Formation for [data access](https://docs.aws.amazon.com/lake-formation/latest/dg/registration-role.html). The role is assumed by Lake Formation to provide temporary credentials to query engines. Only needed when permissionModel is set to Lake Formation or Hybrid. --- ##### `locationBucket`^Optional ```typescript public readonly locationBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket S3 bucket where data is stored. --- ##### `locationPrefix`^Optional ```typescript public readonly locationPrefix: string; ``` - *Type:* string - *Default:* the root of the bucket is used as the location prefix. Top level location where table data is stored. --- ##### `permissionModel`^Optional ```typescript public readonly permissionModel: PermissionModel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.PermissionModel - *Default:* IAM permission model is used The permission model to apply to the Glue Database. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ### DataLakeCatalogProps Properties for the `DataLakeCatalog` Construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataLakeCatalogProps: governance.DataLakeCatalogProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | dataLakeStorage | @cdklabs/aws-data-solutions-framework.storage.DataLakeStorage | The DataLakeStorage object to create the data catalog on. | | autoCrawl | boolean | When enabled, creates a top level Glue Crawler that would run based on the defined schedule in the `autoCrawlSchedule` parameter. | | autoCrawlSchedule | aws-cdk-lib.aws_glue.CfnCrawler.ScheduleProperty | The schedule when the Glue Crawler runs, if enabled. | | crawlerLogEncryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS encryption Key used for the Glue Crawler logs. | | crawlerTableLevelDepth | number | Directory depth where the table folders are located. | | databaseName | string | The suffix of the Glue Data Catalog Database. | | lakeFormationConfigurationRole | aws-cdk-lib.aws_iam.IRole | The IAM Role assumed by the construct resources to perform Lake Formation configuration. | | lakeFormationDataAccessRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by Lake Formation for [data access](https://docs.aws.amazon.com/lake-formation/latest/dg/access-control-underlying-data.html). The role will be used for accessing all the layers of the data lake (bronze, silver, gold). Only needed when permissionModel is set to Lake Formation or Hybrid. | | permissionModel | @cdklabs/aws-data-solutions-framework.utils.PermissionModel | The permission model to apply to the Glue Database. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | --- ##### `dataLakeStorage`^Required ```typescript public readonly dataLakeStorage: DataLakeStorage; ``` - *Type:* @cdklabs/aws-data-solutions-framework.storage.DataLakeStorage The DataLakeStorage object to create the data catalog on. --- ##### `autoCrawl`^Optional ```typescript public readonly autoCrawl: boolean; ``` - *Type:* boolean - *Default:* True When enabled, creates a top level Glue Crawler that would run based on the defined schedule in the `autoCrawlSchedule` parameter. --- ##### `autoCrawlSchedule`^Optional ```typescript public readonly autoCrawlSchedule: ScheduleProperty; ``` - *Type:* aws-cdk-lib.aws_glue.CfnCrawler.ScheduleProperty - *Default:* `cron(1 0 * * ? *)` The schedule when the Glue Crawler runs, if enabled. Default is once a day at 00:01h. --- ##### `crawlerLogEncryptionKey`^Optional ```typescript public readonly crawlerLogEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* Create a new KMS Key if none is provided The KMS encryption Key used for the Glue Crawler logs. --- ##### `crawlerTableLevelDepth`^Optional ```typescript public readonly crawlerTableLevelDepth: number; ``` - *Type:* number - *Default:* calculated based on `locationPrefix` Directory depth where the table folders are located. This helps the Glue Crawler understand the layout of the folders in S3. --- ##### `databaseName`^Optional ```typescript public readonly databaseName: string; ``` - *Type:* string - *Default:* Use the bucket name as the database name and as the S3 location The suffix of the Glue Data Catalog Database. The name of the Glue Database is composed of the S3 Bucket name and this suffix. The suffix is also added to the S3 location inside the data lake S3 Buckets. --- ##### `lakeFormationConfigurationRole`^Optional ```typescript public readonly lakeFormationConfigurationRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created for the entire Data Lake The IAM Role assumed by the construct resources to perform Lake Formation configuration. Only needed when permissionModel is set to Lake Formation or Hybrid --- ##### `lakeFormationDataAccessRole`^Optional ```typescript public readonly lakeFormationDataAccessRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created for the entire Data Lake The IAM Role used by Lake Formation for [data access](https://docs.aws.amazon.com/lake-formation/latest/dg/access-control-underlying-data.html). The role will be used for accessing all the layers of the data lake (bronze, silver, gold). Only needed when permissionModel is set to Lake Formation or Hybrid. --- ##### `permissionModel`^Optional ```typescript public readonly permissionModel: PermissionModel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.PermissionModel - *Default:* IAM permission model is used The permission model to apply to the Glue Database. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ### DataLakeStorageProps Properties for the DataLakeStorage Construct. #### Initializer ```typescript import { storage } from '@cdklabs/aws-data-solutions-framework' const dataLakeStorageProps: storage.DataLakeStorageProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | bronzeBucketArchiveDelay | number | Delay (in days) before archiving BRONZE data to frozen storage (Glacier storage class). | | bronzeBucketInfrequentAccessDelay | number | Delay (in days) before moving BRONZE data to cold storage (Infrequent Access storage class). | | bronzeBucketName | string | Name of the Bronze bucket. | | dataLakeKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt all DataLakeStorage S3 buckets. | | goldBucketArchiveDelay | number | Delay (in days) before archiving GOLD data to frozen storage (Glacier storage class). | | goldBucketInfrequentAccessDelay | number | Delay (in days) before moving GOLD data to cold storage (Infrequent Access storage class). | | goldBucketName | string | Name of the Gold bucket. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | silverBucketArchiveDelay | number | Delay (in days) before archiving SILVER data to frozen storage (Glacier storage class). | | silverBucketInfrequentAccessDelay | number | Delay (in days) before moving SILVER data to cold storage (Infrequent Access storage class). | | silverBucketName | string | Name of the Silver bucket. | --- ##### `bronzeBucketArchiveDelay`^Optional ```typescript public readonly bronzeBucketArchiveDelay: number; ``` - *Type:* number - *Default:* Move objects to Glacier after 90 days. Delay (in days) before archiving BRONZE data to frozen storage (Glacier storage class). --- ##### `bronzeBucketInfrequentAccessDelay`^Optional ```typescript public readonly bronzeBucketInfrequentAccessDelay: number; ``` - *Type:* number - *Default:* Move objects to Infrequent Access after 30 days. Delay (in days) before moving BRONZE data to cold storage (Infrequent Access storage class). --- ##### `bronzeBucketName`^Optional ```typescript public readonly bronzeBucketName: string; ``` - *Type:* string - *Default:* `bronze---` will be used. Name of the Bronze bucket. Use `BucketUtils.generateUniqueBucketName()` to generate a unique name (recommended). --- ##### `dataLakeKey`^Optional ```typescript public readonly dataLakeKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* A single KMS customer key is created. The KMS Key used to encrypt all DataLakeStorage S3 buckets. --- ##### `goldBucketArchiveDelay`^Optional ```typescript public readonly goldBucketArchiveDelay: number; ``` - *Type:* number - *Default:* Objects are not archived to Glacier. Delay (in days) before archiving GOLD data to frozen storage (Glacier storage class). --- ##### `goldBucketInfrequentAccessDelay`^Optional ```typescript public readonly goldBucketInfrequentAccessDelay: number; ``` - *Type:* number - *Default:* Move objects to Infrequent Access after 90 days. Delay (in days) before moving GOLD data to cold storage (Infrequent Access storage class). --- ##### `goldBucketName`^Optional ```typescript public readonly goldBucketName: string; ``` - *Type:* string - *Default:* `gold---` will be used. Name of the Gold bucket. Use `BucketUtils.generateUniqueBucketName()` to generate a unique name (recommended). --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `silverBucketArchiveDelay`^Optional ```typescript public readonly silverBucketArchiveDelay: number; ``` - *Type:* number - *Default:* Objects are not archived to Glacier. Delay (in days) before archiving SILVER data to frozen storage (Glacier storage class). --- ##### `silverBucketInfrequentAccessDelay`^Optional ```typescript public readonly silverBucketInfrequentAccessDelay: number; ``` - *Type:* number - *Default:* Move objects to Infrequent Access after 90 days. Delay (in days) before moving SILVER data to cold storage (Infrequent Access storage class). --- ##### `silverBucketName`^Optional ```typescript public readonly silverBucketName: string; ``` - *Type:* string - *Default:* `silver---` will be used. Name of the Silver bucket. Use `BucketUtils.generateUniqueBucketName()` to generate a unique name (recommended). --- ### DataVpcClientVpnEndpointProps The properties for the ClientVPnEndpoint in DataVpc construct. #### Initializer ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' const dataVpcClientVpnEndpointProps: utils.DataVpcClientVpnEndpointProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | samlMetadataDocument | string | An XML document generated by an identity provider (IdP) that supports SAML 2.0. The document includes the issuer's name, expiration information, and keys that can be used to validate the SAML authentication response (assertions) that are received from the IdP. You must generate the metadata document using the identity management software that is used as your organization's IdP. | | serverCertificateArn | string | The ARN of the server certificate. | | authorizeAllUsersToVpcCidr | boolean | Whether to authorize all users to the VPC CIDR. | | dnsServers | string[] | Information about the DNS servers to be used for DNS resolution. | | logging | boolean | A CloudWatch Logs log group for connection logging. | | logGroup | aws-cdk-lib.aws_logs.ILogGroup | A CloudWatch Logs log group for connection logging. | | port | aws-cdk-lib.aws_ec2.VpnPort | The port number to assign to the Client VPN endpoint for TCP and UDP traffic. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The security groups to apply to the target network. | | selfServicePortal | boolean | Specify whether to enable the self-service portal for the Client VPN endpoint. | | sessionTimeout | aws-cdk-lib.aws_ec2.ClientVpnSessionTimeout | The maximum VPN session duration time. | | splitTunnel | boolean | Indicates whether split-tunnel is enabled on the AWS Client VPN endpoint. | | transportProtocol | aws-cdk-lib.aws_ec2.TransportProtocol | The transport protocol to be used by the VPN session. | --- ##### `samlMetadataDocument`^Required ```typescript public readonly samlMetadataDocument: string; ``` - *Type:* string An XML document generated by an identity provider (IdP) that supports SAML 2.0. The document includes the issuer's name, expiration information, and keys that can be used to validate the SAML authentication response (assertions) that are received from the IdP. You must generate the metadata document using the identity management software that is used as your organization's IdP. --- ##### `serverCertificateArn`^Required ```typescript public readonly serverCertificateArn: string; ``` - *Type:* string The ARN of the server certificate. --- ##### `authorizeAllUsersToVpcCidr`^Optional ```typescript public readonly authorizeAllUsersToVpcCidr: boolean; ``` - *Type:* boolean - *Default:* true Whether to authorize all users to the VPC CIDR. --- ##### `dnsServers`^Optional ```typescript public readonly dnsServers: string[]; ``` - *Type:* string[] - *Default:* DNS server in VPC, e.g. 10.0.0.2 Information about the DNS servers to be used for DNS resolution. --- ##### `logging`^Optional ```typescript public readonly logging: boolean; ``` - *Type:* boolean - *Default:* true A CloudWatch Logs log group for connection logging. --- ##### `logGroup`^Optional ```typescript public readonly logGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup - *Default:* new LogGroup is created A CloudWatch Logs log group for connection logging. --- ##### `port`^Optional ```typescript public readonly port: VpnPort; ``` - *Type:* aws-cdk-lib.aws_ec2.VpnPort - *Default:* true The port number to assign to the Client VPN endpoint for TCP and UDP traffic. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] - *Default:* new Securoty Group is created, allowing the incoming connections on port 443 The security groups to apply to the target network. --- ##### `selfServicePortal`^Optional ```typescript public readonly selfServicePortal: boolean; ``` - *Type:* boolean - *Default:* true Specify whether to enable the self-service portal for the Client VPN endpoint. --- ##### `sessionTimeout`^Optional ```typescript public readonly sessionTimeout: ClientVpnSessionTimeout; ``` - *Type:* aws-cdk-lib.aws_ec2.ClientVpnSessionTimeout - *Default:* 480 minutes The maximum VPN session duration time. --- ##### `splitTunnel`^Optional ```typescript public readonly splitTunnel: boolean; ``` - *Type:* boolean - *Default:* true Indicates whether split-tunnel is enabled on the AWS Client VPN endpoint. --- ##### `transportProtocol`^Optional ```typescript public readonly transportProtocol: TransportProtocol; ``` - *Type:* aws-cdk-lib.aws_ec2.TransportProtocol - *Default:* TCP The transport protocol to be used by the VPN session. --- ### DataVpcProps The properties for the `DataVpc` construct. #### Initializer ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' const dataVpcProps: utils.DataVpcProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | vpcCidr | string | The CIDR to use to create the subnets in the VPC. | | clientVpnEndpointProps | @cdklabs/aws-data-solutions-framework.utils.DataVpcClientVpnEndpointProps | ClientVpnEndpoint propertioes. | | flowLogKey | aws-cdk-lib.aws_kms.IKey | The KMS key used to encrypt the VPC Flow Logs in the CloudWatch Log Group. | | flowLogRetention | aws-cdk-lib.aws_logs.RetentionDays | The retention period to apply to VPC Flow Logs. | | flowLogRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used to send the VPC Flow Logs in CloudWatch. | | natGateways | number | Number of NAT Gateways. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | --- ##### `vpcCidr`^Required ```typescript public readonly vpcCidr: string; ``` - *Type:* string The CIDR to use to create the subnets in the VPC. --- ##### `clientVpnEndpointProps`^Optional ```typescript public readonly clientVpnEndpointProps: DataVpcClientVpnEndpointProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.DataVpcClientVpnEndpointProps - *Default:* None ClientVpnEndpoint propertioes. Required if client vpn endpoint is needed --- ##### `flowLogKey`^Optional ```typescript public readonly flowLogKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* A new KMS key is created The KMS key used to encrypt the VPC Flow Logs in the CloudWatch Log Group. The resource policy of the key must be configured according to the AWS documentation. > [https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html) --- ##### `flowLogRetention`^Optional ```typescript public readonly flowLogRetention: RetentionDays; ``` - *Type:* aws-cdk-lib.aws_logs.RetentionDays - *Default:* One week retention The retention period to apply to VPC Flow Logs. --- ##### `flowLogRole`^Optional ```typescript public readonly flowLogRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new IAM role is created The IAM Role used to send the VPC Flow Logs in CloudWatch. The role must be configured as described in the AWS VPC Flow Log documentation. > [https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html#flow-logs-iam-role](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html#flow-logs-iam-role) --- ##### `natGateways`^Optional ```typescript public readonly natGateways: number; ``` - *Type:* number - *Default:* 3 or the AZs defined in the context Number of NAT Gateways. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ### DataZoneCustomAssetTypeFactoryProps Properties for the DataZoneCustomAssetTypeFactory construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneCustomAssetTypeFactoryProps: governance.DataZoneCustomAssetTypeFactoryProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | domainId | string | The DataZone domain identifier. | | lambdaRole | aws-cdk-lib.aws_iam.IRole | The Lambda role used by the custom resource provider. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy for the custom resource. | --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The DataZone domain identifier. --- ##### `lambdaRole`^Optional ```typescript public readonly lambdaRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created The Lambda role used by the custom resource provider. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN The removal policy for the custom resource. --- ### DataZoneCustomAssetTypeProps Properties for the DataZoneCustomAssetType construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneCustomAssetTypeProps: governance.DataZoneCustomAssetTypeProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | assetTypeName | string | The name of the custom asset type. | | formTypes | @cdklabs/aws-data-solutions-framework.governance.DataZoneFormType[] | The form types of the custom asset type. | | projectId | string | The project identifier owner of the custom asset type. | | assetTypeDescription | string | The description of the custom asset type. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy of the custom asset type. | --- ##### `assetTypeName`^Required ```typescript public readonly assetTypeName: string; ``` - *Type:* string The name of the custom asset type. --- ##### `formTypes`^Required ```typescript public readonly formTypes: DataZoneFormType[]; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneFormType[] The form types of the custom asset type. --- *Example* ```typescript [{ name: 'userForm', model: [{ name: 'firstName', type: 'String', required: true }] }] ``` ##### `projectId`^Required ```typescript public readonly projectId: string; ``` - *Type:* string The project identifier owner of the custom asset type. --- ##### `assetTypeDescription`^Optional ```typescript public readonly assetTypeDescription: string; ``` - *Type:* string - *Default:* No description provided The description of the custom asset type. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RETAIN The removal policy of the custom asset type. --- ### DataZoneFormType Interface representing a DataZoneFormType. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneFormType: governance.DataZoneFormType = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | The name of the form. | | model | @cdklabs/aws-data-solutions-framework.governance.DataZoneFormTypeField[] | The fields of the form. | | required | boolean | Whether the form is required. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the form. --- ##### `model`^Optional ```typescript public readonly model: DataZoneFormTypeField[]; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneFormTypeField[] - *Default:* No model is required. The form is already configured in DataZone. The fields of the form. --- *Example* ```typescript [{ name: 'firstName', type: 'String', required: true }] ``` ##### `required`^Optional ```typescript public readonly required: boolean; ``` - *Type:* boolean - *Default:* false Whether the form is required. --- ### DataZoneFormTypeField Interface representing a DataZoneFormTypeField. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneFormTypeField: governance.DataZoneFormTypeField = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | The name of the field. | | type | string | The type of the field. | | required | boolean | Whether the field is required. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the field. --- ##### `type`^Required ```typescript public readonly type: string; ``` - *Type:* string The type of the field. --- ##### `required`^Optional ```typescript public readonly required: boolean; ``` - *Type:* boolean - *Default:* false Whether the field is required. --- ### DataZoneGsrMskDataSourceProps Properties for configuring a DataZone GSR MSK datasource. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneGsrMskDataSourceProps: governance.DataZoneGsrMskDataSourceProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | clusterName | string | The name of the MSK (Managed Streaming for Apache Kafka) cluster to use. | | domainId | string | The unique identifier for the DataZone domain where the datasource resides. | | projectId | string | The unique identifier for the project associated with this datasource. | | registryName | string | The name of the registry for schema management. | | enableSchemaRegistryEvent | boolean | A flag to trigger the data source based on the Glue Schema Registry events. | | encryptionKey | aws-cdk-lib.aws_kms.Key | The KMS encryption key used to encrypt lambda environment, lambda logs and SSM parameters. | | lambdaRole | aws-cdk-lib.aws_iam.Role | The Role used by the Lambda responsible to manage DataZone MskTopicAssets. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy to apply to the data source. | | runSchedule | aws-cdk-lib.aws_events.Schedule | The cron schedule to run the data source and synchronize DataZone assets with the Glue Schema Registry. | | ssmParameterKey | aws-cdk-lib.aws_kms.Key | The KMS Key used to encrypt the SSM parameter for storing asset information. | --- ##### `clusterName`^Required ```typescript public readonly clusterName: string; ``` - *Type:* string The name of the MSK (Managed Streaming for Apache Kafka) cluster to use. --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The unique identifier for the DataZone domain where the datasource resides. --- ##### `projectId`^Required ```typescript public readonly projectId: string; ``` - *Type:* string The unique identifier for the project associated with this datasource. --- ##### `registryName`^Required ```typescript public readonly registryName: string; ``` - *Type:* string The name of the registry for schema management. --- ##### `enableSchemaRegistryEvent`^Optional ```typescript public readonly enableSchemaRegistryEvent: boolean; ``` - *Type:* boolean - *Default:* false, meaning the EventBridge listener for schema changes is disabled. A flag to trigger the data source based on the Glue Schema Registry events. The data source can be triggered by events independently of the schedule configured with `runSchedule`. --- ##### `encryptionKey`^Optional ```typescript public readonly encryptionKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key - *Default:* AWS managed customer master key (CMK) is used The KMS encryption key used to encrypt lambda environment, lambda logs and SSM parameters. --- ##### `lambdaRole`^Optional ```typescript public readonly lambdaRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role is created The Role used by the Lambda responsible to manage DataZone MskTopicAssets. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN The removal policy to apply to the data source. --- ##### `runSchedule`^Optional ```typescript public readonly runSchedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* `cron(1 0 * * ? *)` if `enableSchemaRegistryEvent` is false or undefined, otherwise no schedule. The cron schedule to run the data source and synchronize DataZone assets with the Glue Schema Registry. The data source can be scheduled independently of the event based trigger configured with `enableSchemaRegistryEvent`. --- ##### `ssmParameterKey`^Optional ```typescript public readonly ssmParameterKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key - *Default:* A new key is created The KMS Key used to encrypt the SSM parameter for storing asset information. --- ### DataZoneMskAssetTypeProps The properties for the DataZoneMskAssetType construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneMskAssetTypeProps: governance.DataZoneMskAssetTypeProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | domainId | string | The DataZone domain identifier. | | dzCustomAssetTypeFactory | @cdklabs/aws-data-solutions-framework.governance.DataZoneCustomAssetTypeFactory | The factory to create the custom asset type. | | projectId | string | The project identifier owner of the custom asset type. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy to apply to the asset type. | --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The DataZone domain identifier. --- ##### `dzCustomAssetTypeFactory`^Optional ```typescript public readonly dzCustomAssetTypeFactory: DataZoneCustomAssetTypeFactory; ``` - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneCustomAssetTypeFactory - *Default:* A new factory is created The factory to create the custom asset type. --- ##### `projectId`^Optional ```typescript public readonly projectId: string; ``` - *Type:* string - *Default:* A new project called MskGovernance is created The project identifier owner of the custom asset type. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN The removal policy to apply to the asset type. --- ### DataZoneMskCentralAuthorizerProps The properties for the DataZoneMskCentralAuthorizer construct. #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneMskCentralAuthorizerProps: governance.DataZoneMskCentralAuthorizerProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | domainId | string | The DataZone Domain ID. | | callbackRole | aws-cdk-lib.aws_iam.Role | The IAM Role used to callback DataZone and acknowledge the subscription grant. | | datazoneEventRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the Event Bridge event to trigger the authorizer. | | deadLetterQueueKey | aws-cdk-lib.aws_kms.Key | The KMS Key used to encrypt the SQS Dead Letter Queue receiving failed events from DataZone. | | logRetention | aws-cdk-lib.aws_logs.RetentionDays | Cloudwatch Logs retention. | | metadataCollectorRole | aws-cdk-lib.aws_iam.Role | The IAM Role used to collect metadata on DataZone assets. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy to apply to the asset type. | | stateMachineRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the Step Function state machine. | --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The DataZone Domain ID. --- ##### `callbackRole`^Optional ```typescript public readonly callbackRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used to callback DataZone and acknowledge the subscription grant. --- ##### `datazoneEventRole`^Optional ```typescript public readonly datazoneEventRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used by the Event Bridge event to trigger the authorizer. --- ##### `deadLetterQueueKey`^Optional ```typescript public readonly deadLetterQueueKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key - *Default:* A new Key is created The KMS Key used to encrypt the SQS Dead Letter Queue receiving failed events from DataZone. --- ##### `logRetention`^Optional ```typescript public readonly logRetention: RetentionDays; ``` - *Type:* aws-cdk-lib.aws_logs.RetentionDays - *Default:* 7 days Cloudwatch Logs retention. --- ##### `metadataCollectorRole`^Optional ```typescript public readonly metadataCollectorRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used to collect metadata on DataZone assets. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN The removal policy to apply to the asset type. --- ##### `stateMachineRole`^Optional ```typescript public readonly stateMachineRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used by the Step Function state machine. --- ### DataZoneMskEnvironmentAuthorizerProps #### Initializer ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' const dataZoneMskEnvironmentAuthorizerProps: governance.DataZoneMskEnvironmentAuthorizerProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | domainId | string | The DataZone Domain ID. | | centralAccountId | string | The central account Id. | | grantMskManagedVpc | boolean | If the authorizer is granting MSK managed VPC permissions. | | grantRole | aws-cdk-lib.aws_iam.Role | The IAM Role used to grant MSK topics. | | logRetention | aws-cdk-lib.aws_logs.RetentionDays | Cloudwatch Logs retention. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy to apply to the asset type. | | stateMachineRole | aws-cdk-lib.aws_iam.Role | The IAM Role used by the Step Function state machine. | --- ##### `domainId`^Required ```typescript public readonly domainId: string; ``` - *Type:* string The DataZone Domain ID. --- ##### `centralAccountId`^Optional ```typescript public readonly centralAccountId: string; ``` - *Type:* string The central account Id. --- ##### `grantMskManagedVpc`^Optional ```typescript public readonly grantMskManagedVpc: boolean; ``` - *Type:* boolean - *Default:* false If the authorizer is granting MSK managed VPC permissions. --- ##### `grantRole`^Optional ```typescript public readonly grantRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used to grant MSK topics. --- ##### `logRetention`^Optional ```typescript public readonly logRetention: RetentionDays; ``` - *Type:* aws-cdk-lib.aws_logs.RetentionDays - *Default:* 7 days Cloudwatch Logs retention. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN The removal policy to apply to the asset type. --- ##### `stateMachineRole`^Optional ```typescript public readonly stateMachineRole: Role; ``` - *Type:* aws-cdk-lib.aws_iam.Role - *Default:* A new role will be created The IAM Role used by the Step Function state machine. --- ### EbsStorageInfo EBS volume information. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const ebsStorageInfo: streaming.EbsStorageInfo = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | encryptionKey | aws-cdk-lib.aws_kms.IKey | The AWS KMS key for encrypting data at rest. | | volumeSize | number | The size in GiB of the EBS volume for the data drive on each broker node. | --- ##### `encryptionKey`^Optional ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* Uses AWS managed CMK (aws/kafka) The AWS KMS key for encrypting data at rest. --- ##### `volumeSize`^Optional ```typescript public readonly volumeSize: number; ``` - *Type:* number - *Default:* 1000 The size in GiB of the EBS volume for the data drive on each broker node. --- ### EmrVirtualClusterProps The properties for the `EmrVirtualCluster` Construct class. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const emrVirtualClusterProps: processing.EmrVirtualClusterProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | The name of the Amazon EMR Virtual Cluster to be created. | | createNamespace | boolean | The flag to create EKS namespace. | | eksNamespace | string | The name of the EKS namespace to be linked to the EMR virtual cluster. | | setNamespaceResourceQuota | boolean | The namespace will be create with ResourceQuota and LimitRange As defined here https://github.com/awslabs/data-solutions-framework-on-aws/blob/main/framework/src/processing/lib/spark-runtime/emr-containers/resources/k8s/resource-management.yaml. | | tags | {[ key: string ]: string} | The tags assigned to the Virtual Cluster. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the Amazon EMR Virtual Cluster to be created. --- ##### `createNamespace`^Optional ```typescript public readonly createNamespace: boolean; ``` - *Type:* boolean - *Default:* Do not create the namespace The flag to create EKS namespace. --- ##### `eksNamespace`^Optional ```typescript public readonly eksNamespace: string; ``` - *Type:* string - *Default:* Use the default namespace The name of the EKS namespace to be linked to the EMR virtual cluster. --- ##### `setNamespaceResourceQuota`^Optional ```typescript public readonly setNamespaceResourceQuota: boolean; ``` - *Type:* boolean - *Default:* true The namespace will be create with ResourceQuota and LimitRange As defined here https://github.com/awslabs/data-solutions-framework-on-aws/blob/main/framework/src/processing/lib/spark-runtime/emr-containers/resources/k8s/resource-management.yaml. --- ##### `tags`^Optional ```typescript public readonly tags: {[ key: string ]: string}; ``` - *Type:* {[ key: string ]: string} - *Default:* none The tags assigned to the Virtual Cluster. --- ### KafkaApiProps Properties for the `KafkaApi` construct. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const kafkaApiProps: streaming.KafkaApiProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | brokerSecurityGroup | aws-cdk-lib.aws_ec2.ISecurityGroup | The AWS security groups to associate with the elastic network interfaces in order to specify who can connect to and communicate with the Amazon MSK cluster. | | clientAuthentication | @cdklabs/aws-data-solutions-framework.streaming.ClientAuthentication | Configuration properties for client authentication. | | clusterArn | string | The ARN of the cluster. | | clusterType | @cdklabs/aws-data-solutions-framework.streaming.MskClusterType | The type of MSK cluster(provisioned or serverless). | | vpc | aws-cdk-lib.aws_ec2.IVpc | Defines the virtual networking environment for this cluster. | | certficateSecret | aws-cdk-lib.aws_secretsmanager.ISecret | This is the TLS certificate of the Principal that is used by the CDK custom resource which set ACLs and Topics. | | environmentEncryption | aws-cdk-lib.aws_kms.IKey | The AWS KMS key to use to encrypt enviornment variable of lambda supporting the MSK Custom Resources creation. | | iamHandlerRole | aws-cdk-lib.aws_iam.IRole | The IAM role to pass to IAM authentication lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal. | | kafkaClientLogLevel | @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel | The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. | | mtlsHandlerRole | aws-cdk-lib.aws_iam.IRole | The IAM role to pass to mTLS lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | serviceToken | string | If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets where the Custom Resource Lambda Function would be created in. | --- ##### `brokerSecurityGroup`^Required ```typescript public readonly brokerSecurityGroup: ISecurityGroup; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup The AWS security groups to associate with the elastic network interfaces in order to specify who can connect to and communicate with the Amazon MSK cluster. --- ##### `clientAuthentication`^Required ```typescript public readonly clientAuthentication: ClientAuthentication; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.ClientAuthentication Configuration properties for client authentication. MSK supports using private TLS certificates or SASL/SCRAM to authenticate the identity of clients. --- ##### `clusterArn`^Required ```typescript public readonly clusterArn: string; ``` - *Type:* string The ARN of the cluster. --- ##### `clusterType`^Required ```typescript public readonly clusterType: MskClusterType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskClusterType The type of MSK cluster(provisioned or serverless). --- ##### `vpc`^Required ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc Defines the virtual networking environment for this cluster. Must have at least 2 subnets in two different AZs. --- ##### `certficateSecret`^Optional ```typescript public readonly certficateSecret: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret This is the TLS certificate of the Principal that is used by the CDK custom resource which set ACLs and Topics. It must be provided if the cluster is using mTLS authentication. The secret in AWS secrets manager must be a JSON in the following format { "key" : "PRIVATE-KEY", "cert" : "CERTIFICATE" } You can use the following utility to generate the certificates https://github.com/aws-samples/amazon-msk-client-authentication --- ##### `environmentEncryption`^Optional ```typescript public readonly environmentEncryption: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The AWS KMS key to use to encrypt enviornment variable of lambda supporting the MSK Custom Resources creation. --- ##### `iamHandlerRole`^Optional ```typescript public readonly iamHandlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role to pass to IAM authentication lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal. --- ##### `kafkaClientLogLevel`^Optional ```typescript public readonly kafkaClientLogLevel: KafkaClientLogLevel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel - *Default:* WARN The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. --- ##### `mtlsHandlerRole`^Optional ```typescript public readonly mtlsHandlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role to pass to mTLS lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `serviceToken`^Optional ```typescript public readonly serviceToken: string; ``` - *Type:* string If there is an already existing service token deployed for the custom resource you can reuse it to reduce the number of resource created. --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* One private subnet with egress is used per AZ. The subnets where the Custom Resource Lambda Function would be created in. --- ### MonitoringConfiguration Monitoring Configuration. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const monitoringConfiguration: streaming.MonitoringConfiguration = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | clusterMonitoringLevel | @cdklabs/aws-data-solutions-framework.streaming.ClusterMonitoringLevel | Specifies the level of monitoring for the MSK cluster. | | enablePrometheusJmxExporter | boolean | Indicates whether you want to enable or disable the JMX Exporter. | | enablePrometheusNodeExporter | boolean | Indicates whether you want to enable or disable the Prometheus Node Exporter. | --- ##### `clusterMonitoringLevel`^Optional ```typescript public readonly clusterMonitoringLevel: ClusterMonitoringLevel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.ClusterMonitoringLevel - *Default:* DEFAULT Specifies the level of monitoring for the MSK cluster. --- ##### `enablePrometheusJmxExporter`^Optional ```typescript public readonly enablePrometheusJmxExporter: boolean; ``` - *Type:* boolean - *Default:* false Indicates whether you want to enable or disable the JMX Exporter. --- ##### `enablePrometheusNodeExporter`^Optional ```typescript public readonly enablePrometheusNodeExporter: boolean; ``` - *Type:* boolean - *Default:* false Indicates whether you want to enable or disable the Prometheus Node Exporter. You can use the Prometheus Node Exporter to get CPU and disk metrics for the broker nodes. --- ### MskProvisionedProps #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const mskProvisionedProps: streaming.MskProvisionedProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | allowEveryoneIfNoAclFound | boolean | if set the to true the following Kafka configuration `allow.everyone.if.no.acl.found` is set to true. When no Cluster Configuration is passed The construct create a cluster configuration and set the following configuration to false and apply it to the cluster. | | brokerInstanceType | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | The EC2 instance type that you want Amazon MSK to use when it creates your brokers. | | brokerNumber | number | The number of Apache Kafka brokers deployed. | | certificateDefinition | @cdklabs/aws-data-solutions-framework.streaming.AclAdminProps | This Props allow you to define the principals that will be adminstartor as well as the principal that will be used by the CDK Custom resources to. | | clientAuthentication | @cdklabs/aws-data-solutions-framework.streaming.ClientAuthentication | Configuration properties for client authentication. | | clusterName | string | The name of the MSK provisioned cluster. | | configuration | @cdklabs/aws-data-solutions-framework.streaming.ClusterConfigurationInfo | The Amazon MSK configuration to use for the cluster. | | currentVersion | string | This parameter is required after executing the first `cdk deploy` It is the version of the MSK cluster that was deployed in the previous `cdk deploy` The cluster might fail in the subsequent updates if it is not set This parameter is obtained by running the following command `aws kafka describe-cluster --cluster-arn YOUR_CLUSTER_ARN`. | | ebsStorage | @cdklabs/aws-data-solutions-framework.streaming.EbsStorageInfo | Information about storage volumes attached to MSK broker nodes. | | environmentEncryption | aws-cdk-lib.aws_kms.IKey | The AWS KMS key to use to encrypt the function environment variables. | | kafkaClientLogLevel | @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel | The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. | | kafkaVersion | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | The version of Apache Kafka. | | logging | @cdklabs/aws-data-solutions-framework.streaming.BrokerLogging | Configure your MSK cluster to send broker logs to different destination types. | | monitoring | @cdklabs/aws-data-solutions-framework.streaming.MonitoringConfiguration | Cluster monitoring configuration. | | placeClusterHandlerInVpc | boolean | If set to true, the cluster handler functions will be placed in the private subnets of the cluster vpc. | | removalPolicy | aws-cdk-lib.RemovalPolicy | What to do when this resource is deleted from a stack. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The AWS security groups to associate with the elastic network interfaces of the Amazon MSK cluster. | | storageMode | @cdklabs/aws-data-solutions-framework.streaming.StorageMode | This controls storage mode for supported storage tiers. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets where to deploy the MSK Provisioned cluster. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where to deploy the MSK Serverless cluster. | | vpcConnectivity | @cdklabs/aws-data-solutions-framework.streaming.VpcClientAuthentication | VPC connection control settings for brokers Defines all client authentication information for VpcConnectivity. | --- ##### `allowEveryoneIfNoAclFound`^Optional ```typescript public readonly allowEveryoneIfNoAclFound: boolean; ``` - *Type:* boolean - *Default:* false if set the to true the following Kafka configuration `allow.everyone.if.no.acl.found` is set to true. When no Cluster Configuration is passed The construct create a cluster configuration and set the following configuration to false and apply it to the cluster. --- ##### `brokerInstanceType`^Optional ```typescript public readonly brokerInstanceType: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType - *Default:* kafka.m5.large The EC2 instance type that you want Amazon MSK to use when it creates your brokers. > [https://docs.aws.amazon.com/msk/latest/developerguide/msk-create-cluster.html#broker-instance-types](https://docs.aws.amazon.com/msk/latest/developerguide/msk-create-cluster.html#broker-instance-types) --- ##### `brokerNumber`^Optional ```typescript public readonly brokerNumber: number; ``` - *Type:* number - *Default:* 1 per availability zone. The number of Apache Kafka brokers deployed. It must be a multiple of the number of availability zones. --- ##### `certificateDefinition`^Optional ```typescript public readonly certificateDefinition: AclAdminProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.AclAdminProps This Props allow you to define the principals that will be adminstartor as well as the principal that will be used by the CDK Custom resources to. --- ##### `clientAuthentication`^Optional ```typescript public readonly clientAuthentication: ClientAuthentication; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.ClientAuthentication - *Default:* IAM is used Configuration properties for client authentication. MSK supports using private TLS certificates or SASL/SCRAM to authenticate the identity of clients. --- ##### `clusterName`^Optional ```typescript public readonly clusterName: string; ``` - *Type:* string - *Default:* default-msk-provisioned The name of the MSK provisioned cluster. --- ##### `configuration`^Optional ```typescript public readonly configuration: ClusterConfigurationInfo; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.ClusterConfigurationInfo - *Default:* none The Amazon MSK configuration to use for the cluster. --- ##### `currentVersion`^Optional ```typescript public readonly currentVersion: string; ``` - *Type:* string This parameter is required after executing the first `cdk deploy` It is the version of the MSK cluster that was deployed in the previous `cdk deploy` The cluster might fail in the subsequent updates if it is not set This parameter is obtained by running the following command `aws kafka describe-cluster --cluster-arn YOUR_CLUSTER_ARN`. --- ##### `ebsStorage`^Optional ```typescript public readonly ebsStorage: EbsStorageInfo; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.EbsStorageInfo - *Default:* 100 GiB EBS volume Information about storage volumes attached to MSK broker nodes. --- ##### `environmentEncryption`^Optional ```typescript public readonly environmentEncryption: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The AWS KMS key to use to encrypt the function environment variables. --- ##### `kafkaClientLogLevel`^Optional ```typescript public readonly kafkaClientLogLevel: KafkaClientLogLevel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel - *Default:* INFO The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. --- ##### `kafkaVersion`^Optional ```typescript public readonly kafkaVersion: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion - *Default:* KafkaVersion.V3_5_1 The version of Apache Kafka. --- ##### `logging`^Optional ```typescript public readonly logging: BrokerLogging; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.BrokerLogging - *Default:* A Cloudwatch log is created Configure your MSK cluster to send broker logs to different destination types. --- ##### `monitoring`^Optional ```typescript public readonly monitoring: MonitoringConfiguration; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MonitoringConfiguration - *Default:* DEFAULT monitoring level Cluster monitoring configuration. --- ##### `placeClusterHandlerInVpc`^Optional ```typescript public readonly placeClusterHandlerInVpc: boolean; ``` - *Type:* boolean If set to true, the cluster handler functions will be placed in the private subnets of the cluster vpc. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* RemovalPolicy.RETAIN What to do when this resource is deleted from a stack. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] - *Default:* create a new security group The AWS security groups to associate with the elastic network interfaces of the Amazon MSK cluster. --- ##### `storageMode`^Optional ```typescript public readonly storageMode: StorageMode; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.StorageMode - *Default:* StorageMode.LOCAL This controls storage mode for supported storage tiers. > [https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html](https://docs.aws.amazon.com/msk/latest/developerguide/msk-tiered-storage.html) --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* the private subnets with egress. The subnets where to deploy the MSK Provisioned cluster. Amazon MSK distributes the broker nodes evenly across these subnets. The subnets must be in distinct Availability Zones. Client subnets can't be in Availability Zone us-east-1e. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* A new VPC is created. The VPC where to deploy the MSK Serverless cluster. Must have at least 2 subnets in two different AZs. --- ##### `vpcConnectivity`^Optional ```typescript public readonly vpcConnectivity: VpcClientAuthentication; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.VpcClientAuthentication VPC connection control settings for brokers Defines all client authentication information for VpcConnectivity. When vpcConnectivity and you provide your own Msk Congifuration You must set `allow.everyone.if.no.acl.found` to `false` --- ### MskServerlessProps Properties for the `MskServerlessCluster` construct. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const mskServerlessProps: streaming.MskServerlessProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | clusterName | string | The name of the MSK Serverless cluster. | | kafkaClientLogLevel | @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel | The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. | | lambdaEnvironmentEncryptionKey | aws-cdk-lib.aws_kms.IKey | The AWS KMS key to use to encrypt enviornment variable of lambda supporting the MSK Custom Resources creation. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The AWS security groups to associate with the elastic network interfaces of the Amazon MSK cluster. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets where to deploy the MSK Serverless cluster. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where to deploy the MSK Serverless cluster. | --- ##### `clusterName`^Optional ```typescript public readonly clusterName: string; ``` - *Type:* string - *Default:* default-msk-serverless The name of the MSK Serverless cluster. --- ##### `kafkaClientLogLevel`^Optional ```typescript public readonly kafkaClientLogLevel: KafkaClientLogLevel; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaClientLogLevel - *Default:* WARN The log level for the lambda that support the Custom Resource for both Managing ACLs and Topics. --- ##### `lambdaEnvironmentEncryptionKey`^Optional ```typescript public readonly lambdaEnvironmentEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey The AWS KMS key to use to encrypt enviornment variable of lambda supporting the MSK Custom Resources creation. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] - *Default:* create a new security group The AWS security groups to associate with the elastic network interfaces of the Amazon MSK cluster. --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* the Vpc default strategy if not specified. The subnets where to deploy the MSK Serverless cluster. The subnets must be in distinct Availability Zones. Client subnets can't be in Availability Zone us-east-1e. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* A new Vpc is created The VPC where to deploy the MSK Serverless cluster. The VPC must have at least 2 subnets in two different AZs. --- ### MskTopic Properties for the `MskTopic` As defined in `ITopicConfig` in [KafkaJS](https://kafka.js.org/docs/admin) SDK. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const mskTopic: streaming.MskTopic = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | numPartitions | number | The number of partitions in the topic. | | topic | string | The name of the topic. | | configEntries | {[ key: string ]: string}[] | The topic level configurations. | | replicationFactor | number | The replication factor of the partitions. | --- ##### `numPartitions`^Required ```typescript public readonly numPartitions: number; ``` - *Type:* number The number of partitions in the topic. --- ##### `topic`^Required ```typescript public readonly topic: string; ``` - *Type:* string The name of the topic. --- ##### `configEntries`^Optional ```typescript public readonly configEntries: {[ key: string ]: string}[]; ``` - *Type:* {[ key: string ]: string}[] - *Default:* no configuration is used The topic level configurations. This parameter cannot be updated after the creation of the topic. --- ##### `replicationFactor`^Optional ```typescript public readonly replicationFactor: number; ``` - *Type:* number - *Default:* For MSK Serverless, the number of AZ. For MSK Provisioned, the cluster default configuration. The replication factor of the partitions. This parameter cannot be updated after the creation of the topic. This parameter should not be provided for MSK Serverless. --- ### OpenSearchApiProps Configuration for the OpenSearch API. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const openSearchApiProps: consumption.OpenSearchApiProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | iamHandlerRole | aws-cdk-lib.aws_iam.IRole | The IAM role to pass to IAM authentication lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal and have the necessary permissions to call OpenSearch API. | | openSearchClusterType | @cdklabs/aws-data-solutions-framework.consumption.OpenSearchClusterType | Type of OpenSearch cluster. | | openSearchEndpoint | string | The OpenSearch Cluster or Serverless collection endpoint to connect to. | | openSearchEndpointRegion | string | AWS Region openSearchEndpoint is provisioned in. | | queryInterval | aws-cdk-lib.Duration | Time between calls to the `isComplete` handler which determines if the resource has been stabilized. | | queryTimeout | aws-cdk-lib.Duration | Total timeout for the entire custom resource operation. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The list of security groups to attach to the lambda functions. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets where the Custom Resource Lambda Function would be created in. | | vpc | aws-cdk-lib.aws_ec2.IVpc | Defines the virtual networking environment for this construct. | --- ##### `iamHandlerRole`^Required ```typescript public readonly iamHandlerRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role to pass to IAM authentication lambda handler This role must be able to be assumed with `lambda.amazonaws.com` service principal and have the necessary permissions to call OpenSearch API. --- ##### `openSearchClusterType`^Required ```typescript public readonly openSearchClusterType: OpenSearchClusterType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.OpenSearchClusterType Type of OpenSearch cluster. --- ##### `openSearchEndpoint`^Required ```typescript public readonly openSearchEndpoint: string; ``` - *Type:* string The OpenSearch Cluster or Serverless collection endpoint to connect to. if you provisoned your cluster using CDK use domainEndpoint property of OpenSearch provisioned cluster or attrCollectionEndpoint property of OpenSearch Serverless collection. --- ##### `openSearchEndpointRegion`^Optional ```typescript public readonly openSearchEndpointRegion: string; ``` - *Type:* string - *Default:* same region as stack. AWS Region openSearchEndpoint is provisioned in. --- ##### `queryInterval`^Optional ```typescript public readonly queryInterval: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* Duration.seconds(5) Time between calls to the `isComplete` handler which determines if the resource has been stabilized. > [dsf.utils.DsfProviderProps *](dsf.utils.DsfProviderProps *) --- ##### `queryTimeout`^Optional ```typescript public readonly queryTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* Duration.minutes(30) Total timeout for the entire custom resource operation. The maximum timeout is 1 hour. > [dsf.utils.DsfProviderProps *](dsf.utils.DsfProviderProps *) --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] - *Default:* If `vpc` is not supplied, no security groups are attached. Otherwise, a dedicated security group is created for each function. The list of security groups to attach to the lambda functions. The security groups MUST be exclusively used by the custom resource. --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* One private subnet with egress is used per AZ. The subnets where the Custom Resource Lambda Function would be created in. Required if vpc parameter is provided. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* no VPC is used. Defines the virtual networking environment for this construct. Typically should use same VPC as OpenSearch cluster or serverless collection. Must have at least 2 subnets in two different AZs. --- ### OpenSearchClusterProps Simplified configuration for the OpenSearch Cluster. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const openSearchClusterProps: consumption.OpenSearchClusterProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | deployInVpc | boolean | If the OpenSearch Domain is created in a default VPC when there is no VPC configured. | | domainName | string | The OpenSearch Domain name. | | samlEntityId | string | The SAML entity ID used for SAML based authentication. | | samlMasterBackendRole | string | The SAML Idp Admin GroupId as returned by {user:groups} in Idp. | | samlMetadataContent | string | The SAML Idp XML Metadata Content, needs to be downloaded from IAM Identity Center. | | availabilityZoneCount | number | The number of availability zones to use. | | dataNodeInstanceCount | number | The number of OpenSearch data nodes to provision. | | dataNodeInstanceType | string | The EC2 Instance Type used for OpenSearch data nodes. | | ebsSize | number | The size of EBS Volumes to use. | | ebsVolumeType | aws-cdk-lib.aws_ec2.EbsDeviceVolumeType | The type of EBS Volumes to use. | | enableAutoSoftwareUpdate | boolean | Enable OpenSearch Auto Software Update. | | enableVersionUpgrade | boolean | Enable OpenSearch Version Upgrade. | | encryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS Key for encryption in OpenSearch (data and logs). | | masterNodeInstanceCount | number | The number of OpenSearch master nodes to provision. | | masterNodeInstanceType | string | The EC2 Instance Type for OpenSearch master nodes. | | multiAzWithStandbyEnabled | boolean | If multi AZ with standby mode is enabled. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | samlRolesKey | string | The SAML Roles Key. | | samlSessionTimeout | aws-cdk-lib.Duration | The timeout of the SAML session. | | samlSubjectKey | string | The SAML Subject Key. | | version | aws-cdk-lib.aws_opensearchservice.EngineVersion | The OpenSearch version. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC to deploy the OpenSearch Domain. | | vpcSubnets | aws-cdk-lib.aws_ec2.SubnetSelection | The VPC private Subnets to deploy the OpenSearch cluster nodes. | | warmInstanceCount | number | The number of Ultra Warn nodes to provision. | | warmInstanceType | number | The type of nodes for Ultra Warn nodes. | --- ##### `deployInVpc`^Required ```typescript public readonly deployInVpc: boolean; ``` - *Type:* boolean If the OpenSearch Domain is created in a default VPC when there is no VPC configured. --- ##### `domainName`^Required ```typescript public readonly domainName: string; ``` - *Type:* string The OpenSearch Domain name. --- ##### `samlEntityId`^Required ```typescript public readonly samlEntityId: string; ``` - *Type:* string The SAML entity ID used for SAML based authentication. --- ##### `samlMasterBackendRole`^Required ```typescript public readonly samlMasterBackendRole: string; ``` - *Type:* string The SAML Idp Admin GroupId as returned by {user:groups} in Idp. --- ##### `samlMetadataContent`^Required ```typescript public readonly samlMetadataContent: string; ``` - *Type:* string The SAML Idp XML Metadata Content, needs to be downloaded from IAM Identity Center. --- ##### `availabilityZoneCount`^Optional ```typescript public readonly availabilityZoneCount: number; ``` - *Type:* number - *Default:* For private Domains, use the number of configured `vpcSubnets` or the number of AZ in the VPC if not configured. For public Domains, 1 AZ is used. The number of availability zones to use. Be sure to configure the number of data nodes to a multiple of the number of AZ. --- ##### `dataNodeInstanceCount`^Optional ```typescript public readonly dataNodeInstanceCount: number; ``` - *Type:* number - *Default:* For public Domains, 1 data node is created. For private Domains, 1 data node per AZ. The number of OpenSearch data nodes to provision. Be sure to configure the number of data nodes to a multiple of the number of AZ. --- ##### `dataNodeInstanceType`^Optional ```typescript public readonly dataNodeInstanceType: string; ``` - *Type:* string - *Default:* The EC2 Instance Type used for OpenSearch data nodes. > [OpenSearchNodes.DATA_NODE_INSTANCE_DEFAULT](OpenSearchNodes.DATA_NODE_INSTANCE_DEFAULT) --- ##### `ebsSize`^Optional ```typescript public readonly ebsSize: number; ``` - *Type:* number - *Default:* 10 The size of EBS Volumes to use. --- ##### `ebsVolumeType`^Optional ```typescript public readonly ebsVolumeType: EbsDeviceVolumeType; ``` - *Type:* aws-cdk-lib.aws_ec2.EbsDeviceVolumeType - *Default:* EbsDeviceVolumeType.GENERAL_PURPOSE_SSD_GP3 is used The type of EBS Volumes to use. --- ##### `enableAutoSoftwareUpdate`^Optional ```typescript public readonly enableAutoSoftwareUpdate: boolean; ``` - *Type:* boolean - *Default:* false Enable OpenSearch Auto Software Update. --- ##### `enableVersionUpgrade`^Optional ```typescript public readonly enableVersionUpgrade: boolean; ``` - *Type:* boolean - *Default:* false Enable OpenSearch Version Upgrade. --- ##### `encryptionKey`^Optional ```typescript public readonly encryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* A new key is created The KMS Key for encryption in OpenSearch (data and logs). --- ##### `masterNodeInstanceCount`^Optional ```typescript public readonly masterNodeInstanceCount: number; ``` - *Type:* number - *Default:* No master nodes are created The number of OpenSearch master nodes to provision. --- ##### `masterNodeInstanceType`^Optional ```typescript public readonly masterNodeInstanceType: string; ``` - *Type:* string - *Default:* The EC2 Instance Type for OpenSearch master nodes. > [OpenSearchNodes.MASTER_NODE_INSTANCE_DEFAULT](OpenSearchNodes.MASTER_NODE_INSTANCE_DEFAULT) --- ##### `multiAzWithStandbyEnabled`^Optional ```typescript public readonly multiAzWithStandbyEnabled: boolean; ``` - *Type:* boolean - *Default:* false If multi AZ with standby mode is enabled. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `samlRolesKey`^Optional ```typescript public readonly samlRolesKey: string; ``` - *Type:* string - *Default:* "Role" is used The SAML Roles Key. --- ##### `samlSessionTimeout`^Optional ```typescript public readonly samlSessionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 480 minutes The timeout of the SAML session. Max allowed value is 24 hours. --- ##### `samlSubjectKey`^Optional ```typescript public readonly samlSubjectKey: string; ``` - *Type:* string - *Default:* No subject key is used The SAML Subject Key. --- ##### `version`^Optional ```typescript public readonly version: EngineVersion; ``` - *Type:* aws-cdk-lib.aws_opensearchservice.EngineVersion - *Default:* The OpenSearch version. > [OPENSEARCH_DEFAULT_VERSION](OPENSEARCH_DEFAULT_VERSION) --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* A new VPC is created if `deployInVpc` is `true`, The VPC to deploy the OpenSearch Domain. > [DataVpc](DataVpc) --- ##### `vpcSubnets`^Optional ```typescript public readonly vpcSubnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* Single private subnet per each AZ. The VPC private Subnets to deploy the OpenSearch cluster nodes. Only used for VPC deployments. You must configure a VPC if you configure this parameter. Provide only one Subnet per AZ. > [DataVpc](DataVpc) --- ##### `warmInstanceCount`^Optional ```typescript public readonly warmInstanceCount: number; ``` - *Type:* number - *Default:* No Ultra Warn nodes are created The number of Ultra Warn nodes to provision. --- ##### `warmInstanceType`^Optional ```typescript public readonly warmInstanceType: number; ``` - *Type:* number - *Default:* The type of nodes for Ultra Warn nodes. > [OpenSearchNodes.WARM_NODE_INSTANCE_DEFAULT](OpenSearchNodes.WARM_NODE_INSTANCE_DEFAULT) --- ### PySparkApplicationPackageProps Properties for the `PySparkApplicationPackage` construct. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const pySparkApplicationPackageProps: processing.PySparkApplicationPackageProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | applicationName | string | The name of the PySpark application. | | entrypointPath | string | The source path in the code base where the entrypoint is stored. | | artifactsBucket | aws-cdk-lib.aws_s3.IBucket | The S3 bucket where to upload the artifacts of the Spark Job This is where the entry point and archive of the virtual environment will be stored. | | assetUploadMemorySize | number | The memory size (in MiB) used by the Lambda function to upload and unzip the assets (entrypoint and dependencies). | | assetUploadRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used by the Lambda function to upload assets (entrypoint and dependencies). | | assetUploadStorageSize | aws-cdk-lib.Size | The ephemeral storage size used by the Lambda function to upload and unzip the assets (entrypoint and dependencies). | | dependenciesFolder | string | The source directory where `requirements.txt` or `pyproject.toml` file is stored. These files are used to install external AND internal Python packages. If your PySpark application has more than one Python file, you need to [package your Python project](https://packaging.python.org/en/latest/tutorials/packaging-projects/). This location must also contain a `Dockerfile` that can [create a virtual environment and build an archive](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html#building-python-virtual-env). | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | venvArchivePath | string | The path of the Python virtual environment archive generated in the Docker container. | --- ##### `applicationName`^Required ```typescript public readonly applicationName: string; ``` - *Type:* string The name of the PySpark application. This name is used as a parent directory in S3 to store the entrypoint and the optional virtual environment archive --- ##### `entrypointPath`^Required ```typescript public readonly entrypointPath: string; ``` - *Type:* string The source path in the code base where the entrypoint is stored. example `~/my-project/src/entrypoint.py` --- ##### `artifactsBucket`^Optional ```typescript public readonly artifactsBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket - *Default:* An S3 Bucket is created The S3 bucket where to upload the artifacts of the Spark Job This is where the entry point and archive of the virtual environment will be stored. --- ##### `assetUploadMemorySize`^Optional ```typescript public readonly assetUploadMemorySize: number; ``` - *Type:* number - *Default:* 512 MB The memory size (in MiB) used by the Lambda function to upload and unzip the assets (entrypoint and dependencies). If you are deploying large files, you will need to increase this number accordingly. --- ##### `assetUploadRole`^Optional ```typescript public readonly assetUploadRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new Role would be created with least privilege permissions The IAM Role used by the Lambda function to upload assets (entrypoint and dependencies). Additional permissions would be granted to this role such as S3 Bucket permissions. --- ##### `assetUploadStorageSize`^Optional ```typescript public readonly assetUploadStorageSize: Size; ``` - *Type:* aws-cdk-lib.Size - *Default:* 1024 MB The ephemeral storage size used by the Lambda function to upload and unzip the assets (entrypoint and dependencies). If you are deploying large files, you will need to increase this number accordingly. --- ##### `dependenciesFolder`^Optional ```typescript public readonly dependenciesFolder: string; ``` - *Type:* string - *Default:* No dependencies (internal or external) are packaged. Only the entrypoint is used in the Spark Job. The source directory where `requirements.txt` or `pyproject.toml` file is stored. These files are used to install external AND internal Python packages. If your PySpark application has more than one Python file, you need to [package your Python project](https://packaging.python.org/en/latest/tutorials/packaging-projects/). This location must also contain a `Dockerfile` that can [create a virtual environment and build an archive](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html#building-python-virtual-env). --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. Resources like Amazon cloudwatch log or Amazon S3 bucket. If DESTROY is selected, the context value '@data-solutions-framework-on-aws/removeDataOnDestroy' in the 'cdk.json' or 'cdk.context.json' must be set to true. --- ##### `venvArchivePath`^Optional ```typescript public readonly venvArchivePath: string; ``` - *Type:* string - *Default:* No virtual environment archive is packaged. Only the entrypoint can be used in the Spark Job. It is required if the `dependenciesFolder` is provided. The path of the Python virtual environment archive generated in the Docker container. This is the output path used in the `venv-pack -o` command in your Dockerfile. --- ### RedshiftDataAccessTargetProps #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataAccessTargetProps: consumption.RedshiftDataAccessTargetProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | targetArn | string | *No description.* | | targetId | string | *No description.* | | targetType | string | *No description.* | --- ##### `targetArn`^Required ```typescript public readonly targetArn: string; ``` - *Type:* string --- ##### `targetId`^Required ```typescript public readonly targetId: string; ``` - *Type:* string --- ##### `targetType`^Required ```typescript public readonly targetType: string; ``` - *Type:* string --- ### RedshiftDataProps The properties for the `RedshiftData` construct. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataProps: consumption.RedshiftDataProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | secret | aws-cdk-lib.aws_secretsmanager.ISecret | The Secrets Manager Secret containing the admin credentials for the Redshift cluster / namespace. | | clusterId | string | The name of the Redshift provisioned to query. | | createInterfaceVpcEndpoint | boolean | If set to true, create the Redshift Data Interface VPC Endpoint in the configured VPC/Subnets. | | executionTimeout | aws-cdk-lib.Duration | The timeout for the query execution. | | existingInterfaceVPCEndpoint | aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint | If this parameter is provided, the data access execution security group would be granted inbound to the interface VPC endpoint's security group. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | secretKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt the admin credentials for the Redshift cluster / namespace. | | subnets | aws-cdk-lib.aws_ec2.SelectedSubnets | The subnets where the Custom Resource Lambda Function would be created in. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where the Custom Resource Lambda Function would be created in. | | workgroupId | string | The `workgroupId` for the Redshift Serverless Workgroup to query. | --- ##### `secret`^Required ```typescript public readonly secret: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret The Secrets Manager Secret containing the admin credentials for the Redshift cluster / namespace. --- ##### `clusterId`^Optional ```typescript public readonly clusterId: string; ``` - *Type:* string - *Default:* The `workgroupId` is used The name of the Redshift provisioned to query. It must be configured if the `workgroupId` is not. --- ##### `createInterfaceVpcEndpoint`^Optional ```typescript public readonly createInterfaceVpcEndpoint: boolean; ``` - *Type:* boolean - *Default:* false If set to true, create the Redshift Data Interface VPC Endpoint in the configured VPC/Subnets. --- ##### `executionTimeout`^Optional ```typescript public readonly executionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 5mins The timeout for the query execution. --- ##### `existingInterfaceVPCEndpoint`^Optional ```typescript public readonly existingInterfaceVPCEndpoint: IInterfaceVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint - *Default:* No security group ingress rule would be created. If this parameter is provided, the data access execution security group would be granted inbound to the interface VPC endpoint's security group. This is assuming that the `createInterfaceVpcEndpoint` parameter is `false`. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `secretKey`^Optional ```typescript public readonly secretKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* no secret key is used The KMS Key used to encrypt the admin credentials for the Redshift cluster / namespace. --- ##### `subnets`^Optional ```typescript public readonly subnets: SelectedSubnets; ``` - *Type:* aws-cdk-lib.aws_ec2.SelectedSubnets - *Default:* No subnets are used. The Custom Resource runs in the Redshift service team subnets. The subnets where the Custom Resource Lambda Function would be created in. A Redshift Data API Interface VPC Endpoint is created in the subnets. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* No VPC is used. The Custom Resource runs in the Redshift service team VPC The VPC where the Custom Resource Lambda Function would be created in. A Redshift Data API Interface VPC Endpoint is created in the VPC. --- ##### `workgroupId`^Optional ```typescript public readonly workgroupId: string; ``` - *Type:* string - *Default:* The `clusterId` is used The `workgroupId` for the Redshift Serverless Workgroup to query. It must be configured if the `clusterId` is not. --- ### RedshiftDataSharingCreateDbFromShareProps Return interface after creating a new database from data share. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataSharingCreateDbFromShareProps: consumption.RedshiftDataSharingCreateDbFromShareProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | resource | aws-cdk-lib.CustomResource | The resource associated with the create database command. | | associateDataShareResource | aws-cdk-lib.custom_resources.AwsCustomResource | If auto-association is turned on, this is the resource associated with the action. | --- ##### `resource`^Required ```typescript public readonly resource: CustomResource; ``` - *Type:* aws-cdk-lib.CustomResource The resource associated with the create database command. --- ##### `associateDataShareResource`^Optional ```typescript public readonly associateDataShareResource: AwsCustomResource; ``` - *Type:* aws-cdk-lib.custom_resources.AwsCustomResource If auto-association is turned on, this is the resource associated with the action. --- ### RedshiftDataSharingCreateDbProps Properties for data sharing consumer. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataSharingCreateDbProps: consumption.RedshiftDataSharingCreateDbProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | databaseName | string | The name of the Redshift database used in the data sharing. | | dataShareName | string | The name of the data share. | | accountId | string | For cross-account grants, this is the consumer account ID. | | dataShareArn | string | The ARN of the datashare. | | namespaceId | string | For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. | | newDatabaseName | string | For consumers, the data share would be located in this database that would be created. | | consumerNamespaceArn | string | The namespace of the consumer, necessary for cross-account data shares. | --- ##### `databaseName`^Required ```typescript public readonly databaseName: string; ``` - *Type:* string The name of the Redshift database used in the data sharing. --- ##### `dataShareName`^Required ```typescript public readonly dataShareName: string; ``` - *Type:* string The name of the data share. --- ##### `accountId`^Optional ```typescript public readonly accountId: string; ``` - *Type:* string - *Default:* No account ID is used. For cross-account grants, this is the consumer account ID. For cross-account consumers, this is the producer account ID. --- ##### `dataShareArn`^Optional ```typescript public readonly dataShareArn: string; ``` - *Type:* string - *Default:* No data share ARN is used. The ARN of the datashare. This is required for any action that is cross account. --- ##### `namespaceId`^Optional ```typescript public readonly namespaceId: string; ``` - *Type:* string - *Default:* No namespace ID is used. For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. For consumers, this is the producer namespace ID. It is required for both single and cross account data sharing. --- ##### `newDatabaseName`^Required ```typescript public readonly newDatabaseName: string; ``` - *Type:* string For consumers, the data share would be located in this database that would be created. --- ##### `consumerNamespaceArn`^Optional ```typescript public readonly consumerNamespaceArn: string; ``` - *Type:* string The namespace of the consumer, necessary for cross-account data shares. --- ### RedshiftDataSharingGrantedProps Return interface after granting access to consumer. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataSharingGrantedProps: consumption.RedshiftDataSharingGrantedProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | resource | aws-cdk-lib.CustomResource | The resource associated with the grant command. | | shareAuthorizationResource | aws-cdk-lib.custom_resources.AwsCustomResource | If auto-authorization is turned on, this is the resource associated with the action. | --- ##### `resource`^Required ```typescript public readonly resource: CustomResource; ``` - *Type:* aws-cdk-lib.CustomResource The resource associated with the grant command. --- ##### `shareAuthorizationResource`^Optional ```typescript public readonly shareAuthorizationResource: AwsCustomResource; ``` - *Type:* aws-cdk-lib.custom_resources.AwsCustomResource If auto-authorization is turned on, this is the resource associated with the action. --- ### RedshiftDataSharingGrantProps Properties for data sharing grants. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataSharingGrantProps: consumption.RedshiftDataSharingGrantProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | databaseName | string | The name of the Redshift database used in the data sharing. | | dataShareName | string | The name of the data share. | | accountId | string | For cross-account grants, this is the consumer account ID. | | dataShareArn | string | The ARN of the datashare. | | namespaceId | string | For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. | | autoAuthorized | boolean | If set to `true`, cross-account grants would automatically be authorized. | --- ##### `databaseName`^Required ```typescript public readonly databaseName: string; ``` - *Type:* string The name of the Redshift database used in the data sharing. --- ##### `dataShareName`^Required ```typescript public readonly dataShareName: string; ``` - *Type:* string The name of the data share. --- ##### `accountId`^Optional ```typescript public readonly accountId: string; ``` - *Type:* string - *Default:* No account ID is used. For cross-account grants, this is the consumer account ID. For cross-account consumers, this is the producer account ID. --- ##### `dataShareArn`^Optional ```typescript public readonly dataShareArn: string; ``` - *Type:* string - *Default:* No data share ARN is used. The ARN of the datashare. This is required for any action that is cross account. --- ##### `namespaceId`^Optional ```typescript public readonly namespaceId: string; ``` - *Type:* string - *Default:* No namespace ID is used. For single account grants, this is the consumer namespace ID. For cross-account grants, `namespaceId` is ignored. For consumers, this is the producer namespace ID. It is required for both single and cross account data sharing. --- ##### `autoAuthorized`^Optional ```typescript public readonly autoAuthorized: boolean; ``` - *Type:* boolean - *Default:* cross-account grants should be authorized manually If set to `true`, cross-account grants would automatically be authorized. See https://docs.aws.amazon.com/redshift/latest/dg/consumer-account-admin.html --- ### RedshiftDataSharingProps Properties for the `RedshiftDataSharing` construct. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftDataSharingProps: consumption.RedshiftDataSharingProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | secret | aws-cdk-lib.aws_secretsmanager.ISecret | The Secrets Manager Secret containing the admin credentials for the Redshift cluster / namespace. | | clusterId | string | The name of the Redshift provisioned to query. | | createInterfaceVpcEndpoint | boolean | If set to true, create the Redshift Data Interface VPC Endpoint in the configured VPC/Subnets. | | executionTimeout | aws-cdk-lib.Duration | The timeout for the query execution. | | existingInterfaceVPCEndpoint | aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint | If this parameter is provided, the data access execution security group would be granted inbound to the interface VPC endpoint's security group. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | secretKey | aws-cdk-lib.aws_kms.IKey | The KMS Key used to encrypt the admin credentials for the Redshift cluster / namespace. | | subnets | aws-cdk-lib.aws_ec2.SelectedSubnets | The subnets where the Custom Resource Lambda Function would be created in. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC where the Custom Resource Lambda Function would be created in. | | workgroupId | string | The `workgroupId` for the Redshift Serverless Workgroup to query. | | redshiftData | @cdklabs/aws-data-solutions-framework.consumption.RedshiftData | Instance of `RedshiftData` construct. | --- ##### `secret`^Required ```typescript public readonly secret: ISecret; ``` - *Type:* aws-cdk-lib.aws_secretsmanager.ISecret The Secrets Manager Secret containing the admin credentials for the Redshift cluster / namespace. --- ##### `clusterId`^Optional ```typescript public readonly clusterId: string; ``` - *Type:* string - *Default:* The `workgroupId` is used The name of the Redshift provisioned to query. It must be configured if the `workgroupId` is not. --- ##### `createInterfaceVpcEndpoint`^Optional ```typescript public readonly createInterfaceVpcEndpoint: boolean; ``` - *Type:* boolean - *Default:* false If set to true, create the Redshift Data Interface VPC Endpoint in the configured VPC/Subnets. --- ##### `executionTimeout`^Optional ```typescript public readonly executionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 5mins The timeout for the query execution. --- ##### `existingInterfaceVPCEndpoint`^Optional ```typescript public readonly existingInterfaceVPCEndpoint: IInterfaceVpcEndpoint; ``` - *Type:* aws-cdk-lib.aws_ec2.IInterfaceVpcEndpoint - *Default:* No security group ingress rule would be created. If this parameter is provided, the data access execution security group would be granted inbound to the interface VPC endpoint's security group. This is assuming that the `createInterfaceVpcEndpoint` parameter is `false`. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `secretKey`^Optional ```typescript public readonly secretKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* no secret key is used The KMS Key used to encrypt the admin credentials for the Redshift cluster / namespace. --- ##### `subnets`^Optional ```typescript public readonly subnets: SelectedSubnets; ``` - *Type:* aws-cdk-lib.aws_ec2.SelectedSubnets - *Default:* No subnets are used. The Custom Resource runs in the Redshift service team subnets. The subnets where the Custom Resource Lambda Function would be created in. A Redshift Data API Interface VPC Endpoint is created in the subnets. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* No VPC is used. The Custom Resource runs in the Redshift service team VPC The VPC where the Custom Resource Lambda Function would be created in. A Redshift Data API Interface VPC Endpoint is created in the VPC. --- ##### `workgroupId`^Optional ```typescript public readonly workgroupId: string; ``` - *Type:* string - *Default:* The `clusterId` is used The `workgroupId` for the Redshift Serverless Workgroup to query. It must be configured if the `clusterId` is not. --- ##### `redshiftData`^Required ```typescript public readonly redshiftData: RedshiftData; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftData Instance of `RedshiftData` construct. --- ### RedshiftNewShareProps Redshift new data share details. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftNewShareProps: consumption.RedshiftNewShareProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | databaseName | string | The database name where the share belongs to. | | dataShareArn | string | The ARN of the data share. | | dataShareName | string | The name of the data share. | | newShareCustomResource | aws-cdk-lib.CustomResource | The custom resource related to the management of the data share. | | producerArn | string | The ARN of the producer. | | producerNamespace | string | The namespace ID of the producer. | --- ##### `databaseName`^Required ```typescript public readonly databaseName: string; ``` - *Type:* string The database name where the share belongs to. --- ##### `dataShareArn`^Required ```typescript public readonly dataShareArn: string; ``` - *Type:* string The ARN of the data share. --- ##### `dataShareName`^Required ```typescript public readonly dataShareName: string; ``` - *Type:* string The name of the data share. --- ##### `newShareCustomResource`^Required ```typescript public readonly newShareCustomResource: CustomResource; ``` - *Type:* aws-cdk-lib.CustomResource The custom resource related to the management of the data share. --- ##### `producerArn`^Required ```typescript public readonly producerArn: string; ``` - *Type:* string The ARN of the producer. --- ##### `producerNamespace`^Required ```typescript public readonly producerNamespace: string; ``` - *Type:* string The namespace ID of the producer. --- ### RedshiftServerlessNamespaceProps RedshiftServerlessNamespace properties. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftServerlessNamespaceProps: consumption.RedshiftServerlessNamespaceProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | dbName | string | The name of the primary database that would be created in the Redshift Serverless Namespace. | | name | string | The name of the Redshift Serverless Namespace. | | adminSecretKey | aws-cdk-lib.aws_kms.Key | The KMS Key used by the managed Secrets Manager Secret storing admin credentials. | | adminUsername | string | The admin username to be used. | | dataKey | aws-cdk-lib.aws_kms.Key | The KMS Key used to encrypt the data. | | defaultIAMRole | aws-cdk-lib.aws_iam.IRole | Default IAM Role associated to the Redshift Serverless Namespace. | | finalSnapshotName | string | If provided, final snapshot would be taken with the name provided. | | finalSnapshotRetentionPeriod | number | The number of days the final snapshot would be retained. | | iamRoles | aws-cdk-lib.aws_iam.IRole[] | List of IAM Roles attached to the Redshift Serverless Namespace. | | logExports | @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespaceLogExport[] | The type of logs to be exported. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | serviceLinkedRoleFactory | @cdklabs/aws-data-solutions-framework.utils.CreateServiceLinkedRole | The Factory for creating Redshift service linked role. | --- ##### `dbName`^Required ```typescript public readonly dbName: string; ``` - *Type:* string The name of the primary database that would be created in the Redshift Serverless Namespace. --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the Redshift Serverless Namespace. --- ##### `adminSecretKey`^Optional ```typescript public readonly adminSecretKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key - *Default:* A new KMS Key is created The KMS Key used by the managed Secrets Manager Secret storing admin credentials. --- ##### `adminUsername`^Optional ```typescript public readonly adminUsername: string; ``` - *Type:* string - *Default:* The default username is "admin" The admin username to be used. --- ##### `dataKey`^Optional ```typescript public readonly dataKey: Key; ``` - *Type:* aws-cdk-lib.aws_kms.Key - *Default:* A new KMS Key is created The KMS Key used to encrypt the data. --- ##### `defaultIAMRole`^Optional ```typescript public readonly defaultIAMRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* No default IAM Role is associated with the Redshift Serverless Namespace Default IAM Role associated to the Redshift Serverless Namespace. --- ##### `finalSnapshotName`^Optional ```typescript public readonly finalSnapshotName: string; ``` - *Type:* string - *Default:* No final snapshot would be taken If provided, final snapshot would be taken with the name provided. --- ##### `finalSnapshotRetentionPeriod`^Optional ```typescript public readonly finalSnapshotRetentionPeriod: number; ``` - *Type:* number - *Default:* Indefinite final snapshot retention The number of days the final snapshot would be retained. Must be between 1-3653 days. --- ##### `iamRoles`^Optional ```typescript public readonly iamRoles: IRole[]; ``` - *Type:* aws-cdk-lib.aws_iam.IRole[] - *Default:* No IAM roles are associated with the Redshift Serverless Namespace List of IAM Roles attached to the Redshift Serverless Namespace. This list of Roles must also contain the `defaultIamRole`. --- ##### `logExports`^Optional ```typescript public readonly logExports: RedshiftServerlessNamespaceLogExport[]; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespaceLogExport[] - *Default:* No logs are exported The type of logs to be exported. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `serviceLinkedRoleFactory`^Optional ```typescript public readonly serviceLinkedRoleFactory: CreateServiceLinkedRole; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.CreateServiceLinkedRole - *Default:* A factory is created The Factory for creating Redshift service linked role. --- ### RedshiftServerlessWorkgroupProps RedshiftServerlessWorkgroup properties. #### Initializer ```typescript import { consumption } from '@cdklabs/aws-data-solutions-framework' const redshiftServerlessWorkgroupProps: consumption.RedshiftServerlessWorkgroupProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | The name of the Redshift Serverless Workgroup. | | namespace | @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespace | The Redshift Serverless Namespace associated with the Workgroup. | | baseCapacity | number | The base capacity of the Redshift Serverless Workgroup in RPU. | | configParameters | aws-cdk-lib.aws_redshiftserverless.CfnWorkgroup.ConfigParameterProperty[] | Additional parameters to set for advanced control over the Redshift Workgroup. | | extraSecurityGroups | aws-cdk-lib.aws_ec2.SecurityGroup[] | The extra EC2 Security Groups to associate with the Redshift Serverless Workgroup (in addition to the primary Security Group). | | port | number | The custom port to use when connecting to workgroup. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets where the Redshift Serverless Workgroup is deployed. | | vpc | aws-cdk-lib.aws_ec2.Vpc | The VPC where the Redshift Serverless Workgroup is deployed. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the Redshift Serverless Workgroup. --- ##### `namespace`^Required ```typescript public readonly namespace: RedshiftServerlessNamespace; ``` - *Type:* @cdklabs/aws-data-solutions-framework.consumption.RedshiftServerlessNamespace The Redshift Serverless Namespace associated with the Workgroup. --- ##### `baseCapacity`^Optional ```typescript public readonly baseCapacity: number; ``` - *Type:* number - *Default:* 128 RPU The base capacity of the Redshift Serverless Workgroup in RPU. --- ##### `configParameters`^Optional ```typescript public readonly configParameters: ConfigParameterProperty[]; ``` - *Type:* aws-cdk-lib.aws_redshiftserverless.CfnWorkgroup.ConfigParameterProperty[] - *Default:* `require_ssl` parameter is set to true. Additional parameters to set for advanced control over the Redshift Workgroup. See {@link https://docs.aws.amazon.com/redshift-serverless/latest/APIReference/API_CreateWorkgroup.html#redshiftserverless-CreateWorkgroup-request-configParameters} for more information on what parameters can be set. --- ##### `extraSecurityGroups`^Optional ```typescript public readonly extraSecurityGroups: SecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.SecurityGroup[] - *Default:* No extra security groups are used The extra EC2 Security Groups to associate with the Redshift Serverless Workgroup (in addition to the primary Security Group). --- ##### `port`^Optional ```typescript public readonly port: number; ``` - *Type:* number - *Default:* 5439 The custom port to use when connecting to workgroup. Valid port ranges are 5431-5455 and 8191-8215. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* Use the private subnets of the VPC The subnets where the Redshift Serverless Workgroup is deployed. --- ##### `vpc`^Optional ```typescript public readonly vpc: Vpc; ``` - *Type:* aws-cdk-lib.aws_ec2.Vpc - *Default:* A default VPC is created The VPC where the Redshift Serverless Workgroup is deployed. --- ### S3DataCopyProps Properties for S3DataCopy construct. #### Initializer ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' const s3DataCopyProps: utils.S3DataCopyProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | sourceBucket | aws-cdk-lib.aws_s3.IBucket | The source S3 Bucket containing the data to copy. | | sourceBucketRegion | string | The source S3 Bucket region. | | targetBucket | aws-cdk-lib.aws_s3.IBucket | The target S3 Bucket. | | executionRole | aws-cdk-lib.aws_iam.IRole | The IAM Role to use in the custom resource for copying data. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | securityGroups | aws-cdk-lib.aws_ec2.ISecurityGroup[] | The list of security groups to attach to the custom resource. | | sourceBucketPrefix | string | The source bucket prefix with a slash at the end. | | subnets | aws-cdk-lib.aws_ec2.SubnetSelection | The subnets to deploy the custom resource in. | | targetBucketPrefix | string | The target S3 Bucket prefix with a slash at the end. | | vpc | aws-cdk-lib.aws_ec2.IVpc | The VPC to deploy the custom resource in. | --- ##### `sourceBucket`^Required ```typescript public readonly sourceBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The source S3 Bucket containing the data to copy. --- ##### `sourceBucketRegion`^Required ```typescript public readonly sourceBucketRegion: string; ``` - *Type:* string The source S3 Bucket region. --- ##### `targetBucket`^Required ```typescript public readonly targetBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The target S3 Bucket. --- ##### `executionRole`^Optional ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A new role is created The IAM Role to use in the custom resource for copying data. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise, the removalPolicy is reverted to RETAIN. --- ##### `securityGroups`^Optional ```typescript public readonly securityGroups: ISecurityGroup[]; ``` - *Type:* aws-cdk-lib.aws_ec2.ISecurityGroup[] - *Default:* If `vpc` is not supplied, no security groups are attached. Otherwise, a dedicated security group is created for each function. The list of security groups to attach to the custom resource. --- ##### `sourceBucketPrefix`^Optional ```typescript public readonly sourceBucketPrefix: string; ``` - *Type:* string - *Default:* No prefix is used The source bucket prefix with a slash at the end. --- ##### `subnets`^Optional ```typescript public readonly subnets: SubnetSelection; ``` - *Type:* aws-cdk-lib.aws_ec2.SubnetSelection - *Default:* The Custom Resource is executed in VPCs owned by AWS Lambda service. The subnets to deploy the custom resource in. --- ##### `targetBucketPrefix`^Optional ```typescript public readonly targetBucketPrefix: string; ``` - *Type:* string - *Default:* No prefix is used The target S3 Bucket prefix with a slash at the end. --- ##### `vpc`^Optional ```typescript public readonly vpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* The Custom Resource is executed in VPCs owned by AWS Lambda service. The VPC to deploy the custom resource in. --- ### S3LoggingConfiguration Details of the Amazon S3 destination for broker logs. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const s3LoggingConfiguration: streaming.S3LoggingConfiguration = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | bucket | aws-cdk-lib.aws_s3.IBucket | The S3 bucket that is the destination for broker logs. | | prefix | string | The S3 prefix that is the destination for broker logs. | --- ##### `bucket`^Required ```typescript public readonly bucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket The S3 bucket that is the destination for broker logs. --- ##### `prefix`^Optional ```typescript public readonly prefix: string; ``` - *Type:* string - *Default:* no prefix The S3 prefix that is the destination for broker logs. --- ### SaslAuthProps SASL authentication properties. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const saslAuthProps: streaming.SaslAuthProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | iam | boolean | Enable IAM access control. | --- ##### `iam`^Optional ```typescript public readonly iam: boolean; ``` - *Type:* boolean - *Default:* false Enable IAM access control. --- ### SaslTlsAuthProps SASL + TLS authentication properties. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const saslTlsAuthProps: streaming.SaslTlsAuthProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | iam | boolean | Enable IAM access control. | | certificateAuthorities | aws-cdk-lib.aws_acmpca.ICertificateAuthority[] | List of ACM Certificate Authorities to enable TLS authentication. | --- ##### `iam`^Optional ```typescript public readonly iam: boolean; ``` - *Type:* boolean - *Default:* false Enable IAM access control. --- ##### `certificateAuthorities`^Optional ```typescript public readonly certificateAuthorities: ICertificateAuthority[]; ``` - *Type:* aws-cdk-lib.aws_acmpca.ICertificateAuthority[] - *Default:* none List of ACM Certificate Authorities to enable TLS authentication. --- ### SaslVpcTlsAuthProps SASL + TLS authentication properties. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const saslVpcTlsAuthProps: streaming.SaslVpcTlsAuthProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | iam | boolean | Enable IAM access control. | | tls | boolean | enable TLS authentication. | --- ##### `iam`^Optional ```typescript public readonly iam: boolean; ``` - *Type:* boolean - *Default:* false Enable IAM access control. --- ##### `tls`^Optional ```typescript public readonly tls: boolean; ``` - *Type:* boolean - *Default:* none enable TLS authentication. --- ### SparkEmrCICDPipelineProps Properties for the `SparkEmrCICDPipeline` construct. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrCICDPipelineProps: processing.SparkEmrCICDPipelineProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | applicationStackFactory | @cdklabs/aws-data-solutions-framework.utils.ApplicationStackFactory | The application CDK Stack to deploy in the different CDK Pipelines Stages. | | source | aws-cdk-lib.pipelines.CodePipelineSource | The connection to allow code pipeline to connect to your code repository You can learn more about connections in this [link](https://docs.aws.amazon.com/dtconsole/latest/userguide/welcome-connections.html). | | sparkApplicationName | string | The name of the Spark application to be deployed. | | cdkApplicationPath | string | The path to the folder that contains the CDK Application. | | integTestEnv | {[ key: string ]: string} | The environment variables to create from the Application CDK Stack outputs and to pass to the integration tests. | | integTestPermissions | aws-cdk-lib.aws_iam.PolicyStatement[] | The IAM Policy statements to add permissions for running the integration tests. | | integTestScript | string | The path to the Shell script that contains integration tests. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | sparkApplicationPath | string | The path to the folder that contains the Spark Application. | | sparkImage | @cdklabs/aws-data-solutions-framework.processing.SparkImage | The EMR Spark image to use to run the unit tests. | --- ##### `applicationStackFactory`^Required ```typescript public readonly applicationStackFactory: ApplicationStackFactory; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ApplicationStackFactory The application CDK Stack to deploy in the different CDK Pipelines Stages. --- ##### `source`^Required ```typescript public readonly source: CodePipelineSource; ``` - *Type:* aws-cdk-lib.pipelines.CodePipelineSource The connection to allow code pipeline to connect to your code repository You can learn more about connections in this [link](https://docs.aws.amazon.com/dtconsole/latest/userguide/welcome-connections.html). --- ##### `sparkApplicationName`^Required ```typescript public readonly sparkApplicationName: string; ``` - *Type:* string The name of the Spark application to be deployed. --- ##### `cdkApplicationPath`^Optional ```typescript public readonly cdkApplicationPath: string; ``` - *Type:* string - *Default:* The root of the repository The path to the folder that contains the CDK Application. --- ##### `integTestEnv`^Optional ```typescript public readonly integTestEnv: {[ key: string ]: string}; ``` - *Type:* {[ key: string ]: string} - *Default:* No environment variables The environment variables to create from the Application CDK Stack outputs and to pass to the integration tests. This is used to interact with resources created by the Application CDK Stack from within the integration tests script. Key is the name of the environment variable to create. Value is generally a CfnOutput name from the Application CDK Stack. --- ##### `integTestPermissions`^Optional ```typescript public readonly integTestPermissions: PolicyStatement[]; ``` - *Type:* aws-cdk-lib.aws_iam.PolicyStatement[] - *Default:* No permissions The IAM Policy statements to add permissions for running the integration tests. --- ##### `integTestScript`^Optional ```typescript public readonly integTestScript: string; ``` - *Type:* string - *Default:* No integration tests are run The path to the Shell script that contains integration tests. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `sparkApplicationPath`^Optional ```typescript public readonly sparkApplicationPath: string; ``` - *Type:* string - *Default:* The root of the repository The path to the folder that contains the Spark Application. --- ##### `sparkImage`^Optional ```typescript public readonly sparkImage: SparkImage; ``` - *Type:* @cdklabs/aws-data-solutions-framework.processing.SparkImage - *Default:* [DEFAULT_SPARK_IMAGE](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/emr-releases.ts#L51) The EMR Spark image to use to run the unit tests. --- ### SparkEmrContainersJobApiProps Configuration for the EMR on EKS job. Use this interface when `SparkEmrContainerJobProps` doesn't give you access to the configuration parameters you need. > [[https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html]]([https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html]) #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrContainersJobApiProps: processing.SparkEmrContainersJobApiProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | schedule | aws-cdk-lib.aws_events.Schedule | The Schedule to run the Step Functions state machine. | | jobConfig | {[ key: string ]: any} | EMR on EKS StartJobRun API configuration. | | executionTimeout | aws-cdk-lib.Duration | Job execution timeout. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `schedule`^Optional ```typescript public readonly schedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* The Step Functions State Machine is not scheduled. The Schedule to run the Step Functions state machine. --- ##### `jobConfig`^Required ```typescript public readonly jobConfig: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} EMR on EKS StartJobRun API configuration. > [[https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html]]([https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html]) --- ##### `executionTimeout`^Optional ```typescript public readonly executionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 30 minutes Job execution timeout. --- ### SparkEmrContainersJobProps Simplified configuration for the `SparkEmrEksJob` construct. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrContainersJobProps: processing.SparkEmrContainersJobProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | schedule | aws-cdk-lib.aws_events.Schedule | The Schedule to run the Step Functions state machine. | | executionRole | aws-cdk-lib.aws_iam.IRole | The IAM execution Role ARN for the EMR on EKS job. | | name | string | The Spark job name. | | sparkSubmitEntryPoint | string | The entry point for the Spark submit job run. | | virtualClusterId | string | The EMR on EKS virtual cluster ID. | | applicationConfiguration | {[ key: string ]: any} | The application configuration override for the Spark submit job run. | | cloudWatchLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group name for log publishing. | | cloudWatchLogGroupStreamPrefix | string | The CloudWatch Log Group stream prefix for log publishing. | | executionTimeout | aws-cdk-lib.Duration | The execution timeout. | | maxRetries | number | The maximum number of retries. | | releaseLabel | @cdklabs/aws-data-solutions-framework.processing.EmrRuntimeVersion | The EMR release version associated with the application. | | s3LogBucket | aws-cdk-lib.aws_s3.IBucket | The S3 Bucket for log publishing. | | s3LogPrefix | string | The S3 Bucket prefix for log publishing. | | sparkSubmitEntryPointArguments | string[] | The arguments for the Spark submit job run. | | sparkSubmitParameters | string | The parameters for the Spark submit job run. | | tags | {[ key: string ]: any} | Tags to be added to the EMR Serverless job. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `schedule`^Optional ```typescript public readonly schedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* The Step Functions State Machine is not scheduled. The Schedule to run the Step Functions state machine. --- ##### `executionRole`^Required ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM execution Role ARN for the EMR on EKS job. --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The Spark job name. --- ##### `sparkSubmitEntryPoint`^Required ```typescript public readonly sparkSubmitEntryPoint: string; ``` - *Type:* string The entry point for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `virtualClusterId`^Required ```typescript public readonly virtualClusterId: string; ``` - *Type:* string The EMR on EKS virtual cluster ID. --- ##### `applicationConfiguration`^Optional ```typescript public readonly applicationConfiguration: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} - *Default:* No configuration is passed to the job. The application configuration override for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `cloudWatchLogGroup`^Optional ```typescript public readonly cloudWatchLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup - *Default:* CloudWatch is not used for logging The CloudWatch Log Group name for log publishing. --- ##### `cloudWatchLogGroupStreamPrefix`^Optional ```typescript public readonly cloudWatchLogGroupStreamPrefix: string; ``` - *Type:* string - *Default:* The application name is used as the prefix The CloudWatch Log Group stream prefix for log publishing. --- ##### `executionTimeout`^Optional ```typescript public readonly executionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 30 minutes The execution timeout. --- ##### `maxRetries`^Optional ```typescript public readonly maxRetries: number; ``` - *Type:* number - *Default:* No retry The maximum number of retries. --- ##### `releaseLabel`^Optional ```typescript public readonly releaseLabel: EmrRuntimeVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.processing.EmrRuntimeVersion - *Default:* [EMR_DEFAULT_VERSION](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/emr-releases.ts#L46) The EMR release version associated with the application. The EMR release can be found in this [documentation](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html) --- ##### `s3LogBucket`^Optional ```typescript public readonly s3LogBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket - *Default:* No logging to S3 The S3 Bucket for log publishing. --- ##### `s3LogPrefix`^Optional ```typescript public readonly s3LogPrefix: string; ``` - *Type:* string - *Default:* No logging to S3 The S3 Bucket prefix for log publishing. --- ##### `sparkSubmitEntryPointArguments`^Optional ```typescript public readonly sparkSubmitEntryPointArguments: string[]; ``` - *Type:* string[] - *Default:* No arguments are passed to the job. The arguments for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `sparkSubmitParameters`^Optional ```typescript public readonly sparkSubmitParameters: string; ``` - *Type:* string - *Default:* No parameters are passed to the job. The parameters for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `tags`^Optional ```typescript public readonly tags: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} - *Default:* No tags are added Tags to be added to the EMR Serverless job. --- ### SparkEmrContainersRuntimeInteractiveSessionProps The properties for the EMR Managed Endpoint to create. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrContainersRuntimeInteractiveSessionProps: processing.SparkEmrContainersRuntimeInteractiveSessionProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | executionRole | aws-cdk-lib.aws_iam.IRole | The Amazon IAM role used as the execution role, this role must provide access to all the AWS resource a user will interact with These can be S3, DynamoDB, Glue Catalog. | | managedEndpointName | string | The name of the EMR managed endpoint. | | virtualClusterId | string | The Id of the Amazon EMR virtual cluster containing the managed endpoint. | | configurationOverrides | any | The JSON object to [override the configuration of EMR on EKS managed endpoint]{@link https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/create-managed-endpoint.html}. | | emrOnEksVersion | @cdklabs/aws-data-solutions-framework.processing.EmrContainersRuntimeVersion | The Amazon EMR version to use. | --- ##### `executionRole`^Required ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The Amazon IAM role used as the execution role, this role must provide access to all the AWS resource a user will interact with These can be S3, DynamoDB, Glue Catalog. --- ##### `managedEndpointName`^Required ```typescript public readonly managedEndpointName: string; ``` - *Type:* string The name of the EMR managed endpoint. --- ##### `virtualClusterId`^Required ```typescript public readonly virtualClusterId: string; ``` - *Type:* string The Id of the Amazon EMR virtual cluster containing the managed endpoint. --- ##### `configurationOverrides`^Optional ```typescript public readonly configurationOverrides: any; ``` - *Type:* any - *Default:* Configuration associated with the [default nodegroup for notebook]{@link EmrEksNodegroup.NOTEBOOK_EXECUTOR } The JSON object to [override the configuration of EMR on EKS managed endpoint]{@link https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/create-managed-endpoint.html}. --- ##### `emrOnEksVersion`^Optional ```typescript public readonly emrOnEksVersion: EmrContainersRuntimeVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.processing.EmrContainersRuntimeVersion - *Default:* The [default Amazon EMR version]{@link EmrEksCluster.DEFAULT_EMR_VERSION } The Amazon EMR version to use. --- ### SparkEmrContainersRuntimeProps The properties for the `SparkEmrContainerRuntime` Construct class. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrContainersRuntimeProps: processing.SparkEmrContainersRuntimeProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | kubectlLambdaLayer | aws-cdk-lib.aws_lambda.ILayerVersion | The Lambda Layer with Kubectl to use for EKS Cluster setup. | | publicAccessCIDRs | string[] | The CIDR blocks that are allowed to access to your cluster’s public Kubernetes API server endpoint. | | createEmrOnEksServiceLinkedRole | boolean | Flag to create an IAM Service Linked Role for EMR on EKS. | | defaultNodes | boolean | The flag to create default Karpenter Node Provisioners for: * Critical jobs which use on-demand instances, high speed disks and workload isolation * Shared workloads which use EC2 Spot Instances and no isolation to optimize costs * Notebooks which leverage a cost optimized configuration for running EMR managed endpoints and spark drivers/executors. | | ec2InstanceRole | aws-cdk-lib.aws_iam.IRole | The IAM Role used for the cluster nodes instance profile. | | eksAdminRole | aws-cdk-lib.aws_iam.IRole | The IAM Role to configure in the EKS master roles. | | eksCluster | aws-cdk-lib.aws_eks.Cluster | The EKS Cluster to setup EMR on. | | eksClusterName | string | The name of the EKS cluster to create. | | eksVpc | aws-cdk-lib.aws_ec2.IVpc | The VPC to use when creating the EKS cluster. | | karpenterVersion | @cdklabs/aws-data-solutions-framework.processing.KarpenterVersion | The Karpenter version to use for autoscaling nodes in the EKS Cluster. | | kubernetesVersion | aws-cdk-lib.aws_eks.KubernetesVersion | The Kubernetes version used to create the EKS Cluster. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | tags | {[ key: string ]: string} | The tags assigned to the EKS cluster. | | vpcCidr | string | The CIDR of the VPC to use when creating the EKS cluster. | --- ##### `kubectlLambdaLayer`^Required ```typescript public readonly kubectlLambdaLayer: ILayerVersion; ``` - *Type:* aws-cdk-lib.aws_lambda.ILayerVersion The Lambda Layer with Kubectl to use for EKS Cluster setup. Starting k8s 1.22, CDK no longer bundle the Kubectl layer with the code due to breaking npm package size. A layer needs to be passed to the Construct. The CDK [documentation](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.KubernetesVersion.html#static-v1_22) contains the libraries that you should add for the right Kubernetes version. --- ##### `publicAccessCIDRs`^Required ```typescript public readonly publicAccessCIDRs: string[]; ``` - *Type:* string[] The CIDR blocks that are allowed to access to your cluster’s public Kubernetes API server endpoint. --- ##### `createEmrOnEksServiceLinkedRole`^Optional ```typescript public readonly createEmrOnEksServiceLinkedRole: boolean; ``` - *Type:* boolean - *Default:* true Flag to create an IAM Service Linked Role for EMR on EKS. --- ##### `defaultNodes`^Optional ```typescript public readonly defaultNodes: boolean; ``` - *Type:* boolean - *Default:* true The flag to create default Karpenter Node Provisioners for: * Critical jobs which use on-demand instances, high speed disks and workload isolation * Shared workloads which use EC2 Spot Instances and no isolation to optimize costs * Notebooks which leverage a cost optimized configuration for running EMR managed endpoints and spark drivers/executors. --- ##### `ec2InstanceRole`^Optional ```typescript public readonly ec2InstanceRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* A role is created with AmazonEKSWorkerNodePolicy, AmazonEC2ContainerRegistryReadOnly, AmazonSSMManagedInstanceCore and AmazonEKS_CNI_Policy AWS managed policies. The IAM Role used for the cluster nodes instance profile. --- ##### `eksAdminRole`^Optional ```typescript public readonly eksAdminRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole - *Default:* An admin role must be passed if `eksCluster` property is not set. The IAM Role to configure in the EKS master roles. It will give access to kubernetes cluster from the AWS console. You will use this role to manage the EKS cluster and grant other access to it. --- ##### `eksCluster`^Optional ```typescript public readonly eksCluster: Cluster; ``` - *Type:* aws-cdk-lib.aws_eks.Cluster - *Default:* An EKS Cluster is created The EKS Cluster to setup EMR on. The cluster needs to be created in the same CDK Stack. If the EKS Cluster is provided, the cluster AddOns and all the controllers (ALB Ingress controller, Karpenter...) need to be configured. When providing an EKS cluster, the methods for adding nodegroups can still be used. They implement the best practices for running Spark on EKS. --- ##### `eksClusterName`^Optional ```typescript public readonly eksClusterName: string; ``` - *Type:* string - *Default:* The [DEFAULT_CLUSTER_NAME](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/spark-runtime/emr-containers/spark-emr-containers-runtime.ts#L65) The name of the EKS cluster to create. --- ##### `eksVpc`^Optional ```typescript public readonly eksVpc: IVpc; ``` - *Type:* aws-cdk-lib.aws_ec2.IVpc - *Default:* A new VPC is created. The VPC to use when creating the EKS cluster. VPC should have at least two private and public subnets in different Availability Zones. All private subnets must have the following tags: * 'for-use-with-amazon-emr-managed-policies'='true' * 'kubernetes.io/role/internal-elb'='1' All public subnets must have the following tag: * 'kubernetes.io/role/elb'='1' Cannot be combined with `vpcCidr`. If combined, `vpcCidr` takes precedence and a new VPC is created. --- ##### `karpenterVersion`^Optional ```typescript public readonly karpenterVersion: KarpenterVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.processing.KarpenterVersion - *Default:* [DEFAULT_KARPENTER_VERSION](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/karpenter-releases.ts#L11) The Karpenter version to use for autoscaling nodes in the EKS Cluster. --- ##### `kubernetesVersion`^Optional ```typescript public readonly kubernetesVersion: KubernetesVersion; ``` - *Type:* aws-cdk-lib.aws_eks.KubernetesVersion - *Default:* [DEFAULT_EKS_VERSION](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/spark-runtime/emr-containers/spark-emr-containers-runtime.ts#L61) The Kubernetes version used to create the EKS Cluster. --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `tags`^Optional ```typescript public readonly tags: {[ key: string ]: string}; ``` - *Type:* {[ key: string ]: string} - *Default:* none The tags assigned to the EKS cluster. --- ##### `vpcCidr`^Optional ```typescript public readonly vpcCidr: string; ``` - *Type:* string - *Default:* The CIDR 10.0.0.0/16 is used The CIDR of the VPC to use when creating the EKS cluster. If provided, a VPC with three public subnets and three private subnets is created. The size of the private subnets is four time the one of the public subnet. --- ### SparkEmrServerlessJobApiProps Configuration for the EMR Serverless Job API. Use this interface when `SparkEmrServerlessJobProps` doesn't give you access to the configuration parameters you need. > [[https://docs.aws.amazon.com/emr-serverless/latest/APIReference/API_StartJobRun.html]]([https://docs.aws.amazon.com/emr-serverless/latest/APIReference/API_StartJobRun.html]) #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrServerlessJobApiProps: processing.SparkEmrServerlessJobApiProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | schedule | aws-cdk-lib.aws_events.Schedule | The Schedule to run the Step Functions state machine. | | jobConfig | {[ key: string ]: any} | EMR Serverless Job Configuration. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `schedule`^Optional ```typescript public readonly schedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* The Step Functions State Machine is not scheduled. The Schedule to run the Step Functions state machine. --- ##### `jobConfig`^Required ```typescript public readonly jobConfig: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} EMR Serverless Job Configuration. > [[https://docs.aws.amazon.com/emr-serverless/latest/APIReference/API_StartJobRun.html]]([https://docs.aws.amazon.com/emr-serverless/latest/APIReference/API_StartJobRun.html]) --- ### SparkEmrServerlessJobProps #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrServerlessJobProps: processing.SparkEmrServerlessJobProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | schedule | aws-cdk-lib.aws_events.Schedule | The Schedule to run the Step Functions state machine. | | applicationId | string | The EMR Serverless Application to execute the Spark Job. | | executionRole | aws-cdk-lib.aws_iam.IRole | The IAM execution Role for the EMR Serverless job. | | name | string | The Spark Job name. | | sparkSubmitEntryPoint | string | The entry point for the Spark submit job run. | | applicationConfiguration | {[ key: string ]: any} | The application configuration override for the Spark submit job run. | | cloudWatchEncryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS Key for encrypting logs on CloudWatch. | | cloudWatchLogGroup | aws-cdk-lib.aws_logs.ILogGroup | The CloudWatch Log Group name for log publishing. | | cloudWatchLogGroupStreamPrefix | string | The CloudWatch Log Group Stream prefix for log publishing. | | cloudWatchLogtypes | string | The types of logs to log in CloudWatch Log. | | executionTimeout | aws-cdk-lib.Duration | The execution timeout. | | persistentAppUi | boolean | Enable Spark persistent UI logs in EMR managed storage. | | persistentAppUIKey | aws-cdk-lib.aws_kms.IKey | The KMS Key ARN to encrypt Spark persistent UI logs in EMR managed storage. | | s3LogBucket | aws-cdk-lib.aws_s3.IBucket | The S3 Bucket for log publishing. | | s3LogEncryptionKey | aws-cdk-lib.aws_kms.IKey | The KMS Key for encrypting logs on S3. | | s3LogPrefix | string | The S3 Bucket prefix for log publishing. | | sparkSubmitEntryPointArguments | string[] | The arguments for the Spark submit job run. | | sparkSubmitParameters | string | The parameters for the Spark submit job run. | | tags | {[ key: string ]: any} | Tags to be added to the EMR Serverless job. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `schedule`^Optional ```typescript public readonly schedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* The Step Functions State Machine is not scheduled. The Schedule to run the Step Functions state machine. --- ##### `applicationId`^Required ```typescript public readonly applicationId: string; ``` - *Type:* string The EMR Serverless Application to execute the Spark Job. --- ##### `executionRole`^Required ```typescript public readonly executionRole: IRole; ``` - *Type:* aws-cdk-lib.aws_iam.IRole The IAM execution Role for the EMR Serverless job. --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The Spark Job name. --- ##### `sparkSubmitEntryPoint`^Required ```typescript public readonly sparkSubmitEntryPoint: string; ``` - *Type:* string The entry point for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `applicationConfiguration`^Optional ```typescript public readonly applicationConfiguration: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} - *Default:* No configuration is passed to the job. The application configuration override for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `cloudWatchEncryptionKey`^Optional ```typescript public readonly cloudWatchEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* No encryption The KMS Key for encrypting logs on CloudWatch. --- ##### `cloudWatchLogGroup`^Optional ```typescript public readonly cloudWatchLogGroup: ILogGroup; ``` - *Type:* aws-cdk-lib.aws_logs.ILogGroup - *Default:* No logging to CloudWatch The CloudWatch Log Group name for log publishing. --- ##### `cloudWatchLogGroupStreamPrefix`^Optional ```typescript public readonly cloudWatchLogGroupStreamPrefix: string; ``` - *Type:* string - *Default:* No prefix is used The CloudWatch Log Group Stream prefix for log publishing. --- ##### `cloudWatchLogtypes`^Optional ```typescript public readonly cloudWatchLogtypes: string; ``` - *Type:* string The types of logs to log in CloudWatch Log. --- ##### `executionTimeout`^Optional ```typescript public readonly executionTimeout: Duration; ``` - *Type:* aws-cdk-lib.Duration - *Default:* 30 minutes The execution timeout. --- ##### `persistentAppUi`^Optional ```typescript public readonly persistentAppUi: boolean; ``` - *Type:* boolean - *Default:* true Enable Spark persistent UI logs in EMR managed storage. --- ##### `persistentAppUIKey`^Optional ```typescript public readonly persistentAppUIKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* Use EMR managed Key The KMS Key ARN to encrypt Spark persistent UI logs in EMR managed storage. --- ##### `s3LogBucket`^Optional ```typescript public readonly s3LogBucket: IBucket; ``` - *Type:* aws-cdk-lib.aws_s3.IBucket - *Default:* No logging to S3 The S3 Bucket for log publishing. --- ##### `s3LogEncryptionKey`^Optional ```typescript public readonly s3LogEncryptionKey: IKey; ``` - *Type:* aws-cdk-lib.aws_kms.IKey - *Default:* No encryption The KMS Key for encrypting logs on S3. --- ##### `s3LogPrefix`^Optional ```typescript public readonly s3LogPrefix: string; ``` - *Type:* string - *Default:* No logging to S3 The S3 Bucket prefix for log publishing. --- ##### `sparkSubmitEntryPointArguments`^Optional ```typescript public readonly sparkSubmitEntryPointArguments: string[]; ``` - *Type:* string[] - *Default:* No arguments are passed to the job. The arguments for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `sparkSubmitParameters`^Optional ```typescript public readonly sparkSubmitParameters: string; ``` - *Type:* string - *Default:* No parameters are passed to the job. The parameters for the Spark submit job run. > [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_StartJobRun.html) --- ##### `tags`^Optional ```typescript public readonly tags: {[ key: string ]: any}; ``` - *Type:* {[ key: string ]: any} - *Default:* No tags are added Tags to be added to the EMR Serverless job. --- ### SparkEmrServerlessRuntimeProps Properties for the `SparkEmrServerlessRuntime` construct. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkEmrServerlessRuntimeProps: processing.SparkEmrServerlessRuntimeProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | name | string | The name of the application. | | architecture | @cdklabs/aws-data-solutions-framework.utils.Architecture | The CPU architecture type of the application. | | autoStartConfiguration | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.AutoStartConfigurationProperty | The configuration for an application to automatically start on job submission. | | autoStopConfiguration | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.AutoStopConfigurationProperty | The configuration for an application to automatically stop after a certain amount of time being idle. | | imageConfiguration | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.ImageConfigurationInputProperty | The unique custom image configuration used for both the Spark Driver and the Spark Executor. | | initialCapacity |

aws-cdk-lib.IResolvable \| aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.InitialCapacityConfigKeyValuePairProperty[]

| The pre-initialized capacity of the application. | | maximumCapacity | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.MaximumAllowedResourcesProperty | The maximum capacity of the application. | | networkConfiguration | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.NetworkConfigurationProperty | The network configuration for customer VPC connectivity for the application. | | releaseLabel | @cdklabs/aws-data-solutions-framework.processing.EmrRuntimeVersion | The EMR release version associated with the application. | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | runtimeConfiguration | aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.ConfigurationObjectProperty[] | The runtime and monitoring configurations to used as defaults for all of the job runs of this application. | | workerTypeSpecifications |

aws-cdk-lib.IResolvable \| {[ key: string ]: aws-cdk-lib.IResolvable \| aws-cdk-lib.aws_emrserverless.CfnApplication.WorkerTypeSpecificationInputProperty}

| The different custom image configurations used for the Spark Driver and the Spark Executor. | --- ##### `name`^Required ```typescript public readonly name: string; ``` - *Type:* string The name of the application. The name must be less than 64 characters. *Pattern* : `^[A-Za-z0-9._\\/#-]+$` --- ##### `architecture`^Optional ```typescript public readonly architecture: Architecture; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.Architecture - *Default:* x86_64 The CPU architecture type of the application. --- ##### `autoStartConfiguration`^Optional ```typescript public readonly autoStartConfiguration: IResolvable | AutoStartConfigurationProperty; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.AutoStartConfigurationProperty - *Default:* True The configuration for an application to automatically start on job submission. --- ##### `autoStopConfiguration`^Optional ```typescript public readonly autoStopConfiguration: IResolvable | AutoStopConfigurationProperty; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.AutoStopConfigurationProperty - *Default:* The application is stopped after 15 minutes of idle time The configuration for an application to automatically stop after a certain amount of time being idle. --- ##### `imageConfiguration`^Optional ```typescript public readonly imageConfiguration: IResolvable | ImageConfigurationInputProperty; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.ImageConfigurationInputProperty - *Default:* EMR base image is used for both the Spark Driver and the Spark Executor The unique custom image configuration used for both the Spark Driver and the Spark Executor. > [https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html) --- ##### `initialCapacity`^Optional ```typescript public readonly initialCapacity: IResolvable | IResolvable | InitialCapacityConfigKeyValuePairProperty[]; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.InitialCapacityConfigKeyValuePairProperty[] - *Default:* No pre-initialized capacity is used The pre-initialized capacity of the application. > [https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/pre-init-capacity.html](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/pre-init-capacity.html) --- ##### `maximumCapacity`^Optional ```typescript public readonly maximumCapacity: IResolvable | MaximumAllowedResourcesProperty; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.MaximumAllowedResourcesProperty - *Default:* Depending on the EMR version The maximum capacity of the application. This is cumulative across all workers at any given point in time during the lifespan of the application is created. No new resources will be created once any one of the defined limits is hit. --- ##### `networkConfiguration`^Optional ```typescript public readonly networkConfiguration: IResolvable | NetworkConfigurationProperty; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.NetworkConfigurationProperty - *Default:* a VPC and a security group are created, these are accessed as construct attribute. The network configuration for customer VPC connectivity for the application. If no configuration is created, the a VPC with 3 public subnets and 3 private subnets is created The 3 public subnets and 3 private subnets are each created in an Availability Zone (AZ) The VPC has one NAT Gateway per AZ and an S3 endpoint --- ##### `releaseLabel`^Optional ```typescript public readonly releaseLabel: EmrRuntimeVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.processing.EmrRuntimeVersion - *Default:* [EMR_DEFAULT_VERSION](https://github.com/awslabs/data-solutions-framework-on-aws/blob/HEAD/framework/src/processing/lib/emr-releases.ts#L46) The EMR release version associated with the application. The EMR release can be found in this [documentation](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html) --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `runtimeConfiguration`^Optional ```typescript public readonly runtimeConfiguration: IResolvable | ConfigurationObjectProperty[]; ``` - *Type:* aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.ConfigurationObjectProperty[] - *Default:* No custom configuration is used The runtime and monitoring configurations to used as defaults for all of the job runs of this application. > [https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/default-configs.html](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/default-configs.html) --- ##### `workerTypeSpecifications`^Optional ```typescript public readonly workerTypeSpecifications: IResolvable | {[ key: string ]: IResolvable | WorkerTypeSpecificationInputProperty}; ``` - *Type:* aws-cdk-lib.IResolvable | {[ key: string ]: aws-cdk-lib.IResolvable | aws-cdk-lib.aws_emrserverless.CfnApplication.WorkerTypeSpecificationInputProperty} - *Default:* EMR base image is used for both the Spark Driver and the Spark Executor The different custom image configurations used for the Spark Driver and the Spark Executor. > [https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/application-custom-image.html) --- ### SparkJobProps The properties for the `SparkJob` construct. #### Initializer ```typescript import { processing } from '@cdklabs/aws-data-solutions-framework' const sparkJobProps: processing.SparkJobProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | removalPolicy | aws-cdk-lib.RemovalPolicy | The removal policy when deleting the CDK resource. | | schedule | aws-cdk-lib.aws_events.Schedule | The Schedule to run the Step Functions state machine. | --- ##### `removalPolicy`^Optional ```typescript public readonly removalPolicy: RemovalPolicy; ``` - *Type:* aws-cdk-lib.RemovalPolicy - *Default:* The resources are not deleted (`RemovalPolicy.RETAIN`). The removal policy when deleting the CDK resource. If DESTROY is selected, context value `@data-solutions-framework-on-aws/removeDataOnDestroy` needs to be set to true. Otherwise the removalPolicy is reverted to RETAIN. --- ##### `schedule`^Optional ```typescript public readonly schedule: Schedule; ``` - *Type:* aws-cdk-lib.aws_events.Schedule - *Default:* The Step Functions State Machine is not scheduled. The Schedule to run the Step Functions state machine. --- ### TlsAuthProps TLS authentication properties. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const tlsAuthProps: streaming.TlsAuthProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | certificateAuthorities | aws-cdk-lib.aws_acmpca.ICertificateAuthority[] | List of ACM Certificate Authorities to enable TLS authentication. | --- ##### `certificateAuthorities`^Optional ```typescript public readonly certificateAuthorities: ICertificateAuthority[]; ``` - *Type:* aws-cdk-lib.aws_acmpca.ICertificateAuthority[] - *Default:* none List of ACM Certificate Authorities to enable TLS authentication. --- ### VpcTlsAuthProps TLS authentication properties. #### Initializer ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' const vpcTlsAuthProps: streaming.VpcTlsAuthProps = { ... } ``` #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | tls | boolean | enable TLS authentication. | --- ##### `tls`^Optional ```typescript public readonly tls: boolean; ``` - *Type:* boolean - *Default:* none enable TLS authentication. --- ## Classes ### ApplicationStackFactory Abstract class that needs to be implemented to pass the application Stack to the CICD pipeline. *Example* ```typescript interface MyApplicationStackProps extends cdk.StackProps { readonly stage: dsf.utils.CICDStage; } class MyApplicationStack extends cdk.Stack { constructor(scope: Construct, id: string, props?: MyApplicationStackProps) { super(scope, id, props); // stack logic goes here... and can be customized using props.stage } } class MyApplicationStackFactory extends dsf.utils.ApplicationStackFactory { createStack(scope: Construct, stage: dsf.utils.CICDStage): cdk.Stack { return new MyApplicationStack(scope, 'MyApplication', { stage: stage } as MyApplicationStackProps); } } ``` #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.ApplicationStackFactory() ``` | **Name** | **Type** | **Description** | | --- | --- | --- | --- #### Methods | **Name** | **Description** | | --- | --- | | createStack | Abstract method that needs to be implemented to return the application Stack. | --- ##### `createStack` ```typescript public createStack(scope: Construct, stage: CICDStage): Stack ``` Abstract method that needs to be implemented to return the application Stack. ###### `scope`^Required - *Type:* constructs.Construct The scope to create the stack in. --- ###### `stage`^Required - *Type:* @cdklabs/aws-data-solutions-framework.utils.CICDStage The stage of the pipeline. --- ### BucketUtils Utils for working with Amazon S3 Buckets. #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.BucketUtils() ``` | **Name** | **Type** | **Description** | | --- | --- | --- | --- #### Static Functions | **Name** | **Description** | | --- | --- | | generateUniqueBucketName | Generate a unique Amazon S3 bucket name based on the provided name, CDK construct ID and CDK construct scope. | --- ##### `generateUniqueBucketName` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.BucketUtils.generateUniqueBucketName(scope: Construct, id: string, name: string) ``` Generate a unique Amazon S3 bucket name based on the provided name, CDK construct ID and CDK construct scope. The bucket name is suffixed the AWS account ID, the AWS region and a unique 8 characters hash. The maximum length for name is 26 characters. ###### `scope`^Required - *Type:* constructs.Construct the current scope where the construct is created (generally `this`). --- ###### `id`^Required - *Type:* string the CDK ID of the construct. --- ###### `name`^Required - *Type:* string the name of the bucket. --- ### CICDStage The list of CICD Stages used in CICD Pipelines. #### Static Functions | **Name** | **Description** | | --- | --- | | of | Custom stage. | --- ##### `of` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.CICDStage.of(stage: string) ``` Custom stage. ###### `stage`^Required - *Type:* string the stage inside the pipeline. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | stage | string | the stage inside the pipeline. | --- ##### `stage`^Required ```typescript public readonly stage: string; ``` - *Type:* string the stage inside the pipeline. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | PROD | @cdklabs/aws-data-solutions-framework.utils.CICDStage | Prod stage. | | STAGING | @cdklabs/aws-data-solutions-framework.utils.CICDStage | Staging stage. | --- ##### `PROD`^Required ```typescript public readonly PROD: CICDStage; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.CICDStage Prod stage. --- ##### `STAGING`^Required ```typescript public readonly STAGING: CICDStage; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.CICDStage Staging stage. --- ### ClientAuthentication Configuration properties for client authentication. #### Static Functions | **Name** | **Description** | | --- | --- | | sasl | SASL authentication. | | saslTls | SASL + TLS authentication. | | tls | TLS authentication. | --- ##### `sasl` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.ClientAuthentication.sasl(props: SaslAuthProps) ``` SASL authentication. ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps --- ##### `saslTls` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.ClientAuthentication.saslTls(saslTlsProps: SaslTlsAuthProps) ``` SASL + TLS authentication. ###### `saslTlsProps`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslTlsAuthProps --- ##### `tls` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.ClientAuthentication.tls(props: TlsAuthProps) ``` TLS authentication. ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.TlsAuthProps --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | saslProps | @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps | - properties for SASL authentication. | | tlsProps | @cdklabs/aws-data-solutions-framework.streaming.TlsAuthProps | - properties for TLS authentication. | --- ##### `saslProps`^Optional ```typescript public readonly saslProps: SaslAuthProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps properties for SASL authentication. --- ##### `tlsProps`^Optional ```typescript public readonly tlsProps: TlsAuthProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.TlsAuthProps properties for TLS authentication. --- ### DataZoneHelpers #### Initializers ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' new governance.DataZoneHelpers() ``` | **Name** | **Type** | **Description** | | --- | --- | --- | --- #### Static Functions | **Name** | **Description** | | --- | --- | | buildModelString | Build a Smithy model string from model fields. | | createSubscriptionTarget | Creates a DataZone subscription target for a custom asset type. | --- ##### `buildModelString` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneHelpers.buildModelString(formType: DataZoneFormType) ``` Build a Smithy model string from model fields. ###### `formType`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.DataZoneFormType The form type containing the model fields. --- ##### `createSubscriptionTarget` ```typescript import { governance } from '@cdklabs/aws-data-solutions-framework' governance.DataZoneHelpers.createSubscriptionTarget(scope: Construct, id: string, customAssetType: CustomAssetType, name: string, provider: string, environmentId: string, authorizedPrincipals: IRole[], manageAccessRole: IRole) ``` Creates a DataZone subscription target for a custom asset type. Subscription targets are used to automatically add asset to environments when a custom asset is subscribed by a project. ###### `scope`^Required - *Type:* constructs.Construct The scope of the construct. --- ###### `id`^Required - *Type:* string The id of the construct. --- ###### `customAssetType`^Required - *Type:* @cdklabs/aws-data-solutions-framework.governance.CustomAssetType The custom asset type that can be added to the environment. --- ###### `name`^Required - *Type:* string The name of the subscription target. --- ###### `provider`^Required - *Type:* string The provider of the subscription target. --- ###### `environmentId`^Required - *Type:* string The DataZone environment identifier. --- ###### `authorizedPrincipals`^Required - *Type:* aws-cdk-lib.aws_iam.IRole[] The authorized principals to be granted when assets are subscribed. --- ###### `manageAccessRole`^Required - *Type:* aws-cdk-lib.aws_iam.IRole The IAM role creating the subscription target. --- ### KafkaVersion Kafka cluster version. #### Static Functions | **Name** | **Description** | | --- | --- | | of | Custom cluster version. | --- ##### `of` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.KafkaVersion.of(version: string) ``` Custom cluster version. ###### `version`^Required - *Type:* string custom version number. --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | version | string | cluster version number. | --- ##### `version`^Required ```typescript public readonly version: string; ``` - *Type:* string cluster version number. --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | V1_1_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**. | | V2_2_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.2.1. | | V2_3_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.3.1. | | V2_4_1_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.4.1. | | V2_5_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.5.1. | | V2_6_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.6.0. | | V2_6_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.6.1. | | V2_6_2 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.6.2. | | V2_6_3 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.6.3. | | V2_7_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.7.0. | | V2_7_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.7.1. | | V2_7_2 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.7.2. | | V2_8_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.8.0. | | V2_8_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 2.8.1. | | V2_8_2_TIERED | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | AWS MSK Kafka version 2.8.2.tiered. | | V3_1_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.1.1. | | V3_2_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.2.0. | | V3_3_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.3.1. | | V3_3_2 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.3.2. | | V3_4_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.4.0. | | V3_5_1 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.5.1. | | V3_6_0 | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.6.0. | | V3_7_X | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.7.x. | | V3_7_X_KRAFT | @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion | Kafka version 3.7.x.kraft. | --- ##### ~~`V1_1_1`~~^Required - *Deprecated:* use the latest runtime instead ```typescript public readonly V1_1_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion **Deprecated by Amazon MSK. You can't create a Kafka cluster with a deprecated version.**. Kafka version 1.1.1 --- ##### `V2_2_1`^Required ```typescript public readonly V2_2_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.2.1. --- ##### `V2_3_1`^Required ```typescript public readonly V2_3_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.3.1. --- ##### `V2_4_1_1`^Required ```typescript public readonly V2_4_1_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.4.1. --- ##### `V2_5_1`^Required ```typescript public readonly V2_5_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.5.1. --- ##### `V2_6_0`^Required ```typescript public readonly V2_6_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.6.0. --- ##### `V2_6_1`^Required ```typescript public readonly V2_6_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.6.1. --- ##### `V2_6_2`^Required ```typescript public readonly V2_6_2: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.6.2. --- ##### `V2_6_3`^Required ```typescript public readonly V2_6_3: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.6.3. --- ##### `V2_7_0`^Required ```typescript public readonly V2_7_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.7.0. --- ##### `V2_7_1`^Required ```typescript public readonly V2_7_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.7.1. --- ##### `V2_7_2`^Required ```typescript public readonly V2_7_2: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.7.2. --- ##### `V2_8_0`^Required ```typescript public readonly V2_8_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.8.0. --- ##### `V2_8_1`^Required ```typescript public readonly V2_8_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 2.8.1. --- ##### `V2_8_2_TIERED`^Required ```typescript public readonly V2_8_2_TIERED: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion AWS MSK Kafka version 2.8.2.tiered. --- ##### `V3_1_1`^Required ```typescript public readonly V3_1_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.1.1. --- ##### `V3_2_0`^Required ```typescript public readonly V3_2_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.2.0. --- ##### `V3_3_1`^Required ```typescript public readonly V3_3_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.3.1. --- ##### `V3_3_2`^Required ```typescript public readonly V3_3_2: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.3.2. --- ##### `V3_4_0`^Required ```typescript public readonly V3_4_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.4.0. --- ##### `V3_5_1`^Required ```typescript public readonly V3_5_1: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.5.1. --- ##### `V3_6_0`^Required ```typescript public readonly V3_6_0: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.6.0. --- ##### `V3_7_X`^Required ```typescript public readonly V3_7_X: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.7.x. --- ##### `V3_7_X_KRAFT`^Required ```typescript public readonly V3_7_X_KRAFT: KafkaVersion; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.KafkaVersion Kafka version 3.7.x.kraft. --- ### MskBrokerInstanceType Kafka cluster version. #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | instance | aws-cdk-lib.aws_ec2.InstanceType | *No description.* | --- ##### `instance`^Required ```typescript public readonly instance: InstanceType; ``` - *Type:* aws-cdk-lib.aws_ec2.InstanceType --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | KAFKA_M5_12XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.12xlarge. | | KAFKA_M5_16XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.16xlarge. | | KAFKA_M5_24XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.24xlarge. | | KAFKA_M5_2XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.2xlarge. | | KAFKA_M5_4XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.4xlarge. | | KAFKA_M5_8XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.8xlarge. | | KAFKA_M5_LARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.large. | | KAFKA_M5_XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m5.xlarge. | | KAFKA_M7G_12XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.12xlarge. | | KAFKA_M7G_16XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.16xlarge. | | KAFKA_M7G_24XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.24xlarge. | | KAFKA_M7G_2XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.2xlarge. | | KAFKA_M7G_4XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.4xlarge. | | KAFKA_M7G_8XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.8xlarge. | | KAFKA_M7G_LARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.large. | | KAFKA_M7G_XLARGE | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.m7g.xlarge. | | KAFKA_T3_SMALL | @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType | Borker instance type kafka.t3.small. | --- ##### `KAFKA_M5_12XLARGE`^Required ```typescript public readonly KAFKA_M5_12XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.12xlarge. --- ##### `KAFKA_M5_16XLARGE`^Required ```typescript public readonly KAFKA_M5_16XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.16xlarge. --- ##### `KAFKA_M5_24XLARGE`^Required ```typescript public readonly KAFKA_M5_24XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.24xlarge. --- ##### `KAFKA_M5_2XLARGE`^Required ```typescript public readonly KAFKA_M5_2XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.2xlarge. --- ##### `KAFKA_M5_4XLARGE`^Required ```typescript public readonly KAFKA_M5_4XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.4xlarge. --- ##### `KAFKA_M5_8XLARGE`^Required ```typescript public readonly KAFKA_M5_8XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.8xlarge. --- ##### `KAFKA_M5_LARGE`^Required ```typescript public readonly KAFKA_M5_LARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.large. --- ##### `KAFKA_M5_XLARGE`^Required ```typescript public readonly KAFKA_M5_XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m5.xlarge. --- ##### `KAFKA_M7G_12XLARGE`^Required ```typescript public readonly KAFKA_M7G_12XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.12xlarge. --- ##### `KAFKA_M7G_16XLARGE`^Required ```typescript public readonly KAFKA_M7G_16XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.16xlarge. --- ##### `KAFKA_M7G_24XLARGE`^Required ```typescript public readonly KAFKA_M7G_24XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.24xlarge. --- ##### `KAFKA_M7G_2XLARGE`^Required ```typescript public readonly KAFKA_M7G_2XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.2xlarge. --- ##### `KAFKA_M7G_4XLARGE`^Required ```typescript public readonly KAFKA_M7G_4XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.4xlarge. --- ##### `KAFKA_M7G_8XLARGE`^Required ```typescript public readonly KAFKA_M7G_8XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.8xlarge. --- ##### `KAFKA_M7G_LARGE`^Required ```typescript public readonly KAFKA_M7G_LARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.large. --- ##### `KAFKA_M7G_XLARGE`^Required ```typescript public readonly KAFKA_M7G_XLARGE: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.m7g.xlarge. --- ##### `KAFKA_T3_SMALL`^Required ```typescript public readonly KAFKA_T3_SMALL: MskBrokerInstanceType; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.MskBrokerInstanceType Borker instance type kafka.t3.small. --- ### ServiceLinkedRoleService #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.ServiceLinkedRoleService(serviceName: string, roleName: string) ``` | **Name** | **Type** | **Description** | | --- | --- | --- | | serviceName | string | *No description.* | | roleName | string | *No description.* | --- ##### `serviceName`^Required - *Type:* string --- ##### `roleName`^Required - *Type:* string --- #### Methods | **Name** | **Description** | | --- | --- | | getCreateServiceLinkedRolePolicy | *No description.* | | getRoleArn | *No description.* | --- ##### `getCreateServiceLinkedRolePolicy` ```typescript public getCreateServiceLinkedRolePolicy(account: string): PolicyStatement ``` ###### `account`^Required - *Type:* string --- ##### `getRoleArn` ```typescript public getRoleArn(account: string): string ``` ###### `account`^Required - *Type:* string --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | roleName | string | *No description.* | | serviceName | string | *No description.* | --- ##### `roleName`^Required ```typescript public readonly roleName: string; ``` - *Type:* string --- ##### `serviceName`^Required ```typescript public readonly serviceName: string; ``` - *Type:* string --- #### Constants | **Name** | **Type** | **Description** | | --- | --- | --- | | EKS | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | | EMR_CONTAINERS | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | | EMR_SERVERLESS | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | | KAFKA | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | | OPENSEARCH | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | | REDSHIFT | @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService | *No description.* | --- ##### `EKS`^Required ```typescript public readonly EKS: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ##### `EMR_CONTAINERS`^Required ```typescript public readonly EMR_CONTAINERS: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ##### `EMR_SERVERLESS`^Required ```typescript public readonly EMR_SERVERLESS: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ##### `KAFKA`^Required ```typescript public readonly KAFKA: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ##### `OPENSEARCH`^Required ```typescript public readonly OPENSEARCH: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ##### `REDSHIFT`^Required ```typescript public readonly REDSHIFT: ServiceLinkedRoleService; ``` - *Type:* @cdklabs/aws-data-solutions-framework.utils.ServiceLinkedRoleService --- ### StepFunctionUtils Utils for working with AWS Step Functions. #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.StepFunctionUtils() ``` | **Name** | **Type** | **Description** | | --- | --- | --- | --- #### Static Functions | **Name** | **Description** | | --- | --- | | camelToPascal | Convert camel case properties to pascal case as required by AWS Step Functions API. | --- ##### `camelToPascal` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.StepFunctionUtils.camelToPascal(config: {[ key: string ]: any}) ``` Convert camel case properties to pascal case as required by AWS Step Functions API. ###### `config`^Required - *Type:* {[ key: string ]: any} --- ### Utils Utilities class used across the different resources. #### Initializers ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' new utils.Utils() ``` | **Name** | **Type** | **Description** | | --- | --- | --- | --- #### Static Functions | **Name** | **Description** | | --- | --- | | generateHash | Generate an 8 character hash from a string based on HMAC algorithm. | | generateUniqueHash | Generate a unique hash of 8 characters from the CDK scope using its path and the stack name. | | getCdkDeploymentRole | Get CDK deployment role. | | loadYaml | Take a document stored as string and load it as YAML. | | randomize | Create a random string to be used as a seed for IAM User password. | | readYamlDocument | Read a YAML file from the path provided and return it. | | stringSanitizer | Sanitize a string by removing upper case and replacing special characters except underscore. | | toPascalCase | Convert a string to PascalCase. | | validateAccountId | Validate a provided string is a valid account ID. | --- ##### `generateHash` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.generateHash(text: string) ``` Generate an 8 character hash from a string based on HMAC algorithm. ###### `text`^Required - *Type:* string the text to hash. --- ##### `generateUniqueHash` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.generateUniqueHash(scope: Construct, id?: string) ``` Generate a unique hash of 8 characters from the CDK scope using its path and the stack name. ###### `scope`^Required - *Type:* constructs.Construct the CDK construct scope. --- ###### `id`^Optional - *Type:* string the CDK ID of the construct. --- ##### `getCdkDeploymentRole` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.getCdkDeploymentRole(scope: Construct) ``` Get CDK deployment role. ###### `scope`^Required - *Type:* constructs.Construct --- ##### `loadYaml` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.loadYaml(document: string) ``` Take a document stored as string and load it as YAML. ###### `document`^Required - *Type:* string the document stored as string. --- ##### `randomize` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.randomize(name: string) ``` Create a random string to be used as a seed for IAM User password. ###### `name`^Required - *Type:* string the string to which to append a random string. --- ##### `readYamlDocument` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.readYamlDocument(path: string) ``` Read a YAML file from the path provided and return it. ###### `path`^Required - *Type:* string the path to the file. --- ##### `stringSanitizer` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.stringSanitizer(toSanitize: string) ``` Sanitize a string by removing upper case and replacing special characters except underscore. ###### `toSanitize`^Required - *Type:* string the string to sanitize. --- ##### `toPascalCase` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.toPascalCase(text: string) ``` Convert a string to PascalCase. ###### `text`^Required - *Type:* string the string to convert to PascalCase. --- ##### `validateAccountId` ```typescript import { utils } from '@cdklabs/aws-data-solutions-framework' utils.Utils.validateAccountId(accountId: string) ``` Validate a provided string is a valid account ID. ###### `accountId`^Required - *Type:* string the account ID to validate. --- ### VpcClientAuthentication Configuration properties for VPC client authentication. #### Static Functions | **Name** | **Description** | | --- | --- | | sasl | SASL authentication. | | saslTls | SASL + TLS authentication. | | tls | TLS authentication. | --- ##### `sasl` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.VpcClientAuthentication.sasl(props: SaslAuthProps) ``` SASL authentication. ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps --- ##### `saslTls` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.VpcClientAuthentication.saslTls(saslTlsProps: SaslVpcTlsAuthProps) ``` SASL + TLS authentication. ###### `saslTlsProps`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslVpcTlsAuthProps --- ##### `tls` ```typescript import { streaming } from '@cdklabs/aws-data-solutions-framework' streaming.VpcClientAuthentication.tls(props: VpcTlsAuthProps) ``` TLS authentication. ###### `props`^Required - *Type:* @cdklabs/aws-data-solutions-framework.streaming.VpcTlsAuthProps --- #### Properties | **Name** | **Type** | **Description** | | --- | --- | --- | | saslProps | @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps | - properties for SASL authentication. | | tlsProps | @cdklabs/aws-data-solutions-framework.streaming.VpcTlsAuthProps | - properties for TLS authentication. | --- ##### `saslProps`^Optional ```typescript public readonly saslProps: SaslAuthProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.SaslAuthProps properties for SASL authentication. --- ##### `tlsProps`^Optional ```typescript public readonly tlsProps: VpcTlsAuthProps; ``` - *Type:* @cdklabs/aws-data-solutions-framework.streaming.VpcTlsAuthProps properties for TLS authentication. --- ## Enums ### AclOperationTypes #### Members | **Name** | **Description** | | --- | --- | | UNKNOWN | *No description.* | | ANY | *No description.* | | ALL | *No description.* | | READ | *No description.* | | WRITE | *No description.* | | CREATE | *No description.* | | DELETE | *No description.* | | ALTER | *No description.* | | DESCRIBE | *No description.* | | CLUSTER_ACTION | *No description.* | | DESCRIBE_CONFIGS | *No description.* | | ALTER_CONFIGS | *No description.* | | IDEMPOTENT_WRITE | *No description.* | --- ##### `UNKNOWN` --- ##### `ANY` --- ##### `ALL` --- ##### `READ` --- ##### `WRITE` --- ##### `CREATE` --- ##### `DELETE` --- ##### `ALTER` --- ##### `DESCRIBE` --- ##### `CLUSTER_ACTION` --- ##### `DESCRIBE_CONFIGS` --- ##### `ALTER_CONFIGS` --- ##### `IDEMPOTENT_WRITE` --- ### AclPermissionTypes #### Members | **Name** | **Description** | | --- | --- | | UNKNOWN | *No description.* | | ANY | *No description.* | | DENY | *No description.* | | ALLOW | *No description.* | --- ##### `UNKNOWN` --- ##### `ANY` --- ##### `DENY` --- ##### `ALLOW` --- ### AclResourceTypes #### Members | **Name** | **Description** | | --- | --- | | UNKNOWN | *No description.* | | ANY | *No description.* | | TOPIC | *No description.* | | GROUP | *No description.* | | CLUSTER | *No description.* | | TRANSACTIONAL_ID | *No description.* | | DELEGATION_TOKEN | *No description.* | --- ##### `UNKNOWN` --- ##### `ANY` --- ##### `TOPIC` --- ##### `GROUP` --- ##### `CLUSTER` --- ##### `TRANSACTIONAL_ID` --- ##### `DELEGATION_TOKEN` --- ### Architecture List of supported CPU architecture, either X86_64 or ARM64. #### Members | **Name** | **Description** | | --- | --- | | X86_64 | *No description.* | | ARM64 | *No description.* | --- ##### `X86_64` --- ##### `ARM64` --- ### Authentication #### Members | **Name** | **Description** | | --- | --- | | IAM | *No description.* | | MTLS | *No description.* | --- ##### `IAM` --- ##### `MTLS` --- ### ClusterMonitoringLevel The level of monitoring for the MSK cluster. > [https://docs.aws.amazon.com/msk/latest/developerguide/monitoring.html#metrics-details](https://docs.aws.amazon.com/msk/latest/developerguide/monitoring.html#metrics-details) #### Members | **Name** | **Description** | | --- | --- | | DEFAULT | Default metrics are the essential metrics to monitor. | | PER_BROKER | Per Broker metrics give you metrics at the broker level. | | PER_TOPIC_PER_BROKER | Per Topic Per Broker metrics help you understand volume at the topic level. | | PER_TOPIC_PER_PARTITION | Per Topic Per Partition metrics help you understand consumer group lag at the topic partition level. | --- ##### `DEFAULT` Default metrics are the essential metrics to monitor. --- ##### `PER_BROKER` Per Broker metrics give you metrics at the broker level. --- ##### `PER_TOPIC_PER_BROKER` Per Topic Per Broker metrics help you understand volume at the topic level. --- ##### `PER_TOPIC_PER_PARTITION` Per Topic Per Partition metrics help you understand consumer group lag at the topic partition level. --- ### EmrContainersRuntimeVersion #### Members | **Name** | **Description** | | --- | --- | | V7_5 | *No description.* | | V7_4 | *No description.* | | V7_3 | *No description.* | | V7_2 | *No description.* | | V7_1 | *No description.* | | V7_0 | *No description.* | | V6_15 | *No description.* | | V6_14 | *No description.* | | V6_13 | *No description.* | | V6_12 | *No description.* | | V6_11_1 | *No description.* | | V6_11 | *No description.* | | V6_10_1 | *No description.* | | V6_10 | *No description.* | | V6_9 | *No description.* | | V6_8 | *No description.* | | V6_7 | *No description.* | | V6_6 | *No description.* | | V6_5 | *No description.* | | V6_4 | *No description.* | | V6_3 | *No description.* | | V6_2 | *No description.* | | V5_33 | *No description.* | | V5_32 | *No description.* | --- ##### `V7_5` --- ##### `V7_4` --- ##### `V7_3` --- ##### `V7_2` --- ##### `V7_1` --- ##### `V7_0` --- ##### `V6_15` --- ##### `V6_14` --- ##### `V6_13` --- ##### `V6_12` --- ##### `V6_11_1` --- ##### `V6_11` --- ##### `V6_10_1` --- ##### `V6_10` --- ##### `V6_9` --- ##### `V6_8` --- ##### `V6_7` --- ##### `V6_6` --- ##### `V6_5` --- ##### `V6_4` --- ##### `V6_3` --- ##### `V6_2` --- ##### `V5_33` --- ##### `V5_32` --- ### EmrRuntimeVersion Enum defining the EMR version as defined in the [Amazon EMR documentation](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html). #### Members | **Name** | **Description** | | --- | --- | | V7_5 | *No description.* | | V7_4 | *No description.* | | V7_3 | *No description.* | | V7_2 | *No description.* | | V7_1 | *No description.* | | V7_0 | *No description.* | | V6_15 | *No description.* | | V6_14 | *No description.* | | V6_13 | *No description.* | | V6_12 | *No description.* | | V6_11_1 | *No description.* | | V6_11 | *No description.* | | V6_10_1 | *No description.* | | V6_10 | *No description.* | | V6_9 | *No description.* | | V6_8 | *No description.* | | V6_7 | *No description.* | | V6_6 | *No description.* | | V6_5 | *No description.* | | V6_4 | *No description.* | | V6_3 | *No description.* | | V6_2 | *No description.* | | V5_33 | *No description.* | | V5_32 | *No description.* | --- ##### `V7_5` --- ##### `V7_4` --- ##### `V7_3` --- ##### `V7_2` --- ##### `V7_1` --- ##### `V7_0` --- ##### `V6_15` --- ##### `V6_14` --- ##### `V6_13` --- ##### `V6_12` --- ##### `V6_11_1` --- ##### `V6_11` --- ##### `V6_10_1` --- ##### `V6_10` --- ##### `V6_9` --- ##### `V6_8` --- ##### `V6_7` --- ##### `V6_6` --- ##### `V6_5` --- ##### `V6_4` --- ##### `V6_3` --- ##### `V6_2` --- ##### `V5_33` --- ##### `V5_32` --- ### EngineVersion #### Members | **Name** | **Description** | | --- | --- | | AUTO | *No description.* | | ATHENA_V3 | *No description.* | | PYSPARK_V3 | *No description.* | --- ##### `AUTO` --- ##### `ATHENA_V3` --- ##### `PYSPARK_V3` --- ### KafkaClientLogLevel The CDK Custom resources uses KafkaJs. This enum allow you to set the log level #### Members | **Name** | **Description** | | --- | --- | | WARN | *No description.* | | DEBUG | *No description.* | | INFO | *No description.* | | ERROR | *No description.* | --- ##### `WARN` --- ##### `DEBUG` --- ##### `INFO` --- ##### `ERROR` --- ### KarpenterVersion The list of supported Karpenter versions as defined [here](https://github.com/aws/karpenter/releases) At this time only v0.37.0 is supported. #### Members | **Name** | **Description** | | --- | --- | | V1_0_1 | *No description.* | --- ##### `V1_0_1` --- ### MskClusterType Enum for MSK cluster types. #### Members | **Name** | **Description** | | --- | --- | | PROVISIONED | *No description.* | | SERVERLESS | *No description.* | --- ##### `PROVISIONED` --- ##### `SERVERLESS` --- ### OpenSearchClusterType #### Members | **Name** | **Description** | | --- | --- | | PROVISIONED | *No description.* | | SERVERLESS | *No description.* | --- ##### `PROVISIONED` --- ##### `SERVERLESS` --- ### OpenSearchNodes Default Node Instances for OpenSearch cluster. #### Members | **Name** | **Description** | | --- | --- | | DATA_NODE_INSTANCE_DEFAULT | *No description.* | | MASTER_NODE_INSTANCE_DEFAULT | *No description.* | | WARM_NODE_INSTANCE_DEFAULT | *No description.* | --- ##### `DATA_NODE_INSTANCE_DEFAULT` --- ##### `MASTER_NODE_INSTANCE_DEFAULT` --- ##### `WARM_NODE_INSTANCE_DEFAULT` --- ### PermissionModel List of supported permissions model. #### Members | **Name** | **Description** | | --- | --- | | LAKE_FORMATION | *No description.* | | HYBRID | *No description.* | | IAM | *No description.* | --- ##### `LAKE_FORMATION` --- ##### `HYBRID` --- ##### `IAM` --- ### RedshiftServerlessNamespaceLogExport Namespace log export types. #### Members | **Name** | **Description** | | --- | --- | | USER_LOG | *No description.* | | CONNECTION_LOG | *No description.* | | USER_ACTIVITY_LOG | *No description.* | --- ##### `USER_LOG` --- ##### `CONNECTION_LOG` --- ##### `USER_ACTIVITY_LOG` --- ### RedshiftServerlessWorkgroupConfigParamKey #### Members | **Name** | **Description** | | --- | --- | | AUTO_MV | *No description.* | | DATESTYLE | *No description.* | | ENABLE_USER_ACTIVITY_LOGGING | *No description.* | | QUERY_GROUP | *No description.* | | SEARCH_PATH | *No description.* | | MAX_QUERY_EXECUTION_TIME | *No description.* | | REQUIRE_SSL | *No description.* | | USE_FIPS_SSL | *No description.* | --- ##### `AUTO_MV` --- ##### `DATESTYLE` --- ##### `ENABLE_USER_ACTIVITY_LOGGING` --- ##### `QUERY_GROUP` --- ##### `SEARCH_PATH` --- ##### `MAX_QUERY_EXECUTION_TIME` --- ##### `REQUIRE_SSL` --- ##### `USE_FIPS_SSL` --- ### ResourcePatternTypes #### Members | **Name** | **Description** | | --- | --- | | UNKNOWN | *No description.* | | ANY | *No description.* | | MATCH | *No description.* | | LITERAL | *No description.* | | PREFIXED | *No description.* | --- ##### `UNKNOWN` --- ##### `ANY` --- ##### `MATCH` --- ##### `LITERAL` --- ##### `PREFIXED` --- ### SparkImage The list of supported Spark images to use in the SparkCICDPipeline. #### Members | **Name** | **Description** | | --- | --- | | EMR_7_5 | *No description.* | | EMR_7_4 | *No description.* | | EMR_7_3 | *No description.* | | EMR_7_2 | *No description.* | | EMR_7_1 | *No description.* | | EMR_7_0 | *No description.* | | EMR_6_15 | *No description.* | | EMR_6_14 | *No description.* | | EMR_6_13 | *No description.* | | EMR_6_12 | *No description.* | | EMR_6_11 | *No description.* | | EMR_6_10 | *No description.* | | EMR_6_9 | *No description.* | --- ##### `EMR_7_5` --- ##### `EMR_7_4` --- ##### `EMR_7_3` --- ##### `EMR_7_2` --- ##### `EMR_7_1` --- ##### `EMR_7_0` --- ##### `EMR_6_15` --- ##### `EMR_6_14` --- ##### `EMR_6_13` --- ##### `EMR_6_12` --- ##### `EMR_6_11` --- ##### `EMR_6_10` --- ##### `EMR_6_9` --- ### State #### Members | **Name** | **Description** | | --- | --- | | ENABLED | *No description.* | | DISABLED | *No description.* | --- ##### `ENABLED` --- ##### `DISABLED` --- ### StorageMode The storage mode for the cluster brokers. #### Members | **Name** | **Description** | | --- | --- | | LOCAL | Local storage mode utilizes network attached EBS storage. | | TIERED | Tiered storage mode utilizes EBS storage and Tiered storage. | --- ##### `LOCAL` Local storage mode utilizes network attached EBS storage. --- ##### `TIERED` Tiered storage mode utilizes EBS storage and Tiered storage. ---