Thursday, March 14, 2024

OCI Knowledge Series: OCI Infrastructure components

 Oracle Cloud Infrastructure (OCI) provides a comprehensive set of infrastructure services that enable you to build and run a wide range of applications in a highly available, secure, and scalable environment. Below are the various components of OCI infrastructure:


These are some of the key components of the Oracle Cloud Infrastructure (OCI) that enable you to build and manage your cloud infrastructure and application

1. Regions: A region is a localized geographic area composed of one or more availability domains. Regions are isolated from each other, and they are independent of each other in terms of fault tolerance and availability. Each region contains multiple data centers called availability domains. 2. Availability Domains (AD): An availability domain is a standalone, independent data center within a region. Availability domains are isolated from each other, with their own power, cooling, and networking infrastructure. This isolation enhances fault tolerance and availability. OCI services deployed within a region are designed to be resilient to failures within an availability domain. 3. Virtual Cloud Network (VCN): A VCN is a customizable, private network within OCI where you can launch your compute instances, block storage, and other resources. It is logically isolated from other virtual networks in the OCI environment, providing you with control over your network settings, such as IP addressing, route tables, and gateways. 4. Subnets: Subnets are subdivisions of a VCN and represent segmented portions of your network. You can divide a VCN into one or more subnets to host different types of resources. Subnets can be public or private, depending on whether they have internet connectivity. 5. Compute Instances: Compute instances, also known as virtual machines (VMs), are virtualized computing environments where you can run your applications. OCI offers various types of compute instances, including generalpurpose, highperformance, and GPU instances, suited for different workload requirements. 6. Block Storage: OCI provides block storage services for storing persistent data. Block volumes can be attached to compute instances as additional disks to provide scalable and highperformance storage. 7. Object Storage: OCI Object Storage is a highly scalable and durable storage service for storing unstructured data, such as documents, images, and videos. It provides a costeffective solution for storing and retrieving large amounts of data. 8. Networking Services: OCI offers a variety of networking services, including load balancers, DNS, VPN, and FastConnect, to enable secure and efficient communication between resources within your VCN and with external networks. 9. Database Services: OCI provides fully managed database services, including Oracle Autonomous Database, MySQL, and NoSQL Database, to support different types of database workloads. 10. Identity and Access Management (IAM): IAM is a centralized service for managing user access and permissions in OCI. It enables you to define and enforce security policies, roles, and permissions to control who can access which resources and perform specific actions. 11. Security Services: OCI offers a range of security services, such as Web Application Firewall (WAF), Key Management, and Security Zones, to protect your applications and data from security threats. 12. Monitoring and Management Tools: OCI provides monitoring, logging, and management tools, including OCI Monitoring, Logging, and Resource Manager, to help you monitor, troubleshoot, and manage your resources effectively. These are some of the key components of the Oracle Cloud Infrastructure (OCI) that enable you to build and manage your cloud infrastructure and application

OCI (Oracle Cloud Infrastructure) SDK to provision VCN, Subnet and establish VPN connectivity

 Oracle Cloud Infrastructure (OCI) Virtual Cloud Network (VCN) is the networking layer of the Oracle Cloud Infrastructure, equivalent to the Virtual Private Cloud (VPC) in other cloud providers. A VCN allows you to set up a customizable and private network in Oracle’s cloud. You can control the VCN’s IP address range, create subnets, and configure route tables and gateways to manage traffic within or outside the VCN.

  1. Private and Isolated Network: A VCN provides an isolated network within the Oracle Cloud Infrastructure.
  2. Customizable: You can set the IP CIDR block, create subnets, and use Network Security Groups or Security Lists to control inbound and outbound traffic.
  3. Route Tables: Define how the traffic is routed within your VCN or to the internet.
  4. Internet Gateway: Allows traffic to flow between your VCN and the internet.
  5. NAT Gateway: Allows instances in a private subnet to initiate outbound connections to the internet without exposing their IP addresses.
  6. VPN Gateway: For secure, encrypted communication between your on-premise network and your VCN.
  7. Load Balancer: Distributes incoming traffic across multiple targets to ensure high availability.
  8. Service Gateway: Provides a path for private traffic between your VCN and supported Oracle services.


This code snippet creates a VCN, subnet, security list, and VPN using the OCI Java SDK, utilizing the Identity service client and the respective create methods. Make sure to handle exceptions appropriately in your production code.


Make sure to replace "your_compartment_id", "YourVCN", "YourSubnet", "YourSecurityList", and "YourVPN" with appropriate values for your Oracle Cloud tenancy, Virtual Cloud Network (VCN), subnet, security list, and VPN display names respectively.

Ensure that your OCI configuration file (typically found at ~/.oci/config) is properly configured with your user credentials and the correct region.


import com.oracle.bmc.identity.IdentityClient;

import com.oracle.bmc.identity.model.CreateVpnDetails;

import com.oracle.bmc.identity.model.CreateSubnetDetails;

import com.oracle.bmc.identity.model.CreateSecurityListDetails;

import com.oracle.bmc.identity.model.CreateSecurityRuleDetails;

import com.oracle.bmc.identity.requests.CreateVpnRequest;

import com.oracle.bmc.identity.requests.CreateSubnetRequest;

import com.oracle.bmc.identity.requests.CreateSecurityListRequest;

import com.oracle.bmc.identity.responses.CreateVpnResponse;

import com.oracle.bmc.identity.responses.CreateSubnetResponse;

import com.oracle.bmc.identity.responses.CreateSecurityListResponse;

import com.oracle.bmc.Region;

import com.oracle.bmc.auth.AuthenticationDetailsProvider;

import com.oracle.bmc.auth.ConfigFileAuthenticationDetailsProvider;

import com.oracle.bmc.model.BmcException;


import java.util.Collections;


public class InfrastructureProvisioning {

    public static void main(String[] args) {

        String compartmentId = "your_compartment_id";

        String vcnDisplayName = "YourVCN";

        String subnetDisplayName = "YourSubnet";

        String securityListDisplayName = "YourSecurityList";

        String vpnDisplayName = "YourVPN";


        // Path to your OCI configuration file

        String configurationFilePath = "~/.oci/config";


        // Get the authentication details from the OCI configuration file

        AuthenticationDetailsProvider provider =

                new ConfigFileAuthenticationDetailsProvider(configurationFilePath, "DEFAULT");


        IdentityClient identityClient = new IdentityClient(provider);

        identityClient.setRegion(Region.US_PHOENIX_1); // Change to appropriate region


        try {

            // Create VCN

            CreateVcnDetails createVcnDetails = CreateVcnDetails.builder()

                    .cidrBlock("10.0.0.0/16")

                    .compartmentId(compartmentId)

                    .displayName(vcnDisplayName)

                    .build();


            CreateVcnRequest createVcnRequest = CreateVcnRequest.builder()

                    .createVcnDetails(createVcnDetails)

                    .build();


            Vcn vcn = identityClient.createVcn(createVcnRequest).getVcn();


            // Create Subnet

            CreateSubnetDetails createSubnetDetails = CreateSubnetDetails.builder()

                    .cidrBlock("10.0.0.0/24")

                    .compartmentId(compartmentId)

                    .displayName(subnetDisplayName)

                    .vcnId(vcn.getId())

                    .build();


            CreateSubnetRequest createSubnetRequest = CreateSubnetRequest.builder()

                    .createSubnetDetails(createSubnetDetails)

                    .build();


            Subnet subnet = identityClient.createSubnet(createSubnetRequest).getSubnet();


            // Create Security List

            CreateSecurityRuleDetails createSecurityRuleDetails = CreateSecurityRuleDetails.builder()

                    .direction(CreateSecurityRuleDetails.Direction.Egress)

                    .destination("0.0.0.0/0")

                    .protocol("all")

                    .build();


            CreateSecurityListDetails createSecurityListDetails = CreateSecurityListDetails.builder()

                    .compartmentId(compartmentId)

                    .displayName(securityListDisplayName)

                    .egressSecurityRules(Collections.singletonList(createSecurityRuleDetails))

                    .ingressSecurityRules(Collections.singletonList(createSecurityRuleDetails))

                    .vcnId(vcn.getId())

                    .build();


            CreateSecurityListRequest createSecurityListRequest = CreateSecurityListRequest.builder()

                    .createSecurityListDetails(createSecurityListDetails)

                    .build();


            SecurityList securityList = identityClient.createSecurityList(createSecurityListRequest).getSecurityList();


            // Create VPN

            CreateVpnDetails createVpnDetails = CreateVpnDetails.builder()

                    .compartmentId(compartmentId)

                    .displayName(vpnDisplayName)

                    .vcnId(vcn.getId())

                    .build();


            CreateVpnRequest createVpnRequest = CreateVpnRequest.builder()

                    .createVpnDetails(createVpnDetails)

                    .build();


            Vpn vpn = identityClient.createVpn(createVpnRequest).getVpn();


            System.out.println("VPN Created: " + vpn.getId());

            System.out.println("Subnet Created: " + subnet.getId());

            System.out.println("Security List Created: " + securityList.getId());

        } catch (BmcException e) {

            System.out.println("Error: " + e.getMessage());

        } finally {

            identityClient.close();

        }

    }

}


Wednesday, December 13, 2023

TypeScript-first schema declaration using ZOD

Zod is a TypeScript-first schema declaration and validation library used to define the shape of data in TypeScript. It allows you to create schemas for your data structures, validate incoming data against those schemas, and ensure type safety within your TypeScript applications.


Here's a simple example demonstrating how Zod can be used:


Typescript code

import * as z from 'zod';


// Define a schema for a user object

const userSchema = z.object({

  id: z.string(),

  username: z.string(),

  email: z.string().email(),

  age: z.number().int().positive(),

  isAdmin: z.boolean(),

});


// Data to be validated against the schema

const userData = {

  id: '123',

  username: 'johndoe',

  email: 'john@example.com',

  age: 30,

  isAdmin: true,

};


// Validate the data against the schema

try {

  const validatedUser = userSchema.parse(userData);

  console.log('Validated user:', validatedUser);

} catch (error) {

  console.error('Validation error:', error);

}

```


In the above example:


1. We import `z` from 'zod', which provides access to Zod's functionality.

2. We define a schema for a user object using `z.object()`. Each property in the object has a specific type and validation constraint defined by Zod methods like `z.string()`, `z.number()`, `z.boolean()`, etc.

3. `userData` represents an object we want to validate against the schema.

4. We use `userSchema.parse()` to validate `userData` against the defined schema. If the data matches the schema, it returns the validated user object; otherwise, it throws a validation error.


Zod helps ensure that the incoming data adheres to the defined schema, providing type safety and validation within TypeScript applications. This prevents runtime errors caused by unexpected data shapes or types.

Friday, October 27, 2023

AWS DMS and Deployment of tasks using AWS CDK-Python

 AWS DMS stands for Amazon Web Services Database Migration Service. It is a fully managed database migration service that helps you migrate databases to AWS quickly and securely. AWS DMS supports both homogeneous migrations, where the source and target databases are of the same engine (e.g., Oracle to Oracle), and heterogeneous migrations, where the source and target databases are of different engines (e.g., Microsoft SQL Server to Amazon Aurora).

Key features of AWS DMS include:

1. Data Replication: AWS DMS can continuously replicate data changes from the source database to the target database, ensuring that the target remains up to date with changes made in the source.

2. Schema Conversion: For heterogeneous migrations, AWS DMS can help convert schema and data types from the source to the target database to ensure compatibility.

3. Minimized Downtime: It allows you to migrate data with minimal downtime by performing an initial data load and then continually synchronizing changes.

4. Database Cloning: You can use DMS to create a clone of your production database for testing and development purposes.

5. Change Data Capture: AWS DMS can capture changes from popular database engines, such as Oracle, SQL Server, MySQL, PostgreSQL, and more, in real-time.

6. Data Filtering and Transformation: You can configure data filtering and transformation rules to control what data gets migrated and how it's transformed during the migration process.

7. Security and Encryption: AWS DMS provides encryption options to ensure the security of your data during migration.

8. Integration with AWS Services: AWS DMS can be integrated with other AWS services, such as AWS Schema Conversion Tool (SCT), AWS Database Assessment Tool (DAT), and AWS Database Query Tool (DQT), to facilitate the migration process.

Overall, AWS DMS is a versatile tool for simplifying and automating database migrations to AWS, making it easier for organizations to move their databases to the cloud while minimizing disruptions to their applications.


Deployment using AWS CDK

To create an AWS Cloud Development Kit (CDK) stack for AWS Database Migration Service (DMS) in Python, you'll need to define the necessary resources, such as replication instances, endpoints, and migration tasks. Below is a basic example of how to create a DMS stack using AWS CDK. Note that you'll need to have the AWS CDK and AWS CLI configured on your system and also install the necessary CDK modules.

from aws_cdk import core

from aws_cdk import aws_dms as dms

from aws_cdk import aws_secretsmanager as secrets_manager

class DMSStack(core.Stack):

    def init(self, scope: core.Construct, id: str, **kwargs) -> None:

        super().__init__(scope, id, **kwargs)

        # Create a secret to store credentials for the source and target databases

        source_secret = secrets_manager.Secret(

            self, "SourceDatabaseSecret",

            description="Secret for source database connection",

            generate_secret_string=secrets_manager.SecretStringGenerator(

                secret_string_template={"username": "source_username"},

                generate_string_key="password",

                password_length=12,

                exclude_characters='"@/',

            ),

        )

        target_secret = secrets_manager.Secret(

            self, "TargetDatabaseSecret",

            description="Secret for target database connection",

            generate_secret_string=secrets_manager.SecretStringGenerator(

                secret_string_template={"username": "target_username"},

                generate_string_key="password",

                password_length=12,

                exclude_characters='"@/',

            ),

        )

        # Define a replication instance

        replication_instance = dms.CfnReplicationInstance(

            self, "ReplicationInstance",

            replication_instance_class="dms.r5.large",

            allocated_storage=100,

        )

        # Define source and target endpoints

        source_endpoint = dms.CfnEndpoint(

            self, "SourceEndpoint",

            endpoint_identifier="source-endpoint",

            endpoint_type="source",

            engine_name="mysql",

            username=source_secret.secret_value_from_json("username").to_string(),

            password=source_secret.secret_value_from_json("password").to_string(),

            server_name="source-database-server",

            port=3306,

            database_name="source_database",

        )

        target_endpoint = dms.CfnEndpoint(

            self, "TargetEndpoint",

            endpoint_identifier="target-endpoint",

            endpoint_type="target",

            engine_name="aurora",

            username=target_secret.secret_value_from_json("username").to_string(),

            password=target_secret.secret_value_from_json("password").to_string(),

            server_name="target-database-cluster",

            port=3306,

            database_name="target_database",

        )

        # Create a migration task

        migration_task = dms.CfnReplicationTask(

            self, "MigrationTask",

            migration_task_identifier="my-migration-task",

            migration_type="full-load",

            source_endpoint_arn=source_endpoint.attr_endpoint_arn,

            target_endpoint_arn=target_endpoint.attr_endpoint_arn,

            table_mappings="...custom table mapping...",

        )

app = core.App()

DMSStack(app, "DMSStack")

app.synth()


In this code, we create a CDK stack that includes:

1. Secrets for storing database credentials.

2. A replication instance for DMS.

3. Source and target endpoints for the source and target databases.

4. A migration task that specifies the type of migration (full-load) and the endpoints to use.

You'll need to customize this code by providing the actual database connection details and table mappings in the migration task. Additionally, you may need to install the required CDK modules and configure AWS CDK on your system before deploying the stack.

Thursday, October 26, 2023

Install AWS Schema Conversion Tool (SCT) on an Amazon Linux 2

To install the AWS Schema Conversion Tool (SCT) on an Amazon Linux 2 instance, you can follow these steps. The AWS Schema Conversion Tool helps you convert your database schema from one database engine to another, making it easier to migrate your data.


1. Prerequisites:

   - An Amazon Linux 2 instance.

   - AWS account credentials with appropriate permissions to download and install the tool.


2. Connect to Your Amazon Linux 2 Instance:

   You can use SSH to connect to your Amazon Linux 2 instance. Make sure you have the necessary permissions and key pair for accessing the instance.


3. Update Your System:

   It's a good practice to start by updating the package repository and installed packages:

   sudo yum update -y

4. Download and Install AWS SCT:

   You can download and install AWS SCT using `curl` and `yum`:

   sudo curl "https://d1un7b5vff5wnt.cloudfront.net/downloads/AWSSchemaConversionToolSetup-x86_64.bin" -o AWSSchemaConversionToolSetup-x86_64.bin

   sudo chmod +x AWSSchemaConversionToolSetup-x86_64.bin

   sudo ./AWSSchemaConversionToolSetup-x86_64.bin

  

   This will launch the AWS Schema Conversion Tool installer. Follow the installation prompts and choose the installation location. It's recommended to install it in a directory that's in your `PATH` for easier access.


5. Start AWS SCT:

   After the installation is complete, you can start the AWS Schema Conversion Tool:

 

   aws-schema-conversion-tool


6. Configure AWS SCT:

   When you first start AWS SCT, you'll need to configure it by providing your AWS account credentials and configuring connection profiles for your source and target databases.


   Follow the on-screen instructions to set up these configurations.


7. Using AWS SCT:

   You can now use AWS SCT to perform schema conversions and database migrations.


Remember that AWS SCT requires Java, so make sure that Java is installed on your Amazon Linux 2 instance.


Once you've completed these steps, you should have AWS SCT up and running on your Amazon Linux 2 instance, and you can use it to convert and migrate your database schemas.

Wednesday, October 18, 2023

AWS SAM template to deploy lamdas function exposing lambda through API Gateway.

This thread discusses the steps to deploy a Lambda function named "getIdentities" with a layer and expose it through API Gateway using AWS SAM.  The Lambda function fetches data from DynamoDB, you can use the following AWS SAM template. This example assumes you're working with Node.js for your Lambda function and DynamoDB as your database:


YAML 

AWSTemplateFormatVersion: '2010-09-09'

Transform: 'AWS::Serverless-2016-10-31'


Resources:

  MyLambdaLayer:

    Type: AWS::Serverless::LayerVersion

    Properties:

      LayerName: MyLayer

      ContentUri: ./layer/

      CompatibleRuntimes:

        - nodejs14.x

      Description: My Lambda Layer


  MyLambdaFunction:

    Type: AWS::Serverless::Function

    Properties:

      Handler: index.handler

      Runtime: nodejs14.x

      Layers:

        - !Ref MyLambdaLayer

      CodeUri: ./function/

      Description: My Lambda Function

      Environment:

        Variables:

          DYNAMODB_TABLE_NAME: !Ref MyDynamoDBTable

      Events:

        MyApi:

          Type: Api

          Properties:

            Path: /getIdentities

            Method: GET


  MyDynamoDBTable:

    Type: AWS::DynamoDB::Table

    Properties:

      TableName: MyDynamoDBTable

      AttributeDefinitions:

        - AttributeName: id

          AttributeType: S

      KeySchema:

        - AttributeName: id

          KeyType: HASH

      ProvisionedThroughput:

        ReadCapacityUnits: 5

        WriteCapacityUnits: 5


Outputs:

  MyApi:

    Description: "API Gateway endpoint URL"

    Value:

      Fn::Sub: "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/getIdentities"

 


In this SAM template:


1. We define a Lambda Layer resource named `MyLambdaLayer`. You should place your layer code in the `./layer/` directory.


2. We define a Lambda function resource named `MyLambdaFunction`. This function uses the layer created in step 1 and is associated with an API Gateway event at the path "/getIdentities" and HTTP method GET. The function code is located in the `./function/` directory, and the handler is set to `index.handler`. We also set an environment variable to specify the DynamoDB table name.


3. We define a DynamoDB table resource named `MyDynamoDBTable` to store your data. Adjust the table name, schema, and provisioned throughput according to your requirements.


4. The Outputs section provides the URL of the API Gateway endpoint where you can invoke the "getIdentities" Lambda function.


Make sure your project directory structure is organized as follows:



project-directory/

  ├── template.yaml

  ├── function/

  │     ├── index.js

  │     └── package.json

  ├── layer/

  │     ├── layer-files

  └── template-configs/

        ├── parameters.json

        ├── metadata.json



To fetch data from DynamoDB, you can use the AWS SDK for JavaScript in your Lambda function code. Here's a simple example of how you can fetch data from DynamoDB using the Node.js SDK:


const AWS = require('aws-sdk');


const dynamodb = new AWS.DynamoDB.DocumentClient();

const tableName = process.env.DYNAMODB_TABLE_NAME;


exports.handler = async (event) => {

  try {

    const params = {

      TableName: tableName,

      Key: {

        id: 'your-key-here',

      },

    };


    const data = await dynamodb.get(params).promise();


    return {

      statusCode: 200,

      body: JSON.stringify(data.Item),

    };

  } catch (error) {

    return {

      statusCode: 500,

      body: JSON.stringify({ error: error.message }),

    };

  }

};



In this code, we're using the AWS SDK to fetch an item from DynamoDB based on a specified key. You should customize the key and error handling based on your use case.

Friday, September 22, 2023

Manage Identities in Amazon Cognito

Amazon Cognito is a service provided by AWS (Amazon Web Services) for managing user identities and authentication in your applications. To create identities in Amazon Cognito using Java, you can use the AWS SDK for Java. Below is an example of Java code to create identities in Amazon Cognito:


Before you start, make sure you have set up an Amazon Cognito User Pool and Identity Pool in your AWS account.


1. Add the AWS SDK for Java to your project. You can use Maven or Gradle to manage dependencies. Here's an example using Maven:



<dependency>

    <groupId>com.amazonaws</groupId>

    <artifactId>aws-java-sdk-cognitoidentity</artifactId>

    <version>1.11.1069</version> <!-- Replace with the latest version -->

</dependency>

 


2. Write Java code to create identities in Amazon Cognito:


```java

import com.amazonaws.auth.AWSStaticCredentialsProvider;

import com.amazonaws.auth.BasicAWSCredentials;

import com.amazonaws.services.cognitoidentity.AmazonCognitoIdentity;

import com.amazonaws.services.cognitoidentity.AmazonCognitoIdentityClient;

import com.amazonaws.services.cognitoidentity.model.GetIdRequest;

import com.amazonaws.services.cognitoidentity.model.GetIdResult;

import com.amazonaws.services.cognitoidentity.model.GetOpenIdTokenRequest;

import com.amazonaws.services.cognitoidentity.model.GetOpenIdTokenResult;

import com.amazonaws.services.cognitoidentity.model.IdentityPoolConfigurationException;


public class ManageCognitoIdentity {

    public static void main(String[] args) {

        // Replace these with your own values

        String identityPoolId = "your-identity-pool-id";

        String accessKeyId = "your-access-key-id";

        String secretAccessKey = "your-secret-access-key";

        

        // Initialize the AWS credentials and Cognito Identity client

        BasicAWSCredentials awsCredentials = new BasicAWSCredentials(accessKeyId, secretAccessKey);

        AmazonCognitoIdentity identityClient = AmazonCognitoIdentityClient.builder()

                .withRegion("your-region") // Replace with your AWS region

                .withCredentials(new AWSStaticCredentialsProvider(awsCredentials))

                .build();

        

        // Get an identity ID

        GetIdRequest getIdRequest = new GetIdRequest().withIdentityPoolId(identityPoolId);

        try {

            GetIdResult idResult = identityClient.getId(getIdRequest);

            String identityId = idResult.getIdentityId();

            System.out.println("Identity ID: " + identityId);

            

            // Get an OpenID token for the identity

            GetOpenIdTokenRequest getTokenRequest = new GetOpenIdTokenRequest().withIdentityId(identityId);

            GetOpenIdTokenResult tokenResult = identityClient.getOpenIdToken(getTokenRequest);

            String openIdToken = tokenResult.getToken();

            System.out.println("OpenID Token: " + openIdToken);

        } catch (IdentityPoolConfigurationException e) {

            System.err.println("Error: Identity pool configuration is invalid.");

            e.printStackTrace();

        }

    }

}

```


Make sure to replace `"your-identity-pool-id"`, `"your-access-key-id"`, `"your-secret-access-key"`, and `"your-region"` with your actual Amazon Cognito Identity Pool ID, AWS access key, secret access key, and the AWS region you're using.


This code first gets an identity ID for a user from the Cognito Identity Pool and then retrieves an OpenID token associated with that identity.

OCI Knowledge Series: OCI Infrastructure components

  Oracle Cloud Infrastructure (OCI) provides a comprehensive set of infrastructure services that enable you to build and run a wide range of...