Saturday, October 31, 2020

AWS - Attaching Lambda Trigger to DynamoDB Table

Attaching an existing Lambda as a DynamoDB Table's Stream trigger so that the Lambda can see the changes as they happen in the DynamoDB Table

Let's see how to attach an existing Lambda as a DynamoDB Table Stream trigger using code. If you disable/enable DynamoDB Stream for any reason the Lambda Trigger would get disconnected and this code would reattach the Lambda trigger back. This code is written in Java

  • Retrieve DynamoDB Table's Stream ARN. 
           String streamArn = null;  
           AmazonDynamoDBStreams dynamoDBStreamsClient = AmazonDynamoDBStreamsClientBuilder.standard().build();  
           // Create request  
           ListStreamsRequest listStreamRequest = new ListStreamsRequest();  
           listStreamRequest.setTableName(dynamoDBTable);  
           // Fetch Streams  
           ListStreamsResult response = dynamoDBStreamsClient.listStreams(listStreamRequest);  
           // Get the StreamARN  
           if (response != null && response.getStreams() != null && response.getStreams().size() > 0) {  
                Stream stream = response.getStreams().get(0);  
                streamArn = stream.getStreamArn();  
           }  
           return streamArn;

When a DynamoDB Table name is passed on to the above code, the 'listStreams' would return only a single Stream ARN, if the Table name is not passed it would return all the Table's Stream ARN.

  • Attach Lambda as a Trigger for DynamoDB Table's Stream. In the below code the 'eventSourceArn' is same as the 'streamArn' we retrieved above
           // create EventSourceMapping Request  
           CreateEventSourceMappingRequest createEventSourceMappingRequest = new CreateEventSourceMappingRequest();  
           // Set the Trigger enabled  
           createEventSourceMappingRequest.setEnabled(true);  
           // Set the Source of the Stream, in our case DynamoDB  
           createEventSourceMappingRequest.setEventSourceArn(eventSourceArn);  
           // Set the Starting position of the Stream to be consumed  
           createEventSourceMappingRequest.setStartingPosition("LATEST");  
           // Name of the Lambda function that gets triggered  
           createEventSourceMappingRequest.setFunctionName(functionName);  
           // Create Lambda Client  
           AWSLambda lambdaClient = AWSLambdaClientBuilder.standard().build();  
           // Execute SourceMapping request  
           CreateEventSourceMappingResult createEventSourceMappingResult = lambdaClient  
                     .createEventSourceMapping(createEventSourceMappingRequest);

Complete source code can be found here

Saturday, October 24, 2020

AWS - Java based client using different CredentialsProvider

AWS Java client using different CredentialsProvider

Credentials to a AWS Java client can be passed in multiple ways. Let's see a few of the common ways of building Java client sourcing Credentials from different sources. In this Blog building a Lambda Client is demonstrated, however the same way any Client can be built (such as AmazonDynamoDB etc.)

  • Credentials from ~/.aws/credentials file. Refer to this Blog for setting up Credentials

AWSLambda awsLambda = AWSLambdaClientBuilder.defaultClient();  

  • Credentials from ~/.aws/credentials file using Profile name

String profileName = "lambdaUser";  
AWSCredentialsProvider credentialsProvider = new ProfileCredentialsProvider(profileName);  
AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider).build();  

  • Credentials from a properties file in the Classpath. By default AwsCredentials.properties file from Classpath is loaded, this can be overridden. 

 // Create a CredentilsProvider object using the AwsCredentials.properties in the classpath  
 AWSCredentialsProvider credentialsProvider = new ClasspathPropertiesFileCredentialsProvider();  
 // Build the Client using CredentialsProvider Object  
 AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider).build();  

  • Credentials passed directly in the Program

 // Access and Secret keys of User  
 String accessKey = "<YOUR_ACCESS_KEY>";  
 String secretKey = "<YOUR_SECRET_KEY>";  
 // Build a Credential object  
 AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);  
 // Create a CredentilsProvider object using the Credential object  
 AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);  
 // Build the Client using CredentialsProvider Object  
 AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider)  

Complete source code can be checked out from the Git Repo

Friday, October 23, 2020

AWS - Enabling/Disabling DynamoDB Stream programmatically

How to Enable/Disable AWS DynamoDB Stream programmatically?

It would be useful for testing purposes, through automation code, to enable or disable AWS DynamoDB Streams to Lambda. Let's see how to achieve that.



           boolean enable = false;//false-to disable, true to enable  
           String dynamoDBTable = "TableName";  
           // Create Async client  
           AmazonDynamoDBAsync ddb = AmazonDynamoDBAsyncClientBuilder.defaultClient();  
           // Create StreamSpecification  
           StreamSpecification streamSpecification = new StreamSpecification();  
           streamSpecification.setStreamEnabled(enable);  
           // StreamViewType is needed only for enabling  
           if (enable) {  
                streamSpecification.setStreamViewType(StreamViewType.NEW_AND_OLD_IMAGES);  
           }  
           // Create update request using the StreamSpecification  
           UpdateTableRequest updateTableRequest = new UpdateTableRequest();  
           updateTableRequest.setTableName(dynamoDBTable);  
           updateTableRequest.setStreamSpecification(streamSpecification);  
           // Execute the update request Asynchronously  
           Future<UpdateTableResult> updateResultFuture = ddb.updateTableAsync(updateTableRequest);  
  • Reattach if any Lambda Trigger back to the Stream otherwise any existing Lambda trigger won't work as they got removed during this process, refer to this Blog
Note: The program is not exiting after executing the request, not sure it's due to wrong implementation of the Async API of 'AmazonDynamoDBAsync' code. Please comment if you have a solution to exit from the program gracefully. I ran the program on Java (11.0.8) with AWS DynamoDB SDK (1.11.881) and also using Java 1.8.0_272 as well with no luck

Friday, October 16, 2020

AWS - Setting up client credentials

 Setting up aws Client credentials for locally executing programs and running CLI commands

Creating user with Programmatic access: 

While running programs locally, the program should have access to aws Client credentials so that the Program can use it to access aws resources such as DynamoDB, SQS etc. While creating new User one has to specify the user is being granted 'Programmatic access' so that "access_key" and "secret_key" will be generated. This could be used for aws cli, SDK etc.



 Client credentials:

  • Client credentials goes to a file in the home directory, In Unix based systems it's at ~/.aws and the file name is credentials
  • Sample content of ~/.aws/credentials

[default]
aws_access_key_id=AKIAIOSFEXUSEREXAMPLE
aws_secret_access_key=wJalrKJDLFJ78EMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8JFIOHDFBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9UDFKJh3yCo8nvbEXAMPLEKEY

  • The first line is the profile name within square bracket

aws region configuration:

  • aws region information goes to a file in the same directory (~/.aws) and the file name is config (no extension)
  • Sample content of ~/.aws/config
[default]
region=us-west-2
output=json
[profile user1]
region=us-east-1
output=text
  •  Note the prefix 'profile' before the 'user1' this is needed in config file and it would be simply 'user1' in credentials file
Aws cli assume default profile while running cli command, if you want to use different user profile then mention profile name 

aws dynamodb describe-table --profile user1 --table-name songs

Switch between Java versions on Fedora OS

Switching between multiple versions of Java on a Fedora OS (all *nix based OS)

To install multiple versions of Java refer this Blog

  • Check the current version of Java using the command
 java -version


Current version of java being used is 11.0.8
  • To switch to different installed version, issue the command
 sudo alternatives --config java

enter password if prompted for sudo

All the versions of Java installed currently will be shown as in image below 

Type in the number under "Selection" Column to switch to that version, for example type 1 to switch to java-1.8.0

  • Verify the version of java is changed as selected above

 java -version
 

After switching to a particular Java version (e.g. 11), see how it's linked to 'java' executable



Note: If you have JAVA_HOME variable set in ~/.bash_profile that needs to be removed. Restart is required upon removal 

Thursday, October 15, 2020

Installing Java on Fedora

How to install Java on Fedora?

  • List available versions
dnf search openjdk


  • Choose the version of Java you want to install, say java-11-openjdk-devel.x86_64
  • Run the command
sudo dnf install java-11-openjdk-devel.x86_64

  • Follow the prompts to complete the Installation