Thursday, December 31, 2020

JavaScript - Understanding well


Let's understand JavaScript well

  • Avoid 'var' to declare variable
What's the output?
1:  var a = "X";  
2:  console.log(a+" - "+b);  
3:  var b = "Y";  
Explanation:
Do you expect the output to be "X - Y"? or an error be thrown? both are wrong.
The output is "X - undefined"
Here's the reason: JavaScript hoisting occurs during the creation phase of the execution context that moves the variable and function declarations to the top of the script. Even though the variable 'b' is declared and initialized with a value "Y" at line # 3, the declaration gets moved to the top of the script with a value undefined.
1:  bar();  
2:  //foo();//ReferenceError: Cannot access 'foo' before initialization   
3:  let foo = () => {  
4:    console.log('hello from foo');  
5:  }  
6:  function bar() {  
7:    console.log('hello from bar');  
8:  }  
Similarly in the above code the function 'bar' is accessible even before it's defined due to hoisting which makes the code difficult to follow as a function is used before it's defined. On the other hand the function 'foo' is not accessible before it's defined as it's created using 'let'. Use only let or const to declare variables and functions

  • Closure
A closure is the combination of a function and the lexical environment within which that function was declared. This environment consists of any local variables that were in-scope at the time the closure was created.
Let's understand this with an example
1:  let simpleFactory = () => {  
2:    return () => {console.log('log from function returned by simpleFactory'); }  
3:  }  
4:  let simpleFun = simpleFactory();  
5:  simpleFun();//log from function returned by simpleFactory  
On line# 1 we define a Factory function which returns a function that simply prints a message on the console. 
Lin# 4 we got the returned function stored in a variable 'simpleFun'. 
Line# 5 we invoke the function stored in the variable 'simpleFun' which prints the log message. 
Simple to understand, right? Let's see another example

1:  let parameterizedFactory = (x) => {  
2:    return (y) => {return x*y;}  
3:  }  
4:  let parameterizedFun = parameterizedFactory(5);  
5:  console.log(parameterizedFun(2));//prints 10  
6:  console.log(parameterizedFun(3));//prints 15  
Now we are adding a little complexity to the Factory function by parameterizing it. The factory now accepts a value in parameter x and that value is used inside the function returned by the factory in line#2. 
We call parameterizedFactory by passing a value '5' which gets assigned in 'x'
The function returned in line# 2 uses the variable 'x' inside its body which is possible because when the function (at line# 2) is created an environment is created for that function which includes all the in-scoped local variables, that's 'x' in this case.
The function got stored in 'parameterizedFun' at line# 4 has access to the variable 'x' with value '5' due to Closure.

The local variable doesn't need stay constant, it can be even modified.
Let's see an example:
1:  let factoryWithLocVar = () => {  
2:    let count = 0;  
3:    return (y) => {count++; return count + y; }  
4:  }  
5:  let funWithLocVar = factoryWithLocVar();  
6:  console.log(funWithLocVar(2));//prints 3  
7:  console.log(funWithLocVar(5));//prints 7  
The function returned in line# 3 refers to a local variable count and even it modifies it during each invocation. The local variable 'count' is modified during each invocation and the modification reflects in subsequent calls to that function.

  • Pure function
A pure function in programming is a function that always produces the same output for the same set of input values and has no side effects. It means it should not change any state or data

Example:
1:  let add = (a,b) => {  
2:    return a+b;  
3:  }  
The above function would return the same result how many ever times it's called with the same input, and it doesn't modify state of any variables. In contrast an impure function may not return the same value in subsequent invocations for the same input also it could modify the state of a global variable (in the example given below it's the sum variable). 

Let's see an example:
1: let sum = 0;

2:  let add = (a,b) => {  
3:    sum += a+b;
4:    return sum;
5: }

More to follow ......

Sunday, December 20, 2020

AWS - Lambda Blue/Green deployment

AWS - Zero downtime Blue/Green deployment for Lambda

If your application is based on Lambda, the Lambda's can be updated in Production with zero downtime using Lambda Versioning and Alias. The Lambda's can be rolled back too easily if something is wrong with the latest code with zero downtime.

Let's see in detail

Understanding some keywords

  • Versions - Lambda creates a new version of your function each time that you publish the function. The versioned Lambda's code is frozen and not editable. The version is automatically named 1, 2, 3 and so on each time published and one can't name a version differently
  • Aliases - A Lambda alias is like a pointer to a specific Lambda version. Users can invoke the Lambda version using the alias. The Alias can be updated to point to different version
  • Blue version - Existing version of Lambda being currently used in Production (In Diagram 2 v2)
  • Green version - New version of Lambda being deployed to Production
Keep in mind, Lambda can be invoked in three ways
  1. Just using its name (e.g. BonusCalculatorLambda). In this case the $LATEST version of the Lambda is invoked by default
  2. Using its Version (e.g. BonusCalculatorLambda:1). In this case the version 1 of the Lambda is invoked
  3. Using its Alias (e.g. BonusCalculatorLambda:PROD). In this case the version of the Lambda currently being pointed by the Alias is invoked (As per the Diagram 2 below version 2)
Let's assume your application is built using Spring Boot or Node.js Express and uses multiple Lambdas to implement its business logic and invokes them using AWS SDK as depicted in the following picture. The gateway application is running on ECS Cluster which is exposed to Clients via an ALB. Each of the Lambda can be updated in Production with zero downtime.

Diagram 1 - Lambda based application


How do we update each Lambda with zero down time and rollback in case something is not good?

Diagram 2 - Before Production deployment state


The above diagram depicts the current state of the Application before deployment for a single Lambda. The Lambda is accessible via an Alias (called PROD) which is now pointing to Version 2 of Lambda (BLUE version). At this point we have the following versions of Lambda v1, v2 and $LATEST.

Here's how to update the Lambda with zero downtime, the same approach needs to be replicated for each Lambda. 
Follow these steps to update the Lambda with changes, refer to Diagram 3 below
  1. Update the Lambda code using a Jenkins job in Production, this would update the $LATEST version of the Lambda, this is the only version which is open for update, rest of the versions (v1, v2) are frozen and can't be modified
  2. Test the updated Lambda using a Test User, the Application would invoke $LATEST version of the Lambda based on some criteria such as logged in User (Test User) etc. For real User the Application invokes Lambda using the PROD Alias (e.g. BonusCalculatorLambda:PROD) which would still continue to invoke v2 (Blue) version of the Lambda. Here's a sample Node.js Lambda client
  3. Once the Test results of $LATEST versions is satisfactory, invoke publish Jenkins job. This would publish v3 of the Lambda. Refer to Diagram 3 below
  4. As a last step run a Jenkins job to update the Alias PROD so that it starts pointing to v3 (Green) version of Lambda
  5. In case the deployment needs to be rolled back, just invoke Alias update Jenkins job on the last Pipeline # so that it points back to previous BLUE version v2
During any of this process the Production users are not impacted and the Service is up all time. Replicate this same process for all your Lambdas

Diagram 3 - After deployment of new version in Production



Here's AWS CLI commands to do all of the things said above, the Lambda is implemented using Node.js:
  • Create index.js file with the following line of code
 exports.handler = async (event) => {  
   const response = {  
     statusCode: 200,  
     body: JSON.stringify('Hello from Lambda!'),  
   };  
   return response;  
 };  

  • Create a zip file to deploy this code as a Lambda (no .zip extension is necessary, it gets added)
 zip BonusCalculatorLambda index.js

  • Create the Lambda, make sure you are running this command from the folder where you have BonusCalculatorLambda.zip This creates the $LATEST version of the Lambda
 aws lambda create-function --function-name BonusCalculatorLambda \
--role arn:aws:iam::123903503456:role/service-role/roleLambdaExecution \ --runtime nodejs12.x \ --handler index.handler \ --zip-file "fileb://BonusCalculatorLambda.zip
"

  • Publish the Lambda to create a version, after this command there would be two version of Lambda 1 and $LATEST, only $LATEST is editable and version 1 is frozen
 aws lambda publish-version --function-name BonusCalculatorLambda --description v1  

  • Create an Alias
 aws lambda create-alias --function-name BonusCalculatorLambda \
      --name PROD \
      --function-version 1

  • Update Lambda code, this updates $LATEST version
 aws lambda update-function-code --function-name BonusCalculatorLambda \
      --zip-file "fileb://BonusCalculatorLambda.zip"

  • Upon creating new version 2, update the Alias to point to new version
 aws lambda update-alias --function-name BonusCalculatorLambda \
      --name PROD --function-version 2

Prerequisite to execute these commands
  • One should have setup Client Credentials, see here for instructions
  • The Lambda needs a role to Execute (in the example roleLambdaExecution) this should have been created already
  • The Client credential one has setup for CLI should have the following Policy action
 {  
   "Version": "2012-10-17",  
   "Statement": [  
     {  
       "Sid": "VisualEditor",  
       "Effect": "Allow",  
       "Action": "iam:PassRole",  
       "Resource": "arn:aws:iam::123903503456:role/service-role/roleLambdaExecution"  
} ] }

Note: For the Blue/Green zero downtime deployment it's not necessary your Lambda's should have been exposed via an application (Spring Boot or Node.js Express). It's a sample Architecture explained in this article. The Lambda can be exposed directly via an ALB or API Gateway and the same approach can be used to deploy changes with zero downtime with slight modifications.

Saturday, December 5, 2020

NodeJS AWS Lamda Client

In this Post let's see how to invoke an AWS Lambda from NodeJS

In case someone wants to invoke an AWS Lambda from NodeJS (could be from Express WebApp) then we can do so using the AWS SDK for JavaScript in Node.js

The following code can be used to invoke the Lambda

  • Create a function which invokes Lambda

 let AWS = require('aws-sdk');  
 const util = require('util');  
 AWS.config.update({ region: 'us-east-1' });  
 let lambda = new AWS.Lambda();  
 //Promisify the Lambda  
 let promisifiedLambda = util.promisify(lambda.invoke.bind(lambda));  
 /**  
  * This function invokes Lambda  
  * @param {*} functionName Name of the Lambda function  
  * @param {*} payload Payload to be passed onto Lambda  
  */  
 let lambdaInvoker = async (functionName, payload) => {  
   let params = {  
     FunctionName: functionName,  
     Payload: `{ "body" : ${JSON.stringify(payload)} }`  
   }  
   let res;  
   try {  
     //Use this format if Lambds is not promisified  
     res = await lambda.invoke(params).promise();  
     //Use this format if Lambda is promisified  
     //res = await promisifiedLambda(params);  
   } catch (err) {  
     console.error(`err:: ${err}`);  
   }  
   console.log(`Lambda invocation successful`);  
   console.log(res);  
   return res.Payload;  
 }  

  • Invoking the function created above passing Lambda Name and Payload
 //Let's invoke the Lambda  
 let lambdaFunctionName = 'cli-target-lambda';  
 let payload = { name: "abc", age: 20 };
 //Invoke lambdaInvoker using IIFE  
 (async () => {  
   try {  
     let response = await lambdaInvoker(lambdaFunctionName, payload);  
     console.log(response);  
   } catch (err) {  
     console.error(`err:: ${err}`);  
   }  
 })();  

The complete source code can be found here

Saturday, November 14, 2020

Java - Copy project dependency (jars) to a folder using Maven Plugin

If one have to execute a Java program from Command prompt then it would be cumbersome to specify all the required Jar files. It would be very easy to specify the jars if all of them are in a single folder. Using a Maven Plugin we can easily copy all the dependency jars to a specific folder and then execute a Java class easily

  • Use the following Maven plug-in, mention the target folder in <outputDirectory> element, currently the folder is target/jars
 <plugin>  
      <groupId>org.apache.maven.plugins</groupId>  
      <artifactId>maven-dependency-plugin</artifactId>  
      <version>3.1.2</version>  
      <executions>  
           <execution>  
                <id>copy-dependencies</id>  
                <phase>package</phase>  
                <goals>  
                     <goal>copy-dependencies</goal>  
                </goals>  
                <configuration>  
                     <outputDirectory>${project.build.directory}/jars</outputDirectory>  
                     <overWriteReleases>false</overWriteReleases>  
                     <overWriteSnapshots>false</overWriteSnapshots>  
                     <overWriteIfNewer>true</overWriteIfNewer>  
                </configuration>  
           </execution>  
      </executions>  
 </plugin>  
  • Execute the command mvn clean package
  • Above command should have copied all the dependency jars under target/jars folder


  • To run a Java program from the jar use the following command
 java -cp target/<YOUR_PROJECT_JAR>:target/jars/* <class_to_be_run>  
  • For example to execute Java Class DynamoDBDisableStream.java issue this command
 java -cp target/aws-java-0.0.1-SNAPSHOT.jar:target/jars/* com.balatamilmani.awsdemo.dynamodb.DynamoDBDisableStream  

Reference: https://maven.apache.org/plugins/maven-dependency-plugin/examples/copying-project-dependencies.html

Wednesday, November 11, 2020

AWS - Invoke Lambda using CLI

Invoking Lambda using CLI

Let's see how to invoke a Lambda from Command line, the CLI version used in this demonstration is "aws-cli/2.0.57"

  • Create simple Lambda, this is written using NodeJS
 exports.handler = async (event, context) => {  
   const response = {  
     statusCode: 200,  
     body: JSON.stringify('Hello from Lambda!'),  
   };  
   return response;  
 };  
  • Make sure you have setup Client Credential to run CLI commands from your local machine, refer this Blog to setup the same
  • Make sure the Credential you have setup has "AWSLambdaRole" policy assigned which entitle the Credential user to invoke Lambda
  • Execute the following command from Command Prompt
 aws lambda invoke --function-name <LAMBDA_NAME> --payload "{\"name\": \"balatamilmani\"}" --invocation-type RequestResponse --cli-binary-format raw-in-base64-out response.json  
    • --invocation-type -> Value should be "RequestResponse" for Synchronize call
    • --cli-binary-format -> Value should be "raw-in-base64-out" for AWS CLI V2

Saturday, October 31, 2020

AWS - Attaching Lambda Trigger to DynamoDB Table

Attaching an existing Lambda as a DynamoDB Table's Stream trigger so that the Lambda can see the changes as they happen in the DynamoDB Table

Let's see how to attach an existing Lambda as a DynamoDB Table Stream trigger using code. If you disable/enable DynamoDB Stream for any reason the Lambda Trigger would get disconnected and this code would reattach the Lambda trigger back. This code is written in Java

  • Retrieve DynamoDB Table's Stream ARN. 
           String streamArn = null;  
           AmazonDynamoDBStreams dynamoDBStreamsClient = AmazonDynamoDBStreamsClientBuilder.standard().build();  
           // Create request  
           ListStreamsRequest listStreamRequest = new ListStreamsRequest();  
           listStreamRequest.setTableName(dynamoDBTable);  
           // Fetch Streams  
           ListStreamsResult response = dynamoDBStreamsClient.listStreams(listStreamRequest);  
           // Get the StreamARN  
           if (response != null && response.getStreams() != null && response.getStreams().size() > 0) {  
                Stream stream = response.getStreams().get(0);  
                streamArn = stream.getStreamArn();  
           }  
           return streamArn;

When a DynamoDB Table name is passed on to the above code, the 'listStreams' would return only a single Stream ARN, if the Table name is not passed it would return all the Table's Stream ARN.

  • Attach Lambda as a Trigger for DynamoDB Table's Stream. In the below code the 'eventSourceArn' is same as the 'streamArn' we retrieved above
           // create EventSourceMapping Request  
           CreateEventSourceMappingRequest createEventSourceMappingRequest = new CreateEventSourceMappingRequest();  
           // Set the Trigger enabled  
           createEventSourceMappingRequest.setEnabled(true);  
           // Set the Source of the Stream, in our case DynamoDB  
           createEventSourceMappingRequest.setEventSourceArn(eventSourceArn);  
           // Set the Starting position of the Stream to be consumed  
           createEventSourceMappingRequest.setStartingPosition("LATEST");  
           // Name of the Lambda function that gets triggered  
           createEventSourceMappingRequest.setFunctionName(functionName);  
           // Create Lambda Client  
           AWSLambda lambdaClient = AWSLambdaClientBuilder.standard().build();  
           // Execute SourceMapping request  
           CreateEventSourceMappingResult createEventSourceMappingResult = lambdaClient  
                     .createEventSourceMapping(createEventSourceMappingRequest);

Complete source code can be found here

Saturday, October 24, 2020

AWS - Java based client using different CredentialsProvider

AWS Java client using different CredentialsProvider

Credentials to a AWS Java client can be passed in multiple ways. Let's see a few of the common ways of building Java client sourcing Credentials from different sources. In this Blog building a Lambda Client is demonstrated, however the same way any Client can be built (such as AmazonDynamoDB etc.)

  • Credentials from ~/.aws/credentials file. Refer to this Blog for setting up Credentials

AWSLambda awsLambda = AWSLambdaClientBuilder.defaultClient();  

  • Credentials from ~/.aws/credentials file using Profile name

String profileName = "lambdaUser";  
AWSCredentialsProvider credentialsProvider = new ProfileCredentialsProvider(profileName);  
AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider).build();  

  • Credentials from a properties file in the Classpath. By default AwsCredentials.properties file from Classpath is loaded, this can be overridden. 

 // Create a CredentilsProvider object using the AwsCredentials.properties in the classpath  
 AWSCredentialsProvider credentialsProvider = new ClasspathPropertiesFileCredentialsProvider();  
 // Build the Client using CredentialsProvider Object  
 AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider).build();  

  • Credentials passed directly in the Program

 // Access and Secret keys of User  
 String accessKey = "<YOUR_ACCESS_KEY>";  
 String secretKey = "<YOUR_SECRET_KEY>";  
 // Build a Credential object  
 AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);  
 // Create a CredentilsProvider object using the Credential object  
 AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);  
 // Build the Client using CredentialsProvider Object  
 AWSLambda awsLambda = AWSLambdaClientBuilder.standard().withCredentials(credentialsProvider)  

Complete source code can be checked out from the Git Repo

Friday, October 23, 2020

AWS - Enabling/Disabling DynamoDB Stream programmatically

How to Enable/Disable AWS DynamoDB Stream programmatically?

It would be useful for testing purposes, through automation code, to enable or disable AWS DynamoDB Streams to Lambda. Let's see how to achieve that.



           boolean enable = false;//false-to disable, true to enable  
           String dynamoDBTable = "TableName";  
           // Create Async client  
           AmazonDynamoDBAsync ddb = AmazonDynamoDBAsyncClientBuilder.defaultClient();  
           // Create StreamSpecification  
           StreamSpecification streamSpecification = new StreamSpecification();  
           streamSpecification.setStreamEnabled(enable);  
           // StreamViewType is needed only for enabling  
           if (enable) {  
                streamSpecification.setStreamViewType(StreamViewType.NEW_AND_OLD_IMAGES);  
           }  
           // Create update request using the StreamSpecification  
           UpdateTableRequest updateTableRequest = new UpdateTableRequest();  
           updateTableRequest.setTableName(dynamoDBTable);  
           updateTableRequest.setStreamSpecification(streamSpecification);  
           // Execute the update request Asynchronously  
           Future<UpdateTableResult> updateResultFuture = ddb.updateTableAsync(updateTableRequest);  
  • Reattach if any Lambda Trigger back to the Stream otherwise any existing Lambda trigger won't work as they got removed during this process, refer to this Blog
Note: The program is not exiting after executing the request, not sure it's due to wrong implementation of the Async API of 'AmazonDynamoDBAsync' code. Please comment if you have a solution to exit from the program gracefully. I ran the program on Java (11.0.8) with AWS DynamoDB SDK (1.11.881) and also using Java 1.8.0_272 as well with no luck

Friday, October 16, 2020

AWS - Setting up client credentials

 Setting up aws Client credentials for locally executing programs and running CLI commands

Creating user with Programmatic access: 

While running programs locally, the program should have access to aws Client credentials so that the Program can use it to access aws resources such as DynamoDB, SQS etc. While creating new User one has to specify the user is being granted 'Programmatic access' so that "access_key" and "secret_key" will be generated. This could be used for aws cli, SDK etc.



 Client credentials:

  • Client credentials goes to a file in the home directory, In Unix based systems it's at ~/.aws and the file name is credentials
  • Sample content of ~/.aws/credentials

[default]
aws_access_key_id=AKIAIOSFEXUSEREXAMPLE
aws_secret_access_key=wJalrKJDLFJ78EMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8JFIOHDFBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9UDFKJh3yCo8nvbEXAMPLEKEY

  • The first line is the profile name within square bracket

aws region configuration:

  • aws region information goes to a file in the same directory (~/.aws) and the file name is config (no extension)
  • Sample content of ~/.aws/config
[default]
region=us-west-2
output=json
[profile user1]
region=us-east-1
output=text
  •  Note the prefix 'profile' before the 'user1' this is needed in config file and it would be simply 'user1' in credentials file
Aws cli assume default profile while running cli command, if you want to use different user profile then mention profile name 

aws dynamodb describe-table --profile user1 --table-name songs

Switch between Java versions on Fedora OS

Switching between multiple versions of Java on a Fedora OS (all *nix based OS)

To install multiple versions of Java refer this Blog

  • Check the current version of Java using the command
 java -version


Current version of java being used is 11.0.8
  • To switch to different installed version, issue the command
 sudo alternatives --config java

enter password if prompted for sudo

All the versions of Java installed currently will be shown as in image below 

Type in the number under "Selection" Column to switch to that version, for example type 1 to switch to java-1.8.0

  • Verify the version of java is changed as selected above

 java -version
 

After switching to a particular Java version (e.g. 11), see how it's linked to 'java' executable



Note: If you have JAVA_HOME variable set in ~/.bash_profile that needs to be removed. Restart is required upon removal 

Thursday, October 15, 2020

Installing Java on Fedora

How to install Java on Fedora?

  • List available versions
dnf search openjdk


  • Choose the version of Java you want to install, say java-11-openjdk-devel.x86_64
  • Run the command
sudo dnf install java-11-openjdk-devel.x86_64

  • Follow the prompts to complete the Installation