CDK - Making CloudFormation easier

 In the previous post I created my first AWS Resource using AWS CloudFormation. This was done using a JSON file called the CloudFormation template.

AWS CloudFormation - my first stack

Before I start - CloudFormation is by far one of the weirdest and coolest things to have come out of AWS. It took me some time to grasp it (and am still grasping) but mann its awesome for AWS users. 

An SNS primer - also Auto Routing SNS messages to different SQS

 It is a standard decoupling pattern to have publishers send their messages to an SNS owned by them. Consumers will setup SQS endpoints that subscribe to SNS and get the messages. However consider that there are many consumers all of whom are interested in different subset of messages sent by the SNS.

Amazon ECS : My first AWS container - Part 2

In the previous post we created a container image for our code and pushed the image onto ECR. In this post, we will setup an ECS application

Amazon ECS : My first AWS container - Part 1

I had done a post sometime back on Container basics and AWS. Here I will explore ECS - another Container offering from AWS

Dynamo Db - locks continued

In the last post I used optimistic locking with the Version attribute provided by DynamoDBMapper. Here I am going to look at the pessimistic locking method

Dynamo Db - locks

When I studies Databases in college, we learnt the concept of locks

A database lock is used to “lock” some data in a database so that only one 
database user/session may update that particular data. So, database locks 
exist to prevent two or more database users from updating the same exact 
piece of data at the same exact time.

The idea was to achieve transactional isolation. Or two operations operating on the same record did not have any side effects on the other.

Dynamo Db - Good To Remember stuff

Post is a collection of interesting pointers on the working of Dynamo Db. I wanted to have them in a single place as a ready reference

Container Deployments - some basic concepts and EKS

I came across AWS's container solutions - EKS. But I realized that to move forward, I actually needed a primer on Containers. The post is a revision on Containerization. I have liberally copied of the Kubernetes Website and from Docker website - as I did my  studies from there.

EC2 and security groups

In the previous post we setup an EC2 instance and tried to connect it using SSH:

ssh -i ~/Desktop/test-burner-pair.pem ec2-54-237-224-254.compute-1.amazonaws.com
ssh: connect to host ec2-54-237-224-254.compute-1.amazonaws.com port 22: 
Operation timed out

My host has a public address, the subnet has a gateway that allows connection to internet, so ideally it should have worked. However there is additional setting with EC2 - security groups

EC2 and networking

In the previous post, we setup an EC2 instance. I wanted to play around with the configurations available here.

AWS EC2 - setting up an instance

Everything that can be written about EC2 has already been written. I however like to make my own short notes and will use this entry as my personal notebook on EC2.

Athena through API

In the last post, we setup tables and database for our s3 data on Athena and queries it through the AWS console. In this post, I will attempt the same thing through Java SDK

Dynamo Db's API Model

While doing a search for Dynamo Db get Item code, I came across several different code samples (obviously) and also realized a not so obvious thing - there are multiple ways to access Dynamo Db through code.

More on DAX

In the previous post we setup a DAX client and tested the GetItem performance. This post will look at other DAX features

AWS Athena

I have been going through use cases where some basic analytics needed to be run on structured logs generated by our system. The way I did it till now, is to spin up an EMR cluster, load my logs on it and execute hive queries.

DAX - speeding/cheapening up Dynamo Db

Amazon DynamoDb comes with its own cache layer DAX or DynamoDB Accelerator. In this post I am going to play around with this feature.

Deploying API Gateway

In the last 2 posts, I have been working on setting up API access to my Dynamo Table through API Gateway. The next step would be to deploy the APIs:

Dynamo Db APIs through API Gateway

In the last post, we setup a GET method for my table through API Gateway. I wanted to go ahead and setup the other methods - DELETE, PUT, POST.

REST + Dynamo = API Gateway

My use case is very straight forward - I have a table and I want to expose CRUD operations on the table - GET, ADD, DELETE, UPDATE.

Playing with ElastiCache - 2

In the previous post we setup a basic Lambda function that wrote to Redis. I updated the function to log some Cloud Watch metrics.

Playing with ElastiCache

I have never used Elastic Cache yet. My caching use cases were solved either via an in memory cache or a hosted cache. I finally got the chance to venture beyond - to cloud caches.

Playing with the Paths of Step functions

AWS Step Functions work by moving from one state to another state. States need a way to communicate with each other - DATA needs to be passed between them

Step Functions - AWS Service Integrations 2

In the previous post we successfully executed dynamo db, sqs and sns interactions directly from step functions. I wanted to try some tweaks to my state machine.

Step Functions - AWS Service Integrations

With all this serverless chatter around Lambda, the next thing was to try and orchestrate these Lambdas with Step Functions. Essentially a Step Function allows you to define the steps to be executed across systems in a visual UI. The interactions, retries, failure conditions and other orchestration behavior (or the flow management) can be done by Step Functions.

Using Lambda To Trigger a State Function

So I took the Hello World Step function example and wanted a Lambda to trigger the execution

SNS SQS and the multi function Lambda

So I had a use case where I needed to listen to a 4/5 different notifications and then setup a Lambda which synced it. I could setup a different Lambda for each SNS notification and then perform the operation on each message type. But I did not want so many of them around - only differing in how they read the message. Instead I chose to setup a single Lamba Function that could process all the varied notification messages.

Triggering Lambda on Schedule

I had a requirement recently where I needed to run a Monitoring code hourly to gather and plot some stats. With monitor duration of 1 second and total executions of 24/day, serverless seemed to be the way to go. However Serverless being an event driven world, I needed some way to trigger my Lambda.

Serverless - Lambda to publish to a stream

In the last post we started with Hello World Lambda - one that was driven by S3 events to write to S3. I wanted to attempt to use Lambdas for transformations. i.e. convert my file data in S3 to a stream.

Serverless - My delayed transition

The tech world has been moving away from servers for quite some time. While I was introduced to Lambda a few years back, I was hesitant to leave what I knew and jump into this new style of development. My initial experiences left me feeling Lambda is expensive, Lambda cannot replace having your own server up all the time, Lambda is for niche use cases etc.

Neo4j - create multiple nodes and connect with graph

In the last post I worked on creating a single node using cypher. There are other things that can be achieved using cypher when it comes to creation.
Consider that I want to add a Movie and its lead actors to the database.

Neo4j - Create a node with Cypher

I decided to use Cypher to create a simple record. Here I tried to create a Movie instance